Perspective-n-Point in Practice: Performance, Robustness, and Accuracy for Mesh-Based Localisation

Abstract. Visual localisation, the task of determining camera poses from images, has matured significantly, offering various solutions for handheld device localisation. This paper investigates the Perspective-n-Point (PnP) problem, a crucial step in visual localisation that is often underexplored in practical applications. We evaluate the performance of state-of-the-art PnP algorithms with real-world data, analysing their impact on localisation accuracy and robustness. Using a dataset comprising a large-scale aerial mesh and smartphone images, we conduct experiments to assess PnP algorithm performance. Specifically, we examine the effects of PnP algorithms in isolation, followed by the incorporation of RANSAC for outlier rejection, and finally, the addition of non linear pose refinement. By maintaining a fixed set of 2D-3D correspondences, this approach allows us to: assess the true outlier rejection capabilities of PnP algorithms, quantify the accuracy improvement achievable with non linear pose refinement, and identify superior PnP algorithms for robust visual localisation.