DEVELOPMENT OF MULTI-VIEW STEREO CONSIDERING ACCURACY OF EXTERIOR ORIENTATION ELEMENTS

Abstract. Structure from motion (SfM) has been widely used to achieve automatic 3D reconstructions. However, as the 3D point clouds obtained via SfM are sparse, multi-view stereo (MVS) was developed to compensate for this sparseness. The accuracy of the 3D surface depends on the accuracy of the orientation elements based on the SfM. Additionally, in the case of an unmanned aerial vehicle (UAV), SfM exhibits a decrease in the accuracy of the orientation elements during complex camera movements. This paper proposes a patch-based MVS (PMVS) method considering the accuracy of the orientation elements. The proposed method involves applying the global SfM, estimating accuracy of exterior orientation (EO) elements, and introducing the accuracy of EO elements to PMVS. The PMVS approximates an object surface by using small rectangular patches, namely local tangent plane approximation. The patches are optimized by minimizing the sum of the photometric discrepancy scores. The accuracy of the EO elements is introduced to the patch optimization as weighting function. This accuracy is defined using the variances of the estimated parameters in the bundle adjustment. We also investigate the types of weighting functions. The results indicate that the proposed method is capable of considering geometric conditions during patch estimation. The proposed method was applied to the three types of image datasets, i.e., images captured using an SLR camera at ground level, images captured using a UAV equipped with a SLR camera, and images captured using an airplane equipped with an oblique camera. Through the experimental results, the improved accuracy and the effectiveness of the proposed method were confirmed.