Fine-Registration between Point Clouds and Aerial Images using Monocular Geometry

Abstract. Accurate registration between aerial imagery and LiDAR point clouds is fundamental to building-level analysis and urban modeling. Although coarse alignment can be achieved through geo-referencing or sensor calibration, residual misalignment often remains and affects the reliable interpretation of roof structures. To address this issue, we propose a Progressive Correlation-based Geometric Registration (PCGR) framework for fine registration between projected LiDAR observations and aerial images using monocular geometry. We leverage MoGe-2, a state-of-the-art tool for depth synthesis from optical images, trained on large-scale data to recover a dense depth map from a single aerial patch around the building. Such models encode strong geometric priors and provide a geometrically consistent representation for building structures. Sparse LiDAR points are aligned with the monocular depth through a progressive coarse-to-fine optimization strategy. The alignment is guided by the Pearson correlation coefficient, ensuring robustness to scale and bias differences. We evaluate the method on those images, exhibit significant deviations from the LiDAR point cloud to correct them for our ongoing task, namely, roof detail analysis in 3D. Experimental results demonstrate consistent improvements in alignment quality, including a systematic reduction in height standard deviation and decreased facet-wise RMS residuals in most cases. These findings indicate that large-scale learned monocular geometry effectively bridges the representation gap between aerial imagery and LiDAR for building-level cross-modal registration.