Robust Joint Instance-Semantic Segmentation for Semantic Enrichment of 3D Roof Reconstruction from Noisy Labels

Abstract. We address the previously unstudied task of roof part instance segmentation in 3D building models, which provides fine-grained semantic and structural information beyond traditional roof surface segmentation. This paper presents a framework that leverages official LoD2 semantic building models and airborne LiDAR point clouds to automatically generate training data for joint semantic and instance segmentation of roof part instances. We introduce a multi-task ConvPoint-based network with bidirectional cross-attention modules for feature sharing, along with a two-stage noise-robust training pipeline designed to mitigate annotation noise and geometric complexity. Experiments on datasets derived from public 3D semantic building models demonstrate that our approach substantially improves segmentation quality under noisy, real-world conditions. The results highlight that progress in automated 3D building reconstruction depends not only on network design but critically on advanced training strategies that can exploit noisy, large-scale semantic building models, providing a reproducible methodology for harnessing public 3D city inventories.