Roof Geometrical Component Extraction Using Bimodal Data and Graph Neural Network

Abstract. Accurate extraction of roof geometrical elements is essential for creating 3D building models, which play a critical role in urban planning, city management, infrastructure development, and disaster management. Roof geometrical elements consist of lines, which represent the intersections of roof planes, and vertices, which define the intersections of roof lines. Due to the presence of shadowed areas or poor contrast in optical images, roof geometrical elements cannot be extracted efficiently in all areas. This study proposes a novel framework using optical imagery and Digital Surface Models (DSM) to extract these elements and construct 3D building models. The proposed approach uses convolutional neural networks (CNNs) to extract roof features from both RGB and DSM data. Next, a graph-based methodology is employed to create roof models, where roof lines and vertices are represented as nodes, and their spatial relationships are captured through an adjacency matrix. Finally, a Graph Neural Network (GNN) is used to analyze these relationships and refine roof component connectivity. In the first stage, the framework was evaluated on a dataset comprising 1,300 buildings in Fredericton, New Brunswick, achieving an Intersection over Union (IoU) of 0.73, an F1-score of 0.7645, and an F2-score of 0.7641. The mAP results of the second stage, 28.3, demonstrate the effectiveness of a graph-based approach in extracting and reconstructing roof components, contributing to more accurate and automated 3D city modeling.