3D RECONSTRUCTION OF SIMPLE BUILDINGS FROM POINT CLOUDS USING NEURAL NETWORKS WITH CONTINUOUS CONVOLUTIONS (CONVPOINT)

Abstract. The automatic reconstruction of 3D building models from airborne laser scanning point clouds or aerial imagery data in a model-driven fashion most often consists of a recognition of standardized building primitives with typically rectangular footprints and parameterized roof shapes based on a pre-defined collection, and a parameter estimation so that the selected primitives best fit the input data. For more complex buildings that consist of multiple parts, several such primitives need to be combined. This paper focuses on the reconstruction of such simple buildings, and explores the potential of Deep Learning by presenting a neural network architecture that takes a 3D point cloud of a single building as input and outputs the geometric information needed to construct a 3D building model in half-space representation with up to four roof faces like saddleback, hip, and pyramid roof. The proposed neural network architecture consists of a roof face segmentation module implemented with continuous convolutions as used in ConvPoint, which performs feature extraction directly from a set of 3D points, and four PointNet modules that predict from sampled subsets of the feature-enriched points the presence of four roof faces and their slopes. Trained with the RoofN3D dataset, which contains roof point segmentations and geometric information for 3D reconstruction purposes for about 118,000 simple buildings, the neural network achieves a performance of about 80% intersection over union (IoU) for roof face segmentation, 1.8° mean absolute error (MAE) for roof slope angles, and 95% overall accuracy (OA) for predicting the presence of faces.