2D TO 3D LABEL PROPAGATION FOR THE SEMANTIC SEGMENTATION OF HERITAGE BUILDING POINT CLOUDS

Abstract. During the last decade, the use of semantic models of 3D buildings and structures kept growing, fostered in particular by the spread of Building Information Models (BIMs), becoming quite popular in several civil engineering and geomatics applications. Nevertheless, semantic model production usually requires quite a lot of human interaction, which may result in quite long and annoying procedures for human operators. The production of 3D semantic models of buildings often takes advantage of already available 3D reconstructions of the considered objects. Given the ever increasing resolution of 3D reconstructions, obtained thanks to the recently developed laser scanners and photogrammetric software, the availability of tools for supporting the automatic or semi-automatic generation of semantic models represents a key step for easing and speeding up the process of semantic model production. In particular, the correct semantic interpretation of the different parts of a 3D point cloud, can be seen as the basic step for the production of a BIM model. The most frequently used methods for point cloud semantic segmentation can be separated in two categories: those directly segmenting the point clouds and those based on the ancillary semantic segmentation of images representing the object of interest, then transferring back the segmentation results to the point cloud. This work focuses on the latter method, considering more specifically the application of heritage building semantic segmentation. To be more specific, this paper investigates the semantic segmentation performance on a set of four heritage buildings, obtained first applying deep-learning based image semantic segmentation and then propagating back the semantic information to the point cloud by means of a voting strategy. The obtained results are quite encouraging, motivating future investigations on improvements of this strategy, in particular when including more buildings in the considered dataset.