Integrating Ontologies and Object Compositional Hierarchies for Dynamic Semantic Segmentation of Point Clouds

Abstract. While there is a continuous progress on the 3D classification of point clouds, no effort has been seen in in maximizing the amount of semantic information which can be extracted from a single classification operation. This line of research is indeed complementary to the traditional semantic segmentation, enhancing its native outputs with increased flexibility and semantic structuring. Following this direction, this paper proposes a Dynamic Semantic Segmentation (DSS), an approach that lets a single classified 3D point cloud be visualised at multiple semantic granularities instead of at a fixed one. The method consists of three interlocking ideas. First, each point is allowed to carry several distinct labels at once, supporting its belonging to multiple object instances. Second, the spatial overlaps between these co-occurring labels are exploited to establish part-whole (mereological) relationships between object instances, yielding a compositional reading of the scene. Third, the labels are organised into a taxonomy and extended to their ancestors, yielding a conceptual reading. In this way, the semantic information needed at every scale is present simultaneously rather than committed to at classification time. The proposed approach builds upon the 3DGraph format and the 3DOnt framework, using an RDF backbone ontology to encode the taxonomy. Results show the potential of the method and its replicability to other scales and scenarios. Because all labels are pre-computed, each segmentation is produced within few seconds on a point cloud with some million points, adding expressive power over the static counterpart at no cost. Further information and visual results about the 3DOnt framework and DSS are available at: https://3dom.fbk.eu/projects/3DOnt.