DIMENSIONAL DISCOVERIES: UNVEILING THE POTENTIAL OF 3D HERITAGE POINT CLOUDS WITH A ROBUST ONTOLOGY FRAMEWORK

: 3D point clouds feature valuable geometric and, often, radiometric and semantic information to support studies, analyses and understanding of the surveyed scene. Due to their generally large size, the use and interpretation of point clouds could be problematic. User-friendly and quick approaches for querying these valuable datasets and retrieving information could surely support end-users, in particular in the heritage sector. This work presents an ontology-based approach to facilitate the query and use of 3D heritage point clouds by means of sets of rules in order to infer properties and characteristics of the surveyed scene. Our approach is focused on linking together 3D spatial data and expert knowledge, in a way that the ontology can elaborate, represent, enrich and query a given point cloud. Results show how different queries can be set-up and how the procedure can be replicated to various queries and datasets.


INTRODUCTION
The preservation, conservation and restoration of historical buildings and monuments require 3D documentation and a diagnostic analysis normally carried out by an interdisciplinary team (Stylianidis and Remondino, 2016).These operations produce a large variety of heterogeneous information (3D geometrical data, 2D restitutions, on-site sketches, textual material, visible and thermal images, etc.) and their integration into a unique information model to support operations is still an open issue in the Heritage community (Ramos and Remondino, 2015;Lin et al., 2019;Adamopoulos and Rinaudo, 2021;Patrucco et al., 2022).Considering only 3D data, their processing, understanding and visualization is often time-consuming and highly demanding in terms of hardware specifications.In the case of Digital Heritage, it is particularly important to develop methods designed to handle 3D data in an efficient, fast, reliable, and easily accessible way.In particular, the improvement of user understanding and accessibility is a much-needed step to allow specialists in the Heritage field to utilise at its full potentiality the (geometric, texture and semantic) information stored inside the data.Nowadays, a significant amount of data in the Heritage field comes in the form of point clouds.These kinds of data have to be processed to transform them from raw (geometric) data into more useful and meaningful information.The semantic enrichment of heritage point clouds is realized segmenting them into meaningful classes which are often scene-and project-dependent.A substantial number of methods were developed to perform 3D heritage classification: supervised methods were normally presented (Mazzacca et al. 2022;Moyano et al., 2021;Grilli et al. 2020;Matrone et al., 2020;Teruggi et al. 2020;Murtiyoso and Grussenmeyer, 2019;Grilli and Remondino, 2019;Poux et al., 2017) whereas unsupervised methods in 3D heritage are still in primordial phase.The usage and query of these semantically enriched 3D data is still problematic and the accessibility to these data is challenging in the presence of large datasets and many classes.An ontology-based approach could help solve these issues, leveraging the full value of the semantic enrichment, and more in general of the data's stored information, by linking classes and properties to 3D data and deducting their relationships with a series of inferential rules (Nicolucci and D'Andrea 2006; Amico and Felicetti, 2021;Casillo et al., 2023).Existing applications of formal ontologies to the heritage field are oriented to general knowledge representation or to process low-level features to perform semantic classification.In the second case, ontologies are used as a tool to link low-level features to correspondent classes, explicitly defining the latter as an individual's possession of a certain set of the former, or to infer the presence and the type of macro individuals from low-level features of the dataset (Hmida et al., 2012).There is no usage of ontologies as having single points as individuals and as being oriented towards semantic enrichment and querying more than recognition and classification.

Paper aims
In this paper, we aim to implement an ontology to facilitate the query of 3D heritage point clouds in a quick and effective manner (Figure 1).To do so we embed an ontology with a set of rules that, starting from a classified point cloud (classes and attributes), can infer transversal characteristics and asses them with simple queries in an accessible, fast and user-friendly way.We provide such ontology with explicit definitions of high-level features (with respect to low-level features) which include also the semantic machine learning-driven classification results and elaborate properties such as the dispositional ones.Besides this nesting operation of high-level features, we also provide the ontology with some expert knowledge about, for example, materials' physical-chemical properties.This knowledge can stand by geometrical, spatial and chromatic information as well as the results of a machinelearning-driven 3D classification process, in order to nest even more complex high-level features.Some works focused more on recognition and classification than on semantic enrichment and querying.These approaches still differ radically from what is presented in this paper, since they never directly manage points and they diverge in their very purpose.

Ontology definition
In computer science, ontologies are means to explicitly specify certain concepts.Therefore, an ontology can be seen as a common denomination and formal definition of properties and interrelations of characteristics that exist for a particular domain (Doerr et al., 2003).Formal ontology was born as a branch of analytic philosophy but has soon found its applied role within the project of the semantic web and as a useful tool for organizing data with a knowledge-based approach (Grimm, 2009).A formal ontology, whose language standard is RDF/XML, contains specifications about hierarchy-organized classes and properties, accompanied by a set of defined inferential rules (Hmida et al, 2012) whose purpose is to allow deductions about the properties and the classes related to the individuals contained in the ontology.The individuals are, within the ontology, the singular instances of the classes and the actual bearers of the properties.The more elaborated are the class hierarchical tree and the interlacing of the properties performed by the rules, the more expert and useful is the knowledge base contained in the ontology.Properties, for their part, are not conceived as Python-like attributes, specifically related to a certain class, but are instead represented as independent entities that can apply transversally to individuals of more than one class.The core concepts behind the usefulness of formal ontologies are • the high human readability and controllability of such organized data: each property and each class are explicitly defined, the inferential reasoner that performs rule-driven deduction can return justifications to its inferences and each rule is explicitly defined in the form of a logic conditional, with a set of premises that, if true, lead to a certain conclusion; • the highly expert knowledge that can be represented within the ontology: high-level features can be embedded in lowerlevel ones by means of dedicated rules and can also be bounded to certain classes so that individuals can easily inherit the properties of the classes they belong to; • the high conceptual richness that can be expressed in the queries: while consulting an ontology, the user can rely both on a dedicated and highly expressive language (SPARQL) and on all the high-level properties specified in the ontology and assigned to the individuals just by means of inferential rules -as further specified in Section 3.2.

Ontology for 3D heritage data
Traditionally, ontologies have been understood as solutions to model a certain knowledge domain.Thus, they have often been linked to general abstract knowledge, not directly ascribed to an actual individual with spatial consistency.This approach made ontologies very oriented toward aspects of interoperability, exchanging of information and concepts' mapping, but generally using 2D data (Pattuelli, 2011;Cornevilli et al., 2020;Ranigar, 2022).This is quite evident in the fact that usually ontologies consider "individuals" the concepts situated at the end of the hierarchical tree.In such a conception, for example, "materials" can be a class and "marble" an individual contained in that class.Note that this approach excludes the possibility of a direct elaboration and representation of the 3D data based on the ontology itself.
Ontologies were also used to support the conservation and management of cultural heritage.One of the first ontology usage applied to 3D heritage is presented in Nicolucci and D'Andrea (2006) where the original ontologies' spirit is used, aiming at a purely conceptual semantic mapping.Files containing 3D models are taken as individuals without any representation of their contents, leading to a lack of spatial consistency, querying possibilities or geometrical or pointlevel semantic enrichment.Inspired by Messaoudi et al. (2017), Figure 2 shows a clean and easy representation of the state-of-the-art of ontologybased information systems for heritage monitoring.It is composed on four main components: 1. an automated reality-based 3D digitization pipeline (Remondino et al., 2013); 2. a hybrid (2D/3D) semantic enrichment process (Grilli and Remondino, 2019;Matrone et al., 2020) 3. a domain ontology describing knowledge related to degradation phenomena; 4. a query engine.Cacciotti et al. (2014) focus on step 3, offering a causally oriented ontology of heritage degradations comprising events or processes that can cause the former.Their work lacks spatial consistency for the individuals, since they are thought as the single damage instances.Therefore, it loses the possibility to query for points and to manage a 3D model of the dataset.Nespeca et al. (2016) is focused on steps 1 and 2. In their work, the semantic information is mainly regarding geometrical midlevel features, focusing on a few geometrical properties and their attribution, never directing to complex nested properties nor to a seriously query-oriented task (point 4).Regardless, their understanding of ontologies is very similar to the presented work in terms of individuals' conception.Their ontology, just like ours, directly manages points as individuals, allowing spatial consistency in the organized domain.In Messaoudi et al. (2017) an ontology is used to link spatialized regions of a point cloud to high-level semantic features and to build a semantic data structure about expert knowledge.The approach is more oriented towards in situ data acquisitions management than for large point clouds' elaboration, representation, enrichment and querying.Yet, the approach requires an expert actor's intervention to manually create the individuals related to the chosen spatial regions.Such a model, even if it takes a huge step forward in the usage of ontologies to keep together spatially consistent point clouds and rich semantic knowledge, depends very much on the direct intervention of the human actor and it can hardly be upscaled.Acierno et al. (2017) presented an information model aimed at supporting the representation and management of knowledge for the architectural heritage conservation processes (step 3).The knowledge base has been connected with a building information modelling environment, providing an effective integration between geometrical and non-geometrical information.
Recently Peng et al. (2023) presented OpenScene3D, a 3D scene understanding tool with open vocabulary.OpenScene does not actually rely on an ontology at all, but it shares with our work the semantically rich query task.Here, the semantic is built with a text/image-embedding process that associates to every word a certain vector in a vector space, in which the closest the distance between two vectors, the closest the semantic distance between the associated words.OpenScene relies not only on a point cloud but also on associated 2D images from various perspectives.It seems that this aspect could constitute a problem when it comes to dealing with an effective upscaling.Even if the open vocabulary feature is definitely impressive, the approach lacks explicit definitions of the relationships between properties, an implementation of an expert knowledge base and the possibility to structure complex queries with SPARQL (2024).Furthermore, query results differ on the fact that OpenScene returns for each point a certain value of a "matching index", thus not returning boolean answers like the work hereafter presented.
Figure 2: Graphical representation of the four steps/components taken as a general scheme and the relative dedicated tools that operates (or will operate) in our work.

Ontology preparation
An important aspect of formal ontologies is their strong context dependence.If, on the one hand, this feature makes this tool very specific and deficient of versatility, on the other hand, it allows an accurate and exhaustive mapping of the expert knowledge related to the specified context.For this reason, the very first step for preparing an ontology is the study of the knowledge that must be represented in it.Once done, the class hierarchical tree must be created, considering all the classes needed and their subclass, superclass and sibling class relations.Protegé Ontology Editor ( 2024) is a very useful tool to do this without manually modifying the ontology file, relying on a highly user-friendly interface.The second step is the definition of properties, which requires the specification of their domain (the classes whose individuals can be bearers of the considered properties) and their range (the types of values that the property can assume or the classes whose individuals can be the second term of the relation).The definition of the properties is usually extrinsic and specified during the third step, i.e., the writing of the inferential rules.Properties are assigned to the individuals by means of premises-conclusion structured rules so that a property's "meaning" can be seen as defined by the set of premises that leads to a conclusion of a "P(x,v)" form, where P(x) is the predication of the "v" value of the "P" property to the "x" individual.As far as the classes' meaning concerns, the matter is slightly different.The user can choose between a rule-based definition (analogous to the one just described for the properties) and a class expression-based one, in which the classes are intrinsically defined with the same premisesconclusion structure.Once the structure is so completed, the ontology can be populated with individuals.Some of their lowlevel properties should be asserted from the outside so that the inferential reasoner can perform its deductions, split the individuals into the pertaining classes, and assign values to the deducted higher-level properties.

Query procedure
Query procedures rely on a highly expressive language named SPARQL (2024).It allows to express and combine all the basic first-order logic operators, to refer to any class, property or individual defined in the ontology and to perform comparisons, filtering and similar operations on the property's values.This possibility, combined with the semantic and conceptual enrichment performed by the ontology on the raw input data, allows complex queries that were not possible before this processing.This feature can be proficiently illustrated by means of an example.A point cloud (las) file contains a certain number of fields, including, besides spatial coordinates and RGB values, information about materials (e.g.bricks, marble, etc.), building techniques (e.g.plaster, opus latericium, etc.) and some different kinds of degradations (e.g.efflorescence, black crust, etc.).Queries performed directly on this file could just refer to this information ("find points classified as marble").A possible way to enrich queries functionalities could be to add additional columns to the (las) file, but: (i) it would require new inferences for the data; (ii) the high-level feature obtained in such a way wouldn't be inferentially expressed on the basis of explicitly defined and easily modifiable rules, leading to a loss in terms of controllability both in the justification and in of controllability both in the justification and in the modification aspects; (iii) the obtained (las) file would have a highly increased size due to more stored information.On the other hand, within the proposed ontology-based approach, the input point cloud can contain just the low-level features, while the higher-level ones are generated "on-the-fly" within the ontology once the query is performed.Note that the ontology file is never meant to be sharing-oriented, since the output of the query, if the user needs to share it, can be a (las) file containing just the needed information, with the possibility to exclude all the ones used for the query.Imagine a scenario in which every material-related class bears properties about its physical and chemical features.A possible query could aim to look for the points that "present an x value of porosity, where x is greater than y".Within the ontology, every material could be associated with a certain value of porosity, so that the query could return an accurate result, but neither the input (las) file nor the possible output one would have to sustain the informational burden of containing such data for each point.
In the present study, the query system is associated with a 3D visualization tool -relying on the Vedo python module (Vedo, 2024).In this way, the analysed point cloud is visualized with its RGB colours and the overlapped query results to emphasize the user accessibility aspect to which the formal ontology itself is devoted.

EXPERIMENTS AND RESULTS
The application of the developed ontology framework for querying heritage point clouds is based on the following steps: 1. read the point cloud, importing geometric/radiometric information and other attributes ("classes") into the ontology; 2. run the Pellet reasoner (Sirin et al., 2007) to draw all inferences; 3. plot the point cloud using the Vedo 3D scene; 4. asks the user for a query in SPARQL; 5. highlight the results of the query in the Vedo 3D scene; 6. allows the user to export the query result as a separate point cloud.Steps 4,5,6 can be executed multiple times with different queries without points 1,2,3 having to be repeated.Given the created ontology procedure and a semantic enriched point cloud, specific queries can be performed e.g., to visualize where a class is located.Figure 4 reports shows a point cloud of a portico (Remondino et al., 2016;Grilli and Remondino, 2019) semantically segmented in Materials, Building techniques and Degradations.Materials' classes can be queried and visualized (Figure 4c-f).Starting from these visualization capabilities and the covariance features (Weinmann et al., 2013;Farella et al., 2019) extracted for the classification process, high-level features' inferring possibilities can be added by: 1. writing inferential rules to compute a certain value (from a defining, into the ontology data, properties representing materials' physical-chemical properties (e.g.porosity of materials, solubility of their chemical components, etc.); 2. scale to one to five) of each property for every point, with respect to their material class; 3. defining a set of high-level features, computed "on-thefly" exploiting covariance features (e.g.verticality, planarity, etc.) representing characteristics useful for restoration or preservation purposes.As an example, let's consider the "degree of risk for a surface of being affected by leaching" (on a scale from zero to five): to be computed, surface points' values of porosity, solubility and verticality, are used and a leaching risk index (LRI) is derived (Figure 5).The inferential rule can be made far more complex, considering more materials and variables, and mapping more complex information.Figure 6 shows the fruitful aspect of a potential combination of high-level properties' semantic definitions and SPARQL's syntactic richness, where the structured grammar of the latter can dispose of the rich vocabulary of the former.
Besides querying and visualizing high-level features, the proposed approach can inquiry "on-the-fly" other characteristics, e.g. the extension of surface retrieved by a query.In Table 1  In the example given in Figure 7 the retrieval of the SELECT command is a value that represents the extension of the surface calculated based on the number of points that satisfy the conditions expressed in the WHERE section.This is possible since we know the average distance of the points in the point cloud from their neighbours.In this way, we can query for spatial features without defining concepts like "surface" inside the ontology, though this could have been another viable option.In the provided example, the value of the surface is obtained by multiplying the number of the points for an empirical "unity surface value" factor, resulting from operations on a simulated triangular mesh.
Figure 7: Code example to perform a query to retrieve how much surface is "affected" by a particular characteristic (Table 1).A very similar approach is presented Nespeca et al. (2016).The idea of calculating the value without generating a mesh, but just multiplying the number of the points for the "unity surface value", is basically the same between the two works.
Another example is presented in Figure 7 and Figure 8.Using the available semantic classification building techniques and degradations) and covariance features (Figure 7), high-level features and indexes (e.g.Lacuna, meaning missing parts that compromise the integrity of an architectural surface) can be derived as visualized in Figure 8.

CONCLUSIONS
We have presented the general features of the developed ontology-based framework to query classified point clouds and visualize the results.To sum up, the core strengths of the proposed method are: 1.The high human readability of data and the high controllability of high-level features' definitions; 2. The highly expert knowledge that can be ingested within the ontology to perform complex queries; 3. The high conceptual richness that can be expressed in the queries; 4. The strong spatial consistency of our ontology's individuals.Ontologies can answer the complexity of point clouds by organizing, enriching and querying them in a human-friendly way, basically following the common idea that the more richness of information we want to capture, the more syntactically structured and semantically rich our language needs to be and the more the need of ontological order in our referential domain increases.The aforementioned semantic richness is mapped on expert knowledge of the specific field so that queries can be performed as reflecting the actual, practical, and specifical needs of the user.This user-friendly aspect is further enhanced by a visualization tool, to make the user intuitively understand the results of the queries.The proposed framework can keep together the 3D spatial consistency of a point cloud and the conceptual semantic richness of the traditional ontology usage.In the present work, our efforts were mainly focused on a pointbased approach to the application of formal ontologies to point clouds.This approach is useful as it allows very accurate query results and surface-oriented or material-oriented queries.At the same time, our approach is also very intuitive, given that what we are working on are actual point clouds.
In the future we want to move towards an object-based approach since it allows to work in the ontology with macro-object.Therefore, the benefit is the possibility to perform queries not on points but on objects.The most intuitive feature is the possibility to count them, while the most potentially fruitful one is the opportunity to express all sorts of relations between objects and to query them.An object-based approach could benefit from the work of Cacciotti et al. (2014), especially regarding the causal modelization.Another future direction concerns the interoperability features of formal ontologies.Ontologies are often used to map concepts in an easily shareable way and, even if we didn't focus on this feature until now, we think that our work can be further enhanced in this direction.To stress the interoperability features we won't create a domain ontology from scratch, but we will consider importing one or more already existing and standard ontologies.

Figure 3 :
Figure 3: Graphical representation of the process query 3D heritage point clouds using an ontology.

Figure 4 :Figure 5 :
Figure 4: Point cloud of a portico its semantic segmentation in four materials'classes (b); query results to visualize bricks (c), plaster (d), stone (e) and metal (f).

Figure 6 :
Figure 6: Results of the query asking for brick points with a LRI >3 (i.e.not yet but potentially affected).

Figure 7 :Figure 8 :
Figure 7: Original RGB point cloud of the archaeological wall covariance feature "surface variation" useful for 3D classification and ontology-based queries (b); semantic of surface materials (c) and building techniques (d).
some results are provided.