SINGLE BUILDING POINT CLOUD SEGMENTATION: TOWARDS URBAN DATA MODELING AND MANAGEMENT

: To manage urban areas, a key step is the development of a geometric survey and its subsequent analysis and processing in order to provide useful information, and to become a good basis for urban modeling. Surveys of urban areas can be developed with various technologies, such as Aerial Laser Scanning, Unmanned Aerial Systems photogrammetry, and Mobile Mapping Systems. To make the resulting point clouds useful for subsequent steps, it is necessary to segment them into classes representing urban elements. On the other hand, there are 2D land representations that provide a variety of information related to the elements in the urban environment, which are linked to databases that have information content related to them. In this context, the element identified as interesting for urban management of the built heritage is the individual building unit. This paper presents an automated method for using map datasets to segment individual building units on a point cloud of an urban area. A unique number is then assigned to the segmented points, linking them directly to the corresponding element in the map database. The resulting point cloud thus becomes a container of the information in the map database, and a basis for possible city modeling. The method was successfully tested on the historic city of Sabbioneta (northern Italy), using two point clouds, one obtained through the use of a Mobile Mapping System and one obtained with Unmanned Aerial System photogrammetry. Two cartographic databases were used, one opensource (OpenStreetMap) and one provided by the regional authorities (regional cartographic database).


INTRODUCTION
Improved Geomatics techniques nowadays allow the acquisition of a large amount of georeferenced data of known precision.In urban areas, the most widely used surveying techniques include Aerial Laser Scanning (ALS), Unmanned Aerial Systems (UAS), Mobile Mapping Systems (MMS), remote sensing.In particular, there has been much development of technologies that, in different ways, lead to a very specific geographic data: the point cloud.Aerial, drone, or ground photogrammetry, vehiclemounted LiDAR systems, and the integration of the two techniques, combined with Inertial Measurement Unit (IMU) and Global Navigation Satellite System (GNSS) receivers, make it possible to have a point description of urban surfaces, eventually with color, intensity and other attribute data.Naturally, each method has advantages and disadvantages, different accuracies, and various fields of use.Just consider the data collected through MMS, which essentially concerns horizontal surfaces (e.g., roads) and vertical surfaces (e.g., building façades) in contrast to acquisition from UAS that allows measurement (and investigation) of roofs in addition to other horizontal surfaces.
These data, collected in the urban environment, have great importance for the management of urban areas in different domains: planning, energy sustainability, reuse, heritage management, etc.Furthermore, data management is also diversifying into new approaches.In addition to Geographic Information Systems (GIS), established tools for land management, and 3D city models, we also find digital twins for smart cities (Wang et al. 2023) and City Information Modeling (CIM), which apply the Building Information Modeling (BIM) logic to the urban built environment and its infrastructure (D.A.S.Y., M., 2023).An additional approach is also emerging that foresees point clouds with a central role, theorizing to link all data not to a geometric model (here intended as built from solids and surfaces, regardless of its genesis) but to point model (Wegen et al. 2022;Nys et al. 2021).Hence, the modeling phase is bypassed, and the focus is mainly on information linkage and data extraction.
On the other hand, there is a large amount of data, organized already through 2D representations and databases, where it is possible to find information about the built urban context and its infrastructures.For example, in the Italian panorama, it is possible to collect information on the built environment through Regional Topographic Database (DBTR), cadastral maps, ISTAT archives, consumer society, etc.Hence, it is possible to obtain heterogeneous data that can be used in various fields.The data on different databases are not always directly integrable, and it is often necessary to identify methodologies to perform a fusion of the data present on them (Pasquinelli et al. 2019).
For optimal development of CIMs and their information content, and more generically for an optimized management of urban data, it is necessary to catalogue the information, select the necessary ones and verify their consistency and quality.Therefore, the correct identification of the elements that constitute the urban environment on the point cloud is a preparatory operation for data integration.Indeed, this allows to subdivide the point cloud referring to each unit and link it, later, to the most useful information in the different domains.There are several methodologies that allow the segmentation of the urban point cloud into elements (e.g., streets, buildings, street furniture, vegetation).However, in the perspective of intending to work on the urban built environment, it becomes necessary not only to segment the class "buildings," but it is fundamental to separate one building from another, even in the point cloud.Some authors have dealt with the correct segmentation of individual buildings on point clouds, each identified as a single instance.Hrutka et al. ( 2022) processed a UAS photogrammetry point cloud to identify single buildings, and to generate building footprints to update old and inaccurate large scale maps.Similarly, Du et al. (2019) presented a building contouring algorithm from large-scale ALS point clouds, which is not sensitive to the quality of point clouds and is capable of handling complex building shapes.Xia, S. and Wang, R. ( 2018), instead, proposed a method to localize buildings in dense and complex residential areas with high accuracy by processing a MMS point cloud.In such cases, the developed methods were based on a direct processing of the point cloud without exploiting preexisting 2D cartographic dataset, and the results could be used to generate building footprints from the point cloud itself.
When available, it should be interesting to directly use already processed and reliable 2D data representations.In this paper we want to identify single buildings on the point cloud by exploiting 2D cartographic data.Such identification becomes useful for a dual purpose, a basis for modeling in CIM, but also the possibility of using the segmented point cloud as a repository for information content, using the building as the linking element.This paper presents a procedure intended as part of a larger project that aims to propose solutions for urban management based on three-dimensional geo-referenced data.The method presented here segments a point cloud of an urban environment, regardless of how it is obtained, into the different units (single buildings) that constitute the surveyed area.The initial data are two 2D representations of the territory built according to a very well-defined method (DBTR of the Lombardy Region and OpenStreetMap dataset), and a georeferenced point cloud of the urban area (using point clouds produced through both MMS and UAS photogrammetry).
The paper is organized as follows: Section 2 presents the case study, the surveys done using a MMS and UAS photogrammetry, the cartographic datasets used, and the method implemented in this paper; Section 3 presents the results of the tests made on the case study; in Section 4 a discussion takes place; and in Section 5 conclusions are presented.

MATERIALS AND METHODS
The automated method presented in this paper allows for the segmentation of an urban point cloud into single building units, exploiting 2D cartographic datasets.Building footprints are retrieved from cartographic layers, polygons are adjusted and an offset is made to cope with the cartography accuracy, and then they are used to segment the input point cloud.The workflow is recalled in Figure 1.

Case study description
The case study selected for testing the method presented in this paper is a historic city, Sabbioneta, located in Italy, in Lombardy Region.The city has been inscribed on the UNESCO World Sabbioneta was built in the second half of 1500 on a pre-existing medieval village.The city has a chessboard scheme, reminiscing the Roman founding city, and resulting from the Renaissance discussion on the Ideal city.The chessboard shape clashes with the hexagonal shape of the fortified walls and is composed of 34 building blocks whose morphological structure has remained unchanged over time (Lorenzi, 2020).Blocks are fairly regular, rectangular or square in shape (from 40 to 90metres), with a predominant east-west orientation.Blocks near the fortified walls break up and are lost in a belt of irregular vegetation.Buildings have generally two or three floors, and they are place very close or joined one to the other to form a block.The internal organization of the blocks includes, in most cases, a division into two equal parts parallel to the longer side, so that the houses can be arranged in two opposing rows each facing one of the main streets.

Point cloud datasets
Sabbioneta was surveyed both with a MMS and with a UAS.The first survey was conducted with the MMS Leica Pesagus:Two, mounted on a car and driven all around the city.Almost the entire road network of the city was scanned, roughly 7 km, for 1.2 billion points (6 GB).The point cloud was coloured using images taken by the camera on-board the system.
The second point cloud used for the tests was obtained through drone photogrammetry.The UAS dji Matrice 300 coupled with a Zenmuse P1 camera was used to take photographs of the whole city, and a photogrammetric workflow was implemented to obtain a dense point cloud.The resulting point cloud was composed of 12 billion points (22 GB), provided with color attribute.

Retrieval of cartographic datasets
The method presented here exploits existing cartographic datasets to identify and segment the point cloud.To do so, it is necessary to identify which cartographic dataset are suitable and to select the correct layers.In this paper, we have selected two data sources: the DBTR, as a verified datum provided by the regional administrations; and OpenStreetMap (OSM) dataset, as an open-source basis.In both cases, the interesting layer is the ones representing the buildings of the city.
The DBTR is downloaded from the Lombardy Region web geoportal (www.geoportale.regione.lombardia.it).It contains several digital data representing ground topographic objects, including -among others-roads, railways, buildings, rivers, water basins, infrastructures, and vegetation.Urban areas are represented with a scale from 1:1000 to 1:2000.The DBTR is organized into several layers and classes; among them, there are also some related to buildings (e.g., "Volumetric Units", "Buildings-maximum ground extent", "Building-footprint"), and the classes deemed useful for this study are "Volumetric Units" and "Buildings-maximum ground extent".The first one is defined as a set of built volumes, identified by their footprint on the ground.The second one is defined as a set of continuously built volumes defined by a unique typology and can correspond to the sum of adjacent "Volumetric Units" (Ferrari et al., 2021).
OSM dataset is an open-source world map generated by volunteering geographic information.It represents the physical features of the ground (e.g., road, building) using tags attached to its basic structure.Each tag corresponds to a specific geographic attribute of the feature it represents (OSM community, 2023a).Regarding the building dataset, it is saved under the "building" tag, and it is basically a layer containing several polygons representing the buildings' footprints.By looking to the dataset metadata of Sabbioneta (OSM community, 2023b) it is possible to see that the buiding dataset was uploaded in 2018 from the municipal technical map (carta tecnica comunale in Italian) of the city, released under ODbL licence by Sabbioneta administration.This map was used to generate the Sabbioneta OSM dataset, including building features.Within OSM dataset, building units can be identified under the "Building" tag, which represents the building outline.To download the buildings layer, the QGIS plugin QuickOSM is used (Ehrig-Page, J.C., 2020).This tool allows to download data from OSM database simply by entering the geographic area, the key (corresponding to the tag) and a value.For example, to download all residential buildings, the combination of key=buildings and value=residential should be used.To download everything under the key=buildings, it is possible to use value=all.

Data preparation
The proposed automated method identifies buildings units on the point cloud based on the building subdivision retrieved from the cartographic dataset.
Before proceeding with building segmentation, a dataset validation is necessary.First, it is necessary to verify that point cloud and cartographic datasets are georeferenced on the same reference system.For the case study, all datasets are referenced on Projection WGS84 UTM zone 32N.
Then, a consideration on the polygons shape should be done.Firstly, the scale of representation of cartographic dataset should be discusses.In fact, depending on the scale of representation, it is possible to identify the level of accuracy of the plotted data, which is related to the approximation of the drawing, and referred to as plotting error; for scale 1:1000 and 1:2000 (i.e., that used for the DBTR and OSM) it is 20 cm and 40 cm, respectively.Secondly, polygons on the cartographic datasets represents the building footprint on ground, which means any element projecting from the façade (e.g., roof eaves, eaves pipes, windowsills, balconies) is not considered within the polygon.Thirdly, both MMS and UAS photogrammetry point clouds have a higher level of detail and a centimeter accuracy.Considering the three scenarios presented, it is possible that by overlaying the map data with the point cloud, the polygons in the map data do not correspond exactly with the buildings in the point cloud, and are not capable of covering entirely the building and all its elements.To overcome this problem, it is necessary to perform an outward expansion of the polygons.
Polygon expansion is an important and at the same time delicate task.This expansion must be carried out carefully so that overlaps between adjacent polygons are not created.In fact, making an expansion on all sides of adjacent polygons will result in areas where point cloud points are simultaneously assigned to both polygons (Fig. 2a).The desired behavior is that in cases where there is an adjacency to other polygons, there is no expansion, and that there is an expansion only on sides of the polygon where there is no adjacency.
To do so, an offset is generated for polygon lines representing outward-facing building sides (i.e., building façades).In contrast, no offset is created for polygon sides representing contact lines between two buildings.For each polygon, the lines not in common with other polygons are identified and the offset is generated using the python opensource library Shapely (Gillies, S. et al., 2007).Specifically, taking a city neighborhood into consideration, all the adjacent polygons contained in the neighborhood were merged into a single polygon (Fig. 2b); now, the perimeter of the new polygon represents the street façades of the buildings in the neighborhood.Then, by overlaying the perimeter thus identified with each polygon, it is possible to identify which sides of the polygon need to be expanded.To expand them outward, an offset is made on both sides of these lines, which are then rejoined to the initial polygon (Fig. 2c).The result is an outward expansion of the polygons, without overlapping with adjacent polygons (Fig. 2d).(a) for all polygons, an offset is performed on all sides; this result is not acceptable because on adjacent sides there should be no offset.(b) to obtain the correct result, it is necessary to identify the sides of the polygon (in red) on which to make the offset.(c) offset is made only to the outside or to the inner courts but not to the adjacent sides.(d) the final result of the operation is what is then used for segmentation.

Building segmentation
To perform building segmentation, an initial coarse selection is made to select only the points relating to the individual building units on the point cloud, followed by a refinement.The procedure is repeated for each polygon (i.e.building unit) in the cartographic dataset: after selection of building unit points on the point cloud, they are associated with a unique identification number, stored as a point attribute.The unique identifier serves as the key to link the geometric datum in the point cloud with the information datum in the map database.
The coarse building unit selection procedure starts from the coordinates of the polygon vertices, from which an oriented Bounding Box (BB) is generate.This BB is assigned a height such that it contains all the points in the cloud, i.e. a height greater than the Zmax and lower than the Zmin of the points in the point cloud.The BB is then used to make an initial selection from the general point cloud.Then, using the polygon perimeter as a guide and looking only at the XY coordinates of the points, the coarse selection is refined to select only those points actually contained within the polygon.To do this, the Open3d (Zhou, Q. Y., et al., 2018) and Geopandas (Jordahl, K., et al., 2020) python libraries are used in combination.
Once the points have been identified, it is necessary to identify them uniquely within the entire point cloud.To do this, they are assigned a new numeric attribute, given by the specific polygon's identification number in the map dataset.

RESULTS
Map data from the "Volumetric Unit" and "Building-maximum ground extent" layers of DBTR are used, and all features with the key "Building" are retrieved from OSM database.All three datasets consist of polygons, and only the ones within the area of the city of Sabbioneta are selected and used.Looking at the datasets, the "Building-maximum ground extent " layer is very similar to the layer obtained from OSM, while the "Volumetric Unit" layer contains many more elements.The number of polygons for the three layers is reported in Table 1.
After some empirical tests, an offset value of 40 cm is selected for polygon expansion.This value is compatible with the plotting error of the 1:2000 scale, and appropriate for the purpose of including in the selection the elements projecting from the façade.This operation was performed on all polygons contained in the map database, resulting in a new updated layer to be used for building selection.
Based on the updated polygons, points from the point cloud are selected and segmented, adding a new attribute with the identifying number retrieved from the information content of the respective polygon from the cartography.This value can be found in the "uuid" and "cr_edf_uui" attributes in the DBTR layers, while it can be found in the "id" attribute of the OSM layer.
The segmentation operation is performed using all three selected map layers one at a time to segment MMS and UAS photogrammetry point clouds.Considering the total time of the process (expanding polygons, loading the point clouds into memory, segmenting, and saving the segmented cloud) and dividing it by the maximum number of polygons in each layer, the average computation time results is 4 seconds for each polygon (performed on an AMD Ryzen 9, 16 core, 128 GB RAM computer).Figure 3 and 4 show screenshots of the segmented point clouds.
In the building segmentation operation, the percentage of polygons actually used varies depending on the point cloud and layer used.Specifically, for the point cloud obtained with UAS photogrammetry there is full use of polygons, while for the MMS point cloud some polygons are not used to perform segmentation.

DISCUSSION
The presented method is capable of segmenting a point cloud exploiting polygons retrieved from cartographic dataset.A link between cartography and point cloud is established by adding as point attribute a unique identifier taken from the cartography.
Since different data sources are used, it is possible to identify the advantages or disadvantages arising from them.Regarding the cartographic databases, it can be noted that the datum retrieved from OSM is very similar to the "Building-maximum ground extent" datum of the DBTR.They both actually represent the global shape of the various buildings, however, they often consider as a single building something that is actually an aggregate of several sub-units (Fig. 5a,b).The DBTR's "Voulmetric Units" layer, on the other hand, shows each individual volumetric unit, and takes into account precisely the latter case (Fig. 5c).However, there may be situations where this subdivision is excessive; for example, when a building has porches at the street level, there is a specific polygon only for the porch area (Fig 6a) resulting in an excessive subdivision compared to the other two layers (Fig. 6 b,c).Note that for the first case the highlighted (in green) building has been divided into two portions, of which the one on the right is just a porch area that belongs to the building.On the next two layers (b and c) the building is identified as a single element.Note also that the entire building is identified differently between OSM (c) and DBTR (b).
The choice to expand polygons is dictated by the need to use data sets with different accuracies and to include in the selection all building elements.One example can be seen in Figure 7, which shows a building of the city, the Teatro all'Antica, segmented on the point cloud obtained by UAS photogrammetry.Doing the segmentation of such building using the original polygon (Fig. 7a), a portion of the façade and part of the eaves in the roof are not selected because they are not within the polygon perimeter.
After expanding the polygon by 40 cm (Fig. 7b), all points in the point cloud inherent to the building are selected.However, even with an offset of 40 cm some portions of the building projecting far outward (e.g., eaves, balconies) are still not selected, showing that a revision of the offset value may be necessary in some cases.Unfortunately, by making such an expansion, in addition to completely selecting the façades, a portion of the sidewalk and the street facing the building are also selected.If the point cloud had been previously segmented, for example, into buildings and street area classes, it would be possible to easily remove from the selection the portions of the street not pertaining to the building.Regarding the expansion of polygons, overlaps do not occur if the exterior façades of the buildings are perfectly in continuity, as is the case in most cases for Sabbioneta.However, in some cases overlaps still occur during polygon expansion; for example it may happen where one building is more projecting than the others, or in the case of internal courtyards, where there are convex corners between buildings, Since these cases are limited to a few dozen polygons out of the total, this behavior is considered acceptable at the moment, but it deserves to be further investigated in the future.
Focusing then on the point clouds used and the resulting segmentation, it can be observed that the number of polygons used for segmentation is always 100% for the UAS photogrammetry point cloud, while there are always lower percentages for the MMS point cloud.This occurs because the main difference between the two point clouds is that they cover different zones of the urban environment.In the UAS photogrammetry point cloud, the roofs of all buildings, plaza areas, and streets are surveyed, while building façades are only partially visible.In contrast, in the MMS point cloud, building façades, plazas and street areas are surveyed, but roofs are seldom present.All cartographic databases have polygons of both streetfacing buildings and buildings placed inner inside the neighborhood.In the UAS point cloud it is possible to locate all buildings because both building typologies have been surveyed (there are points of the interior buildings related to their roofs), while in the MMS point cloud it is possible to select the points of only those buildings that face the street.Those not facing the street are not detected by the latter system used.Consequently, if it is of interest to focus on the roofs of building units, it is convenient to perform segmentation on the UAS point cloud, while if building façades are of interest, then the point cloud generated by MMS becomes more useful.
After segmentation is completed, the point cloud contains a new attribute that links each point to the corresponding polygon on the cartography and thus allows the retrieval of information linked to it.In addition, it can also be possible to extract information, mainly geometric, from the point cloud and append it to the respective polygon in the map database.
Otherwise, it is also possible to save the point cloud of the individual building.This can be useful, for example, to local administrations, which can then have geometric information (street façade and roofing), at a high scale of detail, of each individual building and can, for example, provide it directly to citizens or technical studies that request it.
The segmented point cloud can thus become the starting point for BIM or CIM, but it can also be used as a container for the information.In the latter case, the points of the various building units can serve as an element to which various pieces of information from the various existing mapping databases can be linked.The point cloud would then become the place where to store all the information that can be come from many sources.In that case, however, it would be necessary to identify appropriate software and tools that would allow the information associated with the points to be linked and retrieved through specific databases.The study of such methods and the identification of the most appropriate viewers (commercial or opensource) will be the object of the continuation of this research.

CONCLUSIONS
This paper presented an automated method for segmenting an urban point cloud into individual building units using a cartographic database as a source.The method was tested using point clouds obtained by MMS and by UAS photogrammetry, in combination with cartographic data from DBTR and OSM.The case study identified was the historic city of Sabbioneta.
From the calculations emerges that the method works, although further refinement could be done for the initial stage of polygons refinement from the cartographic data.One major difference lies in the point cloud used, in fact, the MMS one contains only building façades, while the UAS one contains mainly roofs.
Although the segmented point cloud might be used to develop CIM and BIM modeling, a more central use of it is envisaged, as the preferred container of the information that can be directly linked to each building identified in it.Future steps in this research will therefore consist of tests with other mapping databases and with information from other sources, the use of point clouds from other systems (e.g., ALS), and the identification of systems to visualize the point cloud and its information content.

Figure 1 .
Figure 1.Workflow of the method presented in this paper, that allows to segment single building instances from an urban point cloud, leveraging on 2D cartographic datasets.

Figure 5 .
Figure 5. Examples of polygon expansion in a Sabbioneta neighborhood.(a) for all polygons, an offset is performed on all sides; this result is not acceptable because on adjacent sides there should be no offset.(b) to obtain the correct result, it is necessary to identify the sides of the polygon (in red) on which to make the offset.(c) offset is made only to the outside or to the inner courts but not to the adjacent sides.(d) the final result of the operation is what is then used for segmentation.

Figure 3 .
Figure 3. Screenshot of UAS photogrammetry point cloud after single building segmentation using DBTR Buildings-maximum ground extent layer.Points not classified as buildings are in grey, each building unit is coloured with a different colour.

Figure 4 .
Figure 4. Screenshot of MMS photogrammetry point cloud after single building segmentation using DBTR Buildings-maximum ground extent layer.Points not classified as buildings are in grey, each building unit is coloured with a different colour.

Figure 5 .
Figure 5.View of a Sabbioneta neighborhood in the 3 different map layers: OSM (a), DBTR-Buildings (b), DBTR-Volumetric Units (c).Note that for the first two layers the subdivision is very similar, while in the third case there are many more sub-units.

Figure 6 .
Figure 6.View of a Sabbioneta neighborhood in the 3 different map layers: DBTR-Volumetric Units (a), DBTR-Buildings (b), OSM (c).Note that for the first case the highlighted (in green) building has been divided into two portions, of which the one on the right is just a porch area that belongs to the building.On the next two layers (b and c) the building is identified as a single element.Note also that the entire building is identified differently between OSM (c) and DBTR (b).

Figure 7 .
Figure 7. Point cloud of the Teatro all'Antica building segmented using the original polygons (a) and the polygons after making the 40 cm expansion (b).It can be seen that after the expansion the façade on the short side has been fully selected, although part of the eaves is still unselected.

Table 1 .
The number of polygons effectively used is shown by Table1.Number of polygons of the three cartographic datasets used in this paper.The table also reports the polygons effectively used for the single building segmentation on the two point clouds, acquired with a MMS and obtained through UAS photogrammetry.