VIRTUAL INSPECTION BASED ON 3D SURVEY SUPPORTING RISKS DETACHMENT ANALYSIS IN PIETRAFORTE STONE BUILT HERITAGE

: The paper presents the first results of a multidisciplinary research project launched to support the conservation and restoration of the stone façades of the Pitti Palace in Florence with innovative techniques from the fields of geomatics and diagnostic analysis. Monitoring campaigns are periodically conducted on the façades of the palace to identify stone elements in critical conditions; such surveys primarily require close and careful observation of the façade, for which a crane basket is required. The paper proposes first attempt to compare results obtained through a traditional workflow with those coming from a deeper use of the high-resolution 3D model to conduct a virtual inspection and to map elements of vulnerability on a GIS. On a test area, the analysis of the factors considered relevant to the risk of detachment was carried out on the digital model and compared with what the experts observed on-site by carrying out Non-Destructive diagnostic tests. Traditionally conducted monitoring and diagnostic surveys are assumed to validate the proposed method, which, following a simple data analysis, remotely


INTRODUCTION AND PROJECT GOALS
The paper presents the first results of a multidisciplinary research project launched to support the conservation and restoration of the stone façades of the Pitti Palace in Florence with innovative techniques from the fields of geomatics and diagnostic analysis. The pilot project focuses on one of the façades of the courtyard of honour, designed by Bartolomeo Ammannati and built in the second half of the 16th century with Pietraforte sandstone blocks quarried in the very close surroundings of the palace. The rustic ashlar decorations in Pietraforte suffer from the preservation problems characteristic of this material, which in some circumstances also lead to the detachment of scales, fragments and portions of significant size. The issue of built heritage conservation is therefore associated with the safety of visitors crossing the courtyard to enter the palace or access the famous Boboli Gardens through it. The "Gallerie degli Uffizi" office, whose duties include the maintenance and conservation of the historical building, has recently launched a major project to digitise all of the Palace, in collaboration with the University of Florence, Department of Civil and Environmental Engineering (the GeCo Lab, directed by Prof. Grazia Tucci, carried out the 3D metric survey). The Department of Earth Sciences has also been collaborating for several years with the Galleries in carrying out diagnostic surveys (by LAM Lab, directed by Prof. Carlo Alberto Garzonio) (Landi et al. 2019). The collaboration between the two research groups aims to evaluate digital twinning systems for the predictive identification of criticalities in blocks of stone that could lead to detachments and falling material. The paper proposes a method for the integrated management of geometric data (processed from photogrammetric and laser scanner surveys) and the results of diagnostic investigations, with particular reference to Non-Destructive Techniques (NDTs).

THE CORTILE DELL'AMMANNATI IN PITTI PALACE CASE STUDY
Cosimo I de Medici transformed Palazzo Pitti into a palazzovilla on the border of the city and overlooking the rear garden. Bartolomeo Ammannati then designed a U-shaped courtyard open to the hillside and began the construction of the northern façade in 1561, then proceeded with the central body, which also required the demolition of the so-called "old house," until completing the work with the southern façade in 1577. The façades are developed on three floors above ground that are of three architectural orders -Doric, Ionic, and Corinthian, from the ground up. Ammannati uses rusticated orders in which the ashlars are treated differently depending on the level: on the ground floor, the rusticated ashlars have a bulging, domed form and are very close to each other, thus giving a sense of basement; on the upper floors, they are gradually smoothed, with flatten ashlar on the second floor and domed ones on the third, creating a strong light contrast. In view of a restoration project, the southern façade was identified as a pilot in order to experiment the use of innovative tools both for the management of spatial data, made available by the 3D survey, and of any other information collected during diagnostic investigations.

The Pietraforte sandstone
The Pietraforte has been widely used in historic Florentine buildings. Its decay has been pioneeringly studied by (Malesani and Vannucci 1974), who paved the way for studies that have continued over time by Florentine researchers (Cantisani et al. 2013, Fratini et al. 2014, Pecchioni et al. 2020, Coli et al. 2022).
The Pietraforte, from a petrographic point of view, is defined as a lithic sandstone, a sedimentary rock belonging to the turbidic formation present in the allochthonous complex of the Outer Liguridae. The turbid currents flow formed typical convolute laminations, which are visible to a more or less marked degree on the ashlars (Figure 1) (Abbate and Sagri 1970, Malesani and Vannucci 1974, Fontana 1991, Nirta et al. 2005. The sandstone presents a gray-bluish colouration in fresh cut, which takes on a typical ocher colour for alteration due to iron oxidation. However, some areas remain grey for centuries ( Figure 2) (Pecchioni et al. 2020). Pietraforte blocks are also characterized by the presence of "veins" of calcite that tend to break off. On the façade block, it is possible to recognize veins in various inclinations; when a series of them are parallel to each other, they are considered to belong to the same "family" (Figure 3) (Pecchioni et al. 2020). The action of weathering leads to the solubilization of calcite and consequently to the detachment of portions even of significant size. Water also dissolve the rock's carbonate cement, causing intergranular decohesion phenomena to occur, and acts on clay minerals, which expand and retreat with thermo-hygrometric cycles, causing surface disintegration and exfoliation.   Monitoring campaigns are periodically conducted on the façades of the palace to identify the stone elements that present greater criticality concerning their state of preservation and, in particular, to allow their timely securing. The operations require the technicians to work at a close distance from the stone face, primarily to allow them to observe it closely and recognize laminations, calcite veins, and exfoliation phenomena, but also traces of previous restoration, consolidation or securing interventions (such as surface treatments, metal or other material fasteners). It is also necessary to carry out, on some blocks, some NDTs that require direct contact with the structure: sclerometric measurements through the Schmidt hammer test, that evaluate the quality of the stone surface, indicating the strength of the surface portion of the test sample, and the Ultrasonic Velocity Test, for evaluating the consistency of masonry and allowing the identification of internal defects like fractures, voids, and detachments. The combination of these two techniques, tailored to the specific object of study, enables a comprehensive examination of the properties of the stone (Salvatici et al. 2020, Calandra et al. 2022, Centauro et al. 2022. By integrating these techniques at the same points on the stone, a satisfactory characterization can be achieved, despite both techniques providing an estimation of the mechanical characteristics. For this purpose, a crane basket is used and carefully approached to different façade areas ( Figure 4).

3D SURVEY AND DATA COLLECTION
Several studies report the use of 3D surveys for referencing and visualizing deterioration mapping, particularly in the case of historic building façades (Russo et al. 2019, Tsillimantou et al. 2020, Adamopoulos and Rinaudo 2021. In the Ammannati courtyard, a high-resolution 3D survey of the façade was performed by drone photogrammetry. Images were shot by a manually piloted DJI Phantom 4 UAV -due to the high proximity to the building wall, using automatic systems is impossible. The camera network included regular stripes, with a forward overlap of 80% and a side overlap of 70% referred to the photos orthogonal to the façade. The elaborate structure of the decorative apparatus also required converging images to reconstruct the undercut portions better. A total of 2941 photos were then oriented thanks to 16 ground control points arranged in a regular pattern and surveyed by a total station. The RMSE evaluated on 12 checkpoints attests to the sub-centimetre accuracy achieved by the project. An ortho-mosaic with 0.2 cm GSD was then produced, which serves as the cartographic basis for the project, and a DEM. It is thus possible to describe the façade concisely if inevitably approximated, considering that the overhang of the ashlars constitutes one of the main critical issues for the risk of detachment of stone materials.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-M-2-2023 29th CIPA Symposium "Documenting, Understanding, Preserving Cultural Heritage: Humanities and Digital Technologies for Shaping the Future", 25-30 June 2023, Florence, Italy The projection plane adopted in the GIS project corresponds to the one used for the orthophoto and DEM, as well as for the CAD drawings, which are the geometric and metric documents used for referencing and quantifying the thematic information subsequently associated. The research project aimed to experiment with the virtualization of the inspection phase carried out on the various forms of digital reproduction of the façade: the high-resolution orthomosaic, the texturized 3D model, and the oriented images. High-resolution ortho-mosaic: It allows an overall view of the façade and easy zooming in on the area of interest, with the obvious limitation of viewing it in an orthogonal view, that sometimes compromises the interpretation of undercuts ( Figure  5). On the ortho-mosaic it is, in fact, relatively easy to identify ashlars, while it is more complex to delineate stone blocks: often, an architectural element does not consist of a single block, or in turn, a block may encompass more than one architectural element. Moreover, the ashlar drafted-margin tapes are thin and sometimes in different positions (some on the upper part of the ashlar, others on the lower).

Textured 3D model:
The level of detail of a 3D model depends on two complementary aspects: the geometry (polygons count) and the texture resolution. Regarding the first aspect, the model resulting from the digitization of the façade makes it possible to accurately size the ashlars and assess their overhangs and the lack of any portions. Adding a high-resolution texture makes it possible to observe stone features that are significant for this project but not documented by the mesh, such as calcite veins or laminations ( Figure 6). Given the extent of the façade, however, using one texture at the highest resolution for the entire model is not sustainable: textures may be loaded progressively, but dynamically loading and transferring this large amount of data into GPU memory results in loading delays and poor performance.   Digitization processes generally consider the latter aspect incidental to the former. In this case, on the other hand, the relationship between 3D model and oriented cameras was exploited for the virtual inspection of the façade: the selection of a point on the block under analysis allows its rear projection and, thus, the quick identification of all the photos framing that point, avoiding time-consuming manual searches in the dataset of about 3000 images. Observing the original photos allows for maximum resolution, without the resampling and blending implemented in the ortho-projection and ortho-mosaic processes, as well as converging viewpoints helpful in interpreting the side faces of the ashlars and better recognizing the joints between blocks.

MULTIDISCIPLINARY DATA COLLECTION
The collaboration between the two research groups aims to evaluate digital twinning systems for the predictive identification of criticalities in blocks of stone material that could lead to detachments and falling material. Further, a method for the integrated management of geometric data (processed from photogrammetric and laser scanner surveys) and the results of diagnostic investigations (with particular reference to NDTs) is proposed. The need to consider the structural elements of the façade instead of the decorative ones led to prepare two graphical outputs: the drawing of the architectural elements based on the interpretation of the architectural orders, also plotting the ashlars, and the drawing of the individual stone blocks. A GIS project was created with all the raster and vector layers mentioned before by using the open-source software QGIS (QGIS.org). Obviously, the plane of the cartographic representation was fictitiously adopted as corresponding to the projection plane used for the ortho-mosaic and CAD drawings. The relationship between the reference system adopted for the project and that used for the entire survey of Pitti Palace is also documented. The spatial data are, at the same time, the basis for referencing thematic data (analysis of materials, forms of decay, diagnostic investigations) and for validating the criticalities that emerge from the study of the high-resolution 3D model of the portion of the building under investigation, as well as allowing the archiving and complete consultation of the qualitative and quantitative data produced during the survey and analysis campaigns.

Stone blocks mapping
Depending on the study's objectives, the building's components can be identified and organized into more or less complex structures, classifying them into classes and subclasses (Malinverni et al. 2019, Bonora et al. 2023, Korro Bañuelos et al. 2021, Costamagna, Spanò 2012, Brumana et al. 2020. For the present work, the architectural elements of the façade were structured according to the hierarchy defining architectural orders (Figure 8). On the other hand, for diagnostic analysis, the stone block was identified as the minimum spatial unit. In fact, there is not always correspondence between the architectural decoration, arranged in three superimposed orders characterized by ashlars of different shapes and surface finishing, and the structure of the façade. In many cases, a single architectural element is made of different stone ashlars; in others, the same block is carved to represent different architectural elements (Figure 9). The individual ashlars were then manually drawn by polygons thanks to a careful observation of both orthophotos and raw photos -particularly the converging ones helped identify less apparent discontinuities.

Diagnostic parameters related to the risk of detachment
The risk of detachment of stone blocks parts is correlated to aspects, at the moment, considered only in qualitative terms, such as the pattern of laminations, the presence of discontinuities, erosion and exfoliation phenomena, as well as geometric parameters (such as the overhang of the blocks), which in turn can be extracted from the 3D model. The specific experience developed by the LAM Laboratory of the University of Florence on the analysis of historic buildings in Pietraforte has led to the identification of the critical elements to be taken into account and thus to be recognized and mapped on the blocks: convoluted laminations, open discontinuities, presence of red calcite veins, inclination of discontinuities to the plane of the façade, presence of multiple parallel discontinuities, material failures, metal banding, and grouting. The conservation state of the ashlar has also been considered, particularly with regard to erosion, crusts and deposits, biological colonization, cracks and fractures, colouration, material loss, metal parts and restoration work carried out in the past (sometimes with techniques that have proven to be nonoptimal over time).

VIRTUAL INSPECTION WORKFLOW ON A TEST AREA
A current research topic is the attempt to objectively quantify the presence and effects of various forms of decay and then calculate damage indices that provide helpful information to allow the prioritization of conservation and preservation responses (Randazzo et al. 2020).
In the present work, an innovative approach was adapted from the analysis techniques usually applied for mass rock classification and risk assessment. Mechanical properties were evaluated through ultrasonic and Schmidt hammer tests on the blocks deemed most critical after visual investigation. Two analysis processes were conducted in parallel and then compared.
On-the-field inspection: Carried out during the February 2022 monitoring campaign and extended to the entire façade. It involves close observation of the blocks using a crane lift basket for the many parts not otherwise accessible, then identification of blocks deemed most hazardous and the application of NDTs on them for a more objective assessment. Diagnostic investigations carried out on site by LAM technicians required aerial platforms, and the identification of the most vulnerable blocks was made directly from there; the inspections of the entire southern façade took two operators working for seven days on the platform, plus one operator to move the crane.
In the test area, 277 blocks were visually analysed, and the diagnostic experts decided, based on qualitative considerations and especially their experience, to investigate seven of these further with instrumental analysis. Virtual inspection: It has been applied to a test area and implemented for a master's degree thesis (Raffa 2022) based on the 3D data collected in July-August 2020. It plans to map risk elements by observing digital survey outputs, proposing a preliminary screening useful to address further analysis better. The virtual inspection was conducted considering the orthophoto, the 3D model, and the oriented images. In this case, a non-specialized operator did the inspection after brief training, which allowed her to recognize and map the presence of discontinuities and laminations; the overhang of ashlars and the co-presence of degradation forms were then assessed ( Figure  10). From the entire dataset of blocks in the test area (277 blocks), only those with laminations (75 blocks) and discontinuities (142 blocks) were then selected (Figures 11). It should be noted that the available image resolution did not allow for a better detailing of whether discontinuities were open or not. In relation to the risk of detachment, one element to consider is also the shape of the ashlar: it is clear that detachments of material from overhanging blocks can be more dangerous. Considering the DEM, the selection was therefore reduced to overhanging blocks only ( Figure 12). Finally, the co-presence of some forms of degradation (particularly exfoliation and erosion) was assessed, further reducing the selection to only 36 blocks ( Figure 13).    . Results of the virtual inspection leading to highlight 36 blocks being deemed vulnerable. In red, blocks selected from on the field inspection.

Results on the test area
Following the workflow described above, 36 of the 277 ashlars were identified as potentially vulnerable through the virtual inspection. All of the blocks deemed at risk by the diagnostics experts are included in this set.

Laminations Previous interventions
Material loss

Colouration Discontinuities
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-M-2-2023 29th CIPA Symposium "Documenting, Understanding, Preserving Cultural Heritage: Humanities and Digital Technologies for Shaping the Future", 25-30 June 2023, Florence, Italy It can therefore be concluded that the analysis conducted on the various outputs of the façade digital survey would have allowed, in the test area considered, to limit the direct observation from the crane basket to 13% of the inspected blocks, with a consequent reduction in the time of the on site investigation. It should also be considered that the working conditions of the operator at the PC are more comfortable, independent of weather conditions, allow for the verification of interpretative hypotheses by easily sharing information with other experts, and for the minimisation of omissions due to the complex crane handling methods. A reliable risk assessment obviously requires more accurate modelling, which is still in progress, and the contributions that may come from a survey using geomatic techniques must be carefully evaluated, both in terms of high-resolution spatial data and their structuring in specific databases (Salvatici et al. 2023). A further test carried out leads to promising results, however, to be validated in more extensive and diversified areas. A specific query evaluating the presence of discontinuities and laminations not as an alternative but limiting the selection of ashlars to their co-occurrence (and also considering the presence of the other degradation factors mentioned above), highlights only seven blocks to be considered with particular attention -of these six are those identified by the diagnostic experts and further investigated with NDTs. Figure 14. Blocks on which the geologists and conservation scientists deemed it appropriate to investigate further with NDTs.

CONCLUSIONS AND FURTHER PERSPECTIVES
The paper presents how high-resolution spatial data can profitably support a monitoring project, allowing to make on the façade digital twin some of the analysis traditionally performed in situ by expert operators who observe the stone elements from a close distance, even with crane baskets.
The spatial data are, at the same time, the basis for referencing thematic data (analysis of materials, decay, diagnostic investigations) and for validating the criticalities that emerge from the study of the high-resolution 3D model of the portion of the building under investigation, as well as allowing the archiving and updating of data produced during the monitoring campaigns, which are carried out at short time intervals to guarantee the safety of the building and people. All blocks deemed critical in the test area were also highlighted by the virtual inspection, thus demonstrating, albeit preliminarily, the validity of the proposed method. Its finetuning will allow time on the field to be shorter, reduce costs, and carry out the work in more comfortable conditions.