PRELIMINARY TEST ON STRUCTURAL ELEMENTS HEALTH MONITORING WITH A LIDAR-BASED APPROACH

: The safety and usability of infrastructures such as bridges, roads, and buildings must be monitored throughout their useful life. Traditional inspection methods are time-consuming and expensive, and innovative solutions using LiDAR-based techniques have developed. This study presents a semi-automatic method for detecting deteriorations on structural elements of a bridge using an integrated dataset of point clouds and radiometric information. The method involves using a Terrestrial Laser Scanner (TLS) to obtain high-resolution georeferenced point clouds of the bridge beams, which are then filtered to identify four classes of deteriorations. Six Machine Learning Classifiers are tested and compared using Overall Accuracy and F1-score metrics. The Random Forest emerged as the best-performing. It was then optimised by reducing the input features through an importance analysis and the accuracies measured. The results show promise and can be explored further on a larger dataset. The study aims to generalise the methodology to transfer it to actual cases.


INTRODUCTION
Bridges are a crucial component in countries' infrastructure.They enable the transport of raw materials and goods to markets, allowing people access to essential services.Hence, economic activity is closely linked to the bridge's availability.Italy has more than 1034 kilometres of bridges and viaducts ("Bridge collapse highlights Italy's aging infrastructure," 2021).In particular, there are around 2000 highway bridges.Many of the bridges date back to the construction boom in the 1930s and the post-war recovery programs of the 1960s and are now reaching the end of their lifespan.Moreover, most bridges also carry significantly more weight than originally expected ("Keeping European bridges safe," 2019).Based on this, bridges maintenance, inspection, and monitoring should be an aspect of high priority for a country.On one side, it is important to ensure the bridge's readiness to maintain the economic equilibrium of its surrounding zones.However, on the other hand, it is vital to prevent fatal accidents such as the Morandi Bridge collapse in Genova (Italy) that caused 43 deaths (Mattioli, 2019).Specifically, deteriorations can be caused by (i) natural aging of the structure related to prolonged use, type of material and frequency of ordinary and extraordinary maintenance activities; (ii) extreme environmental conditions, such as periods of heavy rainfall or strong gusts of wind; (iii) the occurrence of natural or anthropomorphic disturbances, such as earthquakes or road accidents that can cause an extraordinary structural load (Luechinger et al., 2015).
Currently, the inspection of bridges is a time-consuming procedure that involves a team of engineers that carefully inspect the structure, looking for any flaws, defects, or potentially problematic areas that may require maintenance.In addition, some special machinery like snooper trucks is used frequently to reach the zones of difficult access of the structure, increasing the inspection costs.Moreover, the inspectors should operate at large distances from the ground, risking their lives.
Bridge inspections are specified by different government guidelines, which can vary significantly by country.In Italy, they are given by the Minister of infrastructure and transport, which stipulates the process of risk classification, safety evaluation, and monitoring of bridges (Ministero delle Infrastrutture e dei Trasporti, 2019).Some innovative solutions involve using photogrammetry techniques on Uncrewed Aerial System (UAS) imagery as costeffective tools for bridge inspection.This approach combines the advent of computer vision, recent advances in machine and deep learning techniques, and the spread of UAS, which provide high-resolution georeferenced images and allow for automatically detecting deteriorations such as discontinuities and defects in a more efficient way (Belcore et al., 2022).UAS can also be equipped with multispectral, thermal, and LiDAR sensors.
More recently, LiDAR-based techniques have become one of the most popular methods for object acquisition in the civil engineering domain, specifically for health monitoring and assessment of surface conditions and damages or irregularities combining high-resolution geometric and radiometric information with millimetric accuracy (Cao et al., 2017;Kaartinen et al., 2022).Nevertheless, methods based on the use of cameras are still the most widespread in the monitoring of viaducts (Cha et al., 2018), and regardless of the data source, Most of the analysis related to lidar data uses very complex models based on deep learning for the semantic segmentation of the entire generated point cloud (Rashidi et al., 2020).
In this work, we present an alternative to the current bridge inspections.We propose an inspection method using Terrestrial Laser Scanning technique and Machine Learning (ML) for the detection of damages in the structure.Applying ML on highly dense 3D point clouds requires an accurate dataset preprocessing and an optimised pipeline for transformers and classifier modelling.In fact, the presence of the threedimensional component and the high density of the acquired data makes the application of classical iterative procedures typical of supervised learning procedures very cumbersome.In this case, it is essential to keep the ram allocation under control by planning all phases of the training procedure.
The analysed study area is made up of two beams of a bridge that have fallen into disuse and are located in a controlled environment for research purposes (Savino et al., 2023).With this work, we test the capability of the workflow to improve the bridge inspection processes by reducing the use of expensive machinery, avoiding people's exposure to dangerous operations, and reducing inspection time.In particular, the proposed solution involves a point-based supervised classification model that utilises Random Forest algorithm.The model leverages radiometric features (derived from the visible spectral response of the points) and geometric features.We selected the Random Forest algorithm through a comparative approach, which involved assessing its accuracy metrics and performance time against well-established classifiers.This report is organised as follows.Section 2 describes the material and methods; Section 3 introduces the obtained results.Section 4 discusses the results.

MATERIAL AND METHODS
The case study in a controlled environment consists of two structural beams with the shape of a cushioned inverted pyramid, 7.70 m x 1.5 m x 1.5 m, belonging to a bridge built in 1970 and demolished in September 2018 (Figure 1).The beams show clear signs of ageing; both have scorched and rusted irons.In some parts, the concrete has detached, and in others, the first signs of surface deterioration are visible.

Data acquisition
The first step of the work involve the acquisition of dense geometric and radiometric information of the beam throughout the LiDAR technology.The selected beams were subjected to an extensive on-site terrestrial integrated survey, combining total station and terrestrial laser scanner measurements.These activities allows to obtain an accurate 3D reconstruction of the beam as well as a radiometric characterization thanks to the imaging sensor integrated in the TLS.
To this purpose, the Leica RTC360 was used.Once acquired, the data has been post-processed to optimize the initial on-field scan-to-scan alignment and to evaluate the overall accuracy of the model.The software Cyclone Register 360 by Leica Geosystem has been used to this purpose.Nine scans were coregistered mixing the ICP (Besl and McKay, 1992) and the rigid six-parameter transformation approach.

Pre-processing
The pre-processing phase consists of four main steps to prepare the dataset for classification: i) identifying the classes, ii) extracting features, iii) labelling and splitting the data into training and testing sets, and iv) denoising and scaling step, performed to further enhance the quality of the data.

i)
Classes identification Four classes (Figure 2) describe the classification system, as (Table 1) illustrates.

Figure 2. The four classes i)
Feature extraction To enhance the information content of the data, derivative features were computed for each point.This process, known as The feature extraction process described in this study involved computing both radiometric and geometrical features based on the RGB information of each point.Table 2 provides a summary of the computed features.Geometrical features were calculated by describing the relationship between each point and its nearest neighbouring points, identified using a sphere of radius 0.05m.The geometric features were calculated with the Cloud Compare software ("CloudCompare,", 2023), while the radiometric features in the Python environment using the Scikit library (Pedregosa et al., 2011).
The final classification dataset consisted of 28 features: RGB information, the intensity, 17 geometric features and 7 radiometric features for each point of the cloud ( The largest eigenvalue of the point cloud.

First ord mom
The first-order moment of the point cloud.

Linearity
It measures how elongated or linear the point cloud is along its principal axis.
Mean curvature A measure of the average curvature of the surface at each point in the point cloud.

normal change rate
A measure of how rapidly the surface normal vector changes over the point cloud.

Omnivariance
A measure of the overall variance of the point cloud.It is computed as the product of all three eigenvalues.PCA1 The principal components of the point cloud computed using principal component analysis.PCA1 and PCA2 represent the two largest eigenvectors of the point cloud.

PCA2
Roughness A measure of the surface roughness of the point cloud.

Second eigenvalue
The second-largest eigenvalue of the point cloud.This can be used as a measure of the point cloud's elongation along its secondary axis.Denoising and scaling To ensure high-quality data, both the training and test datasets were cleaned of null or infinite records.Additionally, each feature was scaled using the MinMaxScaler algorithm available in the Scikit learn library.The scaler was applied to determine the minimum and maximum values of each feature and then scale the data by subtracting the minimum value and dividing the result by the range (i.e., maximum value minus minimum value).This scales the data to a range of [0, 1], which benefits many machine learning classifiers.
The scaler was fit to the training data only to avoid data leakage, and the same scaling was applied to the test data.This ensures that the scaling of the test data is consistent with that of the training data and prevents the model from seeing any information about the test data during training.

Classification model and prediction
The chosen approach for this study is a supervised point-based classification model.To enhance the model's robustness, we compared various classifiers.We selected some of the most popular and widely used classifiers and implemented them using an automatic pipeline in Python.Specifically, the classifiers we evaluated were Random Forest, Neural Network, Support Vector Machine (SVM), Decision Tree, Perceptron, and Naive Bayes.The parameters selected for each are shown in ( The performances of the classifiers were compared by calculating the overall accuracy, the F1, the precision, the recall scores and the training timing on the test dataset. The best-performing algorithm, Random Forest (RF), was applied to the entire (not labelled) dataset.Random forest algorithm embeds the computing of feature importance.(Breiman, 2001).Based on the GINI impurity criteria, the variables with large GINI gains are more significant because they have lower impurities.The threshold of 0.75*median values of importance was selected.Finally, the reduced RF model was validated and applied to the entire dataset.

TLS data acquisition
The nine acquired scans has been cleaned from the moving objects as well as from the surrounding environment, resulting in a final point cloud of 13,295,430 points.15 markers located on the beams have been side-shot measured with a total station and connected to a georeferenced topographic network materialized around the beams.These marked has been used to perform relative scan registration and to asses simultaneously the point cloud accuracy.At the same time, an ICP algorithm has been performed on the nine scans performing 21 scan-toscan bounding.The RMSE for the markers is 3 millimeters as reported in Table 4. with the overall statistics.The final result is a set of georeferenced metric coordinates, each associated with intensity values and radiometric information in Red, Green, and Blue (RGB) channels.0.001 0.002 Table 4. Overall errors statistics after scan coregistration.

Pre-processing
The dataset for classification consists of 566'458 training points and 242'768 test points with 28 features.The cloud was imported as Panda dataframe variable, scaled between 0 and 1, denoised from null and infinite values, and associated with position-indexed information (X, Y, Z).

Classification model and prediction
The classifiers performed outstanding, reaching values never below 0.90 overall accuracy (OA).As Table 5 shows Random Forest and neural network models reach the highest OA, followed by SVM and decision tree.

DISCUSSION
In order to develop a method that was capable of identifying the deterioration of a reinforced concrete infrastructure, it was decided to focus on the most common ones, which are responsible for the reduction in the resistance of the structural element.In particular, exposing irons to air and water can cause the armor to corrode, which can lead to rust.Rust causes the reinforcement to increase in volume, causing the surrounding concrete to fracture and the structure to lose strength.Moreover, the exposure of the bars can cause deformations in the reinforced concrete structure due to thermal expansion of the reinforcement.These reasons led us to the choice of the four classes (healthy surface, bare irons, deteriorated concrete, surface alterations, i.e.g, water infiltration) described in Section 2. The four classes used in this study are meant to serve as a general illustration, as the types of deteriorations observed in these types of beams are not highly variable.However, the identified classes can be viewed as initial macro-classes, within which more detailed sub-classes can be identified.
The feature extraction process necessary for applying predictive models included the computation of radiometric and geometric features.Therefore using a large number of features (17 geometric, 7 radiometric, and 3 RGB), it was possible to best generalise the model, avoiding the problem of overfitting and improving its final accuracy.
Selecting the appropriate input features remains a challenge in this study.In this preliminary work, we did not extensively analyse each class's response to different features.Given the small size of the dataset, we opted to compute as many features as possible and then eliminate the less important ones.This backward approach enables the evaluation of all features, but it can be time-consuming.
For the correct training of the predictive model, it was also necessary to scale the features with respect to a common range (typically between zero and one).This procedure helped for (i) reducing the sensitivity to small changes in the data, (ii) improving the convergence of machine learning algorithms, and (iii) avoiding the predominance of one feature over the others (this problem occurs when the values of a feature are very different to the values of the other features as per radiometric values [0-255] and other features).It would be necessary to test other scalers to determine if they outperform the MinMaxScaler in some way, as using the MinMaxScaler may cause the data resolution to flatten, leading to data leakage.However, in this study, the MinMaxScaler was chosen because it is less sensitive to the presence of outliers compared to other scalers, such as the StandardScaler or RobustScaler.
In order to use point-based supervised classification algorithms, it was necessary to carry out the manual labelling operation of a portion of the dataset; in turn, this is randomly split into a training (70%) and a test (30%) dataset.In this way, the training portion was used precisely to train the model to recognise the classes according to the relationships between the features; the test portion, on the other hand, was used to calculate the effectiveness of the prediction through the metrics of Overall Accuracy, Recall, Precision, and the F1 Score.The efficiency of the prediction model depends on the quality of the manual labelling, during which procedure it is necessary to identify portions of the dataset that exhaustively exemplify the considered classes.Furthermore, this operation is strongly dependent on the manual dexterity of the operator and his ability in segmenting the point cloud.Another aspect to consider when choosing the sample data to be labelled for each class is the number of samples themselves, since many classifiers respond to the ratio of training classes.In general, it is preferable to have a number of examples for each class proportionate to the extent of each class in the entire dataset.For this reason, some classes (such as bare irons) that affect a small portion of the entire dataset were selected for manual labelling in fewer numbers than other classes (e.g.healthy surfaces).
In our study, we considered different classifiers in order to identify the one that best suited the pre-established objective.
The best algorithm was chosen considering the aforementioned metrics and time.As shown in Table 5, the Random Forest model is the best-performing method overall.While it's not the fastest method (the time it takes to train the model is the third highest), it performs better in terms of Overall accuracy, Precision, Recall and F1 Score.The Decision Tree method is the second-best classifier, which takes much less time to train at the expense of slightly lower values on the metrics.
Further considerations were carried out considering the best classifier (RF).Results show that the "third eigenvalue" is the more determining parameter, followed by "PCA1" and "Surface variation".The third eigenvalue represents the variance explained by the third principal component.The third principal component is a linear combination of the original variables that explains the largest amount of residual variance after the first two principal components have been considered.The first principal component (PCA1), is the linear combination of variables that explains most of the variance in the data; it represents the direction along which the data vary the most and can be interpreted as a sort of "main axis" of the same.Finally, the Surface Variation it is a metric used to describe the roughness or complexity of a surface.It indicates how much height variation exists on a surface, i.e. the difference between the highest and lowest point of the surface itself.
Performing an overall visual analysis of the beam's deterioration classification, it is well performed on both beams under investigation.Damages of greater extent and severity are correctly identified, and some errors are made regarding less widespread and more specific damages.However, it is necessary to improve the model to identify in greater detail small areas where different classes coexist.Certainly, it is necessary to test the model on larger areas, expanding the training and test datasets, and evaluate its replicability in different operating and deterioration conditions; moreover, implementing a voxel-based approach could improve the classification, shortening the time and reduce the salt and pepper effect.
In any case, to the best of our knowledge, this study fits into a scientific gap and represents a first attempt to propose a standardised procedure for the classification of structural deteriorations by combining the use of LiDAR point clouds and artificial intelligence.Although the model presented here is relatively simple to implement, it lays the foundation for some important considerations.For relatively straightforward tasks such as detecting deterioration in viaducts, simple classification models based on the analysis of individual points are already effective.This leads us to believe that LiDAR point clouds could be used instead of carrying out visual inspections or using drone images as a database, which would speed up detection and localisation.While the image classification process is generally faster, the problem of geolocating the damaged area and determining its extent remains.Scaling the shot (knowing the internal parameters of the camera) and locating it in areas where the GNSS signal (the most common method for localisation) is often poor, given that the satellite signal is shielded by the bridge, can be challenging.Moreover, when working with images, it is not uncommon for the same defect to appear in different shots and therefore be counted twice.To overcome this, it is possible to work on photogrammetric point clouds derived from images, but with current technologies, the processing still takes a long time and is not suitable for near real-time analysis in the field.LiDAR point clouds, on the other hand, can be generated very quickly (e.g. in this work, 1.30 min per scan) and automatically aligned.The same algorithm could also be applied to LiDAR clouds generated by SLAM systems, which are mounted on autonomous vehicles.We plan to continue our work on this aspect by proposing a near real-time solution for the rapid and effective detection of defects using lidar sensors mounted on aerial drones.

CONCLUSIONS
The issue of structural monitoring of bridges is of primary importance to ensure the safety and economy of a country, especially in Italy.However, traditional monitoring techniques are commonly time-consuming and safely risky.In this study, we propose and test an alternative workflow for monitoring this kind of infrastructure; our approach can improve the bridge inspection processes and increase the safety of the surveyors involved.
The use of a high-precision Terrestrial Laser Scanner data suitably pre-processed combined with the application of machine learning algorithms lays the foundations for carrying out monitoring analyses in a faster and safer way; at the same time, it can guarantee the same reliability compared to traditional techniques, which are more time-consuming and safety-risky.
The acquired data consists of a coloured LiDAR point cloud from which geometric, radiometric, and intensity features have been extracted, which were further denoised and scaled.After labelling a portion of the dataset and subsequently dividing it into training and test data, several unsupervised point-based classification methods were tested.The results are promising, and all the tested models returned excellent performances (Overall Accuracy greater than 90%).Among these, it was decided to consider the most efficient method (RF) in terms of OA and time of processing, with which it was possible to quantify and evaluate the importance of each feature.
Nevertheless, further considerations and evaluations must be explored, and different conditions should be examined in depth.
The classification and prediction model can be further improved, i.e., by increasing the number of training points for each class; moreover, it is necessary to evaluate the replicability of the method, monitoring additional bridge beams, characterised by damage and alterations of different entities, extensions, and severity.

Figure 1 .
Figure 1.Up: photo of the studied beams.Bottom: TLSgenerated point clouds of the studied beams.
From a visual point of view, the classifier clearly detects the uncovered bars and concrete alteration areas of the input cloud.It is less precise on the alteration areas of healthy surfaces and surfaces.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-2/W3-2023 ISPRS Intl.Workshop "Photogrammetric and computer vision techniques for environmental and infraStructure monitoring, Biometrics and Biomedicine" PSBB23, 24-26 April 2023, Moscow, Russia feature extraction, involves selecting and transforming raw data into a set of informative features that are easier to work with and analyse.Feature extraction is particularly crucial when dealing with large and complex datasets, as it can improve classification accuracy and reduce the risk of overfitting in machine learning.

Table 2 .
Derivative features calculated for the point cloud.

Table 3 ).
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-2/W3-2023 ISPRS Intl.Workshop "Photogrammetric and computer vision techniques for environmental and infraStructure monitoring, Biometrics and Biomedicine" PSBB23, 24-26 April 2023, Moscow, Russia

Table 3 .
Tested algorithms and parameters.

Table 5 .
Overall accuracy of the tested classifiersOf the best-performing classifiers, RF was chosen because the model is relatively more straightforward and quicker to train.Feature importance analysis selected 11 features out of 28 (Table6), showing strong significance of the Third eigenvalue over the other features, followed by PCA1 and surface variability.

Table 6 .
Importance values of the 11 most important features The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-2/W3-2023 ISPRS Intl.Workshop "Photogrammetric and computer vision techniques for environmental and infraStructure monitoring, Biometrics and Biomedicine" PSBB23, 24-26 April 2023, Moscow, Russia The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-2/W3-2023 ISPRS Intl.Workshop "Photogrammetric and computer vision techniques for environmental and infraStructure monitoring, Biometrics and Biomedicine" PSBB23, 24-26 April 2023, Moscow, Russia