ASSESSING THE MEASUREMENT QUALITY OF UAV-BORNE LASER SCANNING IN STEEP AND SNOW-COVERED AREAS

: Avalanche monitoring in the Norwegian mountains has potential for preventing disasters and informing hikers. Using a Unmanned Aerial Vehicle (UAV)-borne laser scanner, this work proposes a protocol to assess the quality of point cloud data for monitoring avalanche risks. Roof models are used as control planes to analyze the collected points. The distances to the control planes are used to investigate error measures, and inliers and plane equations are compared with the roof model dimensions. These parameters provide insights into the reliability of the point cloud data. The comparison shows the impact of flight speed and altitude on accuracy. While varying flight speed does not affect error measures, both speed and altitude significantly affect the number of collected points. Point coverage is concentrated near the top of the roof models, resulting in a calculated model volume that is only 50% of the expected value. Comparing the Random Sample Consensus (RANSAC)-derived plane equations with the roof model planes reveals that more points are collected above the roof model planes. The standard deviations of the inliers range from 0.011 m to 0.023 m, and the root mean square error (RMSE) ranges from 0.060 m to 0.019 m. These findings indicate the reliability of UAVs for monitoring steep and snow-covered areas without the need for reference points to correct positions.


INTRODUCTION
Avalanches in the Norwegian mountains have devastating consequences due to the difficult terrain.To address this, monitoring high-risk avalanche areas and providing real-time risk alerts is crucial.UAVs can collect data in challenging terrains like the Norwegian mountains.This project focuses on using a LiDAR sensor mounted on a UAV to collect point clouds from the snow surface, enabling the detection of snow height and structural changes over time.Challenges remain in assessing data quality and understanding the limitations of UAV-borne laser scanners.Accurate measurement of data is essential, and understanding the impact of UAV parameters on data quality is vital.The RANSAC algorithm is used to assess the quality of point clouds in steep regions, and plane equations are developed for precise identification of steep areas.

Flight Parameters
The experiment is carried out across a number of sessions with varying altitude and flight speed conditions.The flight route is the same for all sessions.There are four different flying speeds: 3, 5, 8, and 10 m/s.The heights are 40 and 80 meters above the ground.To account for outside influences on the experiment, the first session is repeated with the same settings in the final session.This is shown in Table 1.

Arrangement of Roof Models
In Figure 1 a bird's-eye perspective of the leveled tripod setup is shown.Roof models are attached after measuring their positions with an Real-time kinematics (RTK) Global Navigation Satellite System (GNSS) rover.Pairs of roof models are arranged in a straight line.The fifth roof ridge is pointed at a 45°a ngle from the midpoint of the third roof ridge.
Figure 1.A proposed configuration of roof models

Data Analysis
The (LiDAR Control Plane) LCP, a signal representation of the roof models, position and UAV point cloud are analyzed in the experiment.LCP locations are verified and compared to LiDAR point clouds.Visual validation and point cloud analysis are necessary.Measurements of extracted points are contrasted with RANSAC predictions.Error measurements are carried out for inliers and compared to the plane equation.

Positions of LCPs & Construction of True Plane
During the experiment, the RTK rover calculates each LCP's position.They are computed four times before and twice after each flight.Before being used as true points, their validity is assessed.
Figure 2 depicts each point establishing the limits of the roof models and mid points.The true plane is constructed by establishing vectors for the two roof slopes using the previously calculated true points.

Point Cloud Error Measures Compared to True
Plane The precision of the position of LiDAR points in a point cloud is described by calculating the perpendicular distance to the real plane.To compute the left and right plane equations, each point in the point cloud is separated into left and right portions.The points are further divided into points above and below the plane using positive and negative distances.The perpendicular distance is calculated using the equa- , where Q is a point at (x0, y0, x0) and the plane P is specified by an equation.
The minimal distance represents how far the points are gathered below the actual plane, while the maximum distance measures how far the points are spaced from the LCP above the plane.The median and MAD (Median Absolute Deviation) are used to identify overweight points and assess the spread of points from the median.
The absolute perpendicular distances to the roof models are used to calculate the arithmetic mean, standard deviation, and RMSE.These calculations help evaluate the precision and distribution of LiDAR points in a point cloud.

RANSAC Algorithm on Point Cloud
The RANSAC algorithm is used to locate inliers on each side of the LCPs.This allows for error calculation by comparing the estimated plane equation with the actual plane.The total number of inliers on each LCP is analyzed to understand their distribution above and below each plane.A new plane equation is created based on the inliers, and its parameters are compared to the real plane equation.The smallest Bounding Box (BBox) of the inliers and the projection of the plane's points determine the bounds of the plane.A plot is used to visualize the planes and inliers.

Inlier Error Measures & RANSAC Plane
The analysis includes calculation of various metrics using the original point cloud and inliers found by the RANSAC algorithm.These metrics include minimum and maximum distances from the plane, mean and MAD of perpendicular distance, arithmetic mean, standard deviation, and RMSE of absolute perpendicular distance.The accuracy is assessed by comparing these metrics and the angle between the RANSAC planes and the actual plane.The midpoint of the inliers is determined using the minimum BBox borders.The volume of the RANSAC planes is compared with the volume of the LCPs by calculating the convex hull of the shape formed by the borders.

Equipment & Constructed True Plane
A DJI Matrice 300 UAV equipped with an RTK receiver is the apparatus being used for the experiment.The Norwegian Mapping Authority's (NMA) CPOS is coupled to the RTK.The Zenmuse L1 laser scanner is selected, and its specs are listed in (DJI Enterprise, 2022).For the comparison, known points are collected from tripods in the field using the Leica GS16 RTK rover.
A list of rover specifications is provided in (Leica Geosystems, 2023).
The roof sides of the built-in roof models have diameters of 40 cm and 80 cm and an angle of 90°between them.They have a base area of 0.45 m 2 , a slope of 45°, and a volume of 0.06 m 3 .Aluminum and wood are used materials to make LCPs.Lumber and water-resistant wood sheets make the roof slopes and a screw attachment is included underneath for stability.In order to maintain the same position for each roof model during all of the sessions, they are screwed to tripods. Figure 3a depicts the screw's aluminum attachment.
The DJI Terra software provides the anticipated point densities for the sessions, which are reported in To ensure that the computations of the genuine points are appropriately calculated, GISLINE Land is employed.The screw's attachment point and the top of the roof model are separated by 0.262 meters.From a baseline through the ridge of the roofs, the LCP 1 and LCP 4 are pointed in the same general direction.With a baseline via the roof ridge, LCPs 2 and 3 are likewise facing the same way.LCP 5 is angled 45°s in the direction of the center of LCP 3. Figure 3b depicts the placement of the LCP on a tripod and the alignment of the roof ridges.Figure 2 shows the outcomes of the calculation of the true points yielding eight points.

Point Cloud Data Results
The program CloudCompare is used to separate and isolate the point clouds belonging to each LCP.The segment tool is used to further separate the points into left and right side of the roof model.
The point values are verified to be as expected using the height in NN2000 and the EUREF89 UTM 32 reference frame.The points are not pre-processed point clouds or GPS-corrected points.The purpose of not correcting them is to make it easier to evaluate the points exactly as they are and to instantly verify their accuracy.In the CloudCompare program, the sessions S2 and S7 are contrasted visually.Both cases have used the same point size for the visualization.Figure 4a and Figure 4b show how the variations are different.
Table 5 presents the results of the actual detected points on each LCP during each session from the points extracted using the software CloudCompare.It lists the total number of points (T), the number of points on the left (L), and the number of points on the right (R).The points for sessions S4, S5, S6, and S7 either lacked a clear framework to divide them into left and right, or they were impossible to identify.S2 is the session where the most points were gained.The roof model with the most points accumulated among all roof models is LCP 5.With a flight altitude of 40 m above ground level, S8 is the session with the fewest points collected per LCP.Table 6 lists the point density on the plane as the arithmetic mean of the five LCPs from each individual session.No valid values were obtained for all sessions with a flying altitude of 80 m above ground level due to the absence of observed roof points.
Along with the genuine planes created from the equations of planes with true points as borders, the extraction of points from each LCP is visualized.Figure 5 and Figure 6 show the visualizations from LCP 1 and LCP 5 from S2.The axes are not orthonormal.
Table 7 displays the distribution of points from the point cloud above and below each true plane for every session.The LCP 5 displays a greater proportion of points above the true plane, following the generally observed pattern.Only for S2 on LCP 3, S3 on LCP 3 and LCP 4, and S8 on LCP 4 are more points from the point cloud seen below the true plane.
The perpendicular distance to the true plane is determined using the results from Table 7.The maximum above point is listed in Table 8 as the greatest distance.The same table lists the greatest value below as the minimum distance.For each distance measured in each session, the MAD and median of the distances are shown.Amongst all sessions, a maximum value of 0.139 m in S1 for LCP 3 is identified.In S2 for LCP 3, the lowest figure for all sessions is -0.157 m.The median value ranges from -0.001 m for S3 on LCP 3 to 0.054 m for S1 on LCP 5.The MAD ranges from 0.014 m in S3 and S9 for LCP 5 to 0.038 m in S1 on LCP 4.
Figure 7 presents the arithmetic mean of the absolute values

RANSAC Estimations of Point Cloud
The RANSAC estimated point cloud findings are reported in this section.The RANSAC method is used to calculate inliers and the plane equation that is created from the inliers of the planes.Additionally, the results of calculations for volume, angle, midpoint, and error measurements are shown.A residual threshold of 0.02 is used for the RANSAC algorithm.In Table 9, it is stated how many inliers are recovered by the RANSAC algorithm at each LCP from every session.For LCP 5 in S2, 196 points are extracted, which is the highest number of inliers.When compared to the other LCPs, the LCP 5 has the biggest number of points in each session.S8 is the session with the fewest inliers.The LCP 2 garnered 40 points in S8.
Therein, the least number of points were extracted.The Table 10 contains the distribution of the inlier points both above and below every true plane for each session.For LCP 1, LCP 2, and LCP 5, zero points below the real plane are found during S1.This also happens for LCP 5 in S9.Low numbers of points discovered below the real planes are seen in both S1 and S9 data.A minimal number of points, up to a maximum of 6, are obtained for LCP 5 in all sessions but S2.The sole instance where more points have been gathered below the true plane than above is in LCP 3 in S2.
Table 11 displays the planes' derived equations from RANSAC S2.There are two planes (left and right) on each roof model.According to the illustration in Figure 1, these are identified.All of the left plane's coefficients are negative, as are all of the plane equation's c coefficients.
Figure 10 shows the planes for S2 whose borders are determined by the minimal BBox of the inliers.The inliers that are retrieved from each side are displayed in blue and magenta.The  Table 12 lists the outcomes of the RANSAC inliers both above and below the true planes.In Table 12, the maximum distance is shown as the highest point above.The same table lists the greatest value below as the minimum distance.For each distance measured in each session, the MAD and median of the distances are shown.For all sessions, the maximum value is 0.095 m in S1 for LCP 5.For all sessions, the lowest figure is -0.079 m in S8 for LCP 4. The median value ranges from -0.002 m from S3 on LCP 3 to 0.056 m for S1 on LCP 5.The MAD ranges from 0.009 m in S8 for LCP 5 to 0.035 m in S1 on LCP  The RANSAC inliers' perpendicular distances to the true plane are calculated, and the arithmetic mean of those absolute values are determined.Figure 11 presents these findings.For S1, the highest mean values are derived.The greatest mean value is 0.055 m for LCP 5 in S1.The lowest value throughout all sessions is in S3, when LCP 2 has a mean value of 0.014 m.LCP 3 have the lowest value in S1, S3, S8, and S9.The highest mean values of LCP 5 are found in sessions one and nine (S1 and S9).
Figure 12 displays the standard deviation determined using data on the absolute perpendicular distances between RANSAC inliers and true planes.LCPs 4 and 5 in S1 had the largest standard deviations of all LCPs at 0.023 m.In S2 of LCP 2, the lowest value is reported.
Figure 13 displays the results of the RMSE of the absolute perpendicular distances from each LCP to each RANSAC inlier for each session.For LCP 5 in S1, the highest RMSE is recorded, 0.060 m.All RMSE values for S2, S3, and S8 are less than 0.040 m.LCP 2 in S2 has the lowest value at 0.017 m.
Table 13 provides the angle between each RANSAC plane in each session.A 90°angle is present in three of the RANSAC   The Figure 15 contains each midpoint's difference.The green hue represents the E-direction, the blue height, and the red Ndirection difference (NN2000).The differences are each indicated in metres.The height value for each difference is negative, suggesting that it is higher than the true plane's midpoint.LCP 1 and LCP 2 have the greatest impact on the N-direction in S1, S2, S8, and S9.LCP 1 has the greatest N-direction divergence  Table 14 displays the volume determined from the RANSAC plane borders.The RANSAC algorithm provide the maximum volume for roof model 5 for S2.The roof model 4 S8 calculation yields the lowest volume.Of all the roof models, the RANSAC roof model 5 has the biggest average volume.

DISCUSSION
The flight and true points measurement were taken on the same day under cloudy weather conditions.The temperature during the measurements falls within the accepted range stated by DJI and Leica.Good geometry is expected with over 30 satellites present.Conducting the experiment in a field eliminates signal reflection, but ionospheric activity during the night and early morning can affect the results, especially vertically.The tripods used for measurements were placed deep into the snow, but there is still a possibility of movement during flights, resulting in higher standard deviation measurements.The results meet the expected accuracy of the RTK GNSS CPOS measurement and Norwegian standards.
The accuracy of the built roof models is between 1-2 mm, which may slightly impact baseline calculation and constructed plane equations.However, the true plane equations yield expected results.The coefficients for the left and right planes are the same, with only a notation change.The d coefficient may vary due to roof model arrangement and dimensions.
When visualizing the point cloud in CloudCompare software, points collected on roof models are easily identifiable.However, some points were collected below the surface or tripod, affecting further calculations.Points collected at an altitude of 80 m above ground level do not resemble roof models and appear as separate objects.
The number of detected points on each LCP varies.For some sessions, the actual detected points are slightly over half of the expected points.No points are detected as roof models for sessions at 80 m altitude.These findings confirm that parameters like speed and altitude can impact results.Point clouds from 80 m altitude sessions are not used for error measures.
The expected amount of points is the same for sessions with different parameters, but the actual results differ.LCP 5 has a higher number of points, possibly due to its proximity to the UAV-borne LiDAR and LCP angle.Further analysis is needed to understand how values in CloudCompare are referred to the true plane and confirm assumptions about the point cloud and true plane together.The collected point cloud data aligns with a study on LiDAR workflow for snow depth detection in the Swiss Alps (Koutantou et al., 2021).However, point density in steep areas is lower than expected, impacting detailed information on roof models.Only point clouds with enough points to detect all roof models are used.Some points appear shifted to the right, possibly due to inaccuracies in the division process or offsets in the point clouds.
Table 5 shows a generally similar distribution of points on each side, with the highest difference being 29 points in S2 for LCP 2. Other differences are around 10 points, as expected.The UAV's northward movement between landings may contribute to the observed errors.Similar errors have been reported in the Zenmuse L1 LiDAR scanner and other studies, suggesting offsets.(Troner et al., 2021) (Ekaso et al., 2020).
Table 7 summarizes the distribution of points above and below each LCP for each session.Most points are detected above the LCP, except for LCP 3 and LCP 4 where more points are detected below for at least one session.This could be due to an offset since the points are not GPS corrected.
The arithmetic mean values in the first and last session are slightly higher, but there is no significant difference between the sessions.LCP 5 has higher values for three sessions because it has the most points collected.The arithmetic mean results can be compared to the result of the arithmetic mean of 0.034 m by (Haala et al., 2022).The values in the first and last session with a speed of 5 m/s and altitude of 40 m above ground level are above 0.034 m, but the other values are below it.The standard deviation from the distances to the true plane is below 0.0300 m, indicating higher accuracy of the points in the point cloud than expected.However, there is no significant difference between the sessions.The RMSE values show that the first and last session have higher values.For LCP 5, the RMSE value at S1 is 0.06 m, but the lowest values are below 0.03 m.These values compared to the 0.0356 m value given by (Haala et al., 2022) indicate a bad fit, although some values are below it.The investigation on the accuracy of Zenmuse L1 conducted by (Troner et al., 2021) shows an accuracy value of 0.036 m in all directions with an altitude of 50 m before transformations, which is lower than the largest value of accuracy derived from this project.There is no significant difference between the sessions.
The initial point cloud's error measure does not show any significant difference when comparing sessions with different speed parameters.This aligns with a study on speed parameters for UAV using an RGB camera instead of a LiDAR sensor (Ekaso et al., 2020).Although there is no significance, the number of points can still indicate the level of detail that can be detected during each flight.The inliers of the point cloud on LCP 5 have the highest number of points, which is expected since LCP 5 has the most points in the original point clouds.The distribution of inliers above and below the true plane mostly consists of points above the plane, except for LCP 3 in S2.In this specific session, there seems to be an offset, with the majority of points below the plane.The RANSAC estimated plane equations of the inliers differ significantly from the true plane equations.For LCP 5, the inliers appear to be more precise in recreating the roof model.In S8 with the highest speed, the lower point density is evident but does not seem to impact the ability to create the RANSAC planes.The results of the minimum and maximum values of the inliers on the true plane indicate a higher tendency to choose points that are higher above the planes.The estimations result in a lower MAD value compared to the initial point cloud.The arithmetic mean of the absolute distance of the inliers to the true plane shows similar results to the mean of the initial point clouds.There are some noticeable differences, with lower values obtained for LCP 1 in each session.
For the standard deviation of the inliers, the values compared to the initial point cloud are lower or close to the same value.The RMSE values of the RANSAC inliers are all lower or close to the same.For S2 and S9, the values have noticeably improved at each LCP, but still not better than the accuracy reported by (Haala et al., 2022) or (Ekaso et al., 2020).The RMSE from (Troner et al., 2021) shows an accuracy of 0.040 m and 0.160 m derived in steep areas.The GEOSFAIR project conducted by NRPA shows an accuracy of 0.052 m and 0.155 m (Frauenfelder et al., 2022).The conclusion of the GEOSFAIR project is that the values are within an acceptable range of quality for further use.
The RANSAC planes constructed from the inliers allow for a comparison of the angle between the true plane and the RANSAC plane.The expected angle is 90°.However, LCP 5, which has the most points collected, does not have the best achieved angle.Instead, it has a more stable angle compared to the other sessions.The largest angle observed is 131°.When comparing the mid-point of the inliers with the mid-point on the true plane, it is evident that more points are collected at the top of the true planes.This leads to the reconstruction of a smallersized plane with a higher mid-point compared to the true plane's mid-point.For some LCPs, the mid-points are shifted further in the north direction.The expected volume is 0.06 m 3 , but due to the points being collected only at the top of the LCPs and the high mid-point on all LCPs, the actual volume is less than half of the true plane volume.Among the sessions, LCP 5 consistently has the highest volumes, which can be attributed to its ability to collect the most points and achieve a higher volume on the lower part of the LCP.
Determining position and ranging accuracy is challenging due to difficulty in separating errors.Errors are likely caused by both factors.Point cloud movement indicates systematic shift and suggests position inaccuracies.Point height above LCP can be attributed to accuracy or ranging error.Point roughness reflects ranging accuracy.

SUMMARY & CONCLUSION
The main goal of this project is to assess the accuracy and reliability of UAV-borne LiDAR technology in steep and snowcovered areas.The project aims to develop a protocol for evaluating the quality of LiDAR monitoring in steep areas, determine the impact of speed and altitude on accuracy, and assess the feasibility of using the protocol for measuring steep and snowcovered areas.
The experiment involves arranging five LCPs on a field with a snow surface.True points of the LCPs and point clouds from nine flight sessions are collected and analyzed.The results include calculations of median and MAD of distances above and below the true plane, as well as mean, standard deviation, and RMSE of absolute distances.The angle, mid-point, and volume of the RANSAC fitted planes of the inliers are also presented.
The discussion of the experimental results highlights the analysis of accuracy in steep areas using LCPs.The speed parameter of the UAV does not significantly affect error measures, but the number of detected points indicates the level of detail achievable.Surprisingly, an altitude of 80m above ground level has a low possibility of detecting objects compared to the expected number of points.Higher speed or altitude decreases the level of detection.
The hypothesis that UAV-borne LiDAR can be used for monitoring steep and snow-covered areas is confirmed, and a protocol for assessing the quality of UAV-borne laser scanning is achieved.However, if higher accuracy is required, the designed approach may not meet the requirements.
The comparison of points collected on different LCPs suggests that the arrangement of LCPs should be tested with different approaches.The direction of the flight path and the angle of roof ridges may affect the number of collected points.Additionally, using an automatic approach or algorithm for point extraction from roof models would be beneficial to improve accuracy.
Further experiments should be conducted with different height parameters below 80m and with LCPs of greater area.Testing different slopes, including lower angles, and comparing measurements with and without snow on the roof models are also recommended for future studies.
Overall, the project successfully investigates the accuracy and reliability of UAV-borne LiDAR technology in steep and snowcovered areas and provides insights into the feasibility and limitations of using this technology for avalanche monitoring.

Figure 2 .
Figure 2. True points in the direction of the baseline The top position is initially established by adding the distance from the tripod to the top of the roof model in order to locate the points measured with the RTK GNSS rover and actual points of the roof model.The findings of the tripod position measurement are then used to create genuine points by orthogonal measurements.
Figure 3. Roof model construction used as signals for point cloud collection

Figure 4 .
Figure 4. Point cloud from session 2 and 9 with the flight speed 3 m/s or 10 m/s and altitude 40 m or 80m above ground

Figure 7 .
Figure 7. Mean values [m] of S1, S2, S3, S8, and S9's absolute perpendicular distances from their actual planesIn Figure8, the standard deviation derived from the data of absolute perpendicular distances is presented.For LCP 4 in S1, the largest standard deviation value is 0.028 m.LCP 4 in S3 has the lowest value, 0.014 m.

Figure 9
Figure 9 displays the findings regarding the RMSE of the absolute perpendicular distances The largest RMSE are shown in S1, where LCPs 1 and 5 are 0.054 m and 0.060 m, respectively.All other RMSE findings are ¡ 0.05 m, except for LCP 5 from S9.For LCP 3 and LCP 4 in S3 and LCP 3 in S8, the lowest value is 0.027 m.

Figure 8 .
Figure 8.Standard deviation [m] of the absolute perpendicular distance to the true plane of S1, S2, S3, S8, and S9

Figure 10 .
Figure 10.RANSAC planes built from inliers from session 2, where the orange-colored planes are the planes that RANSAC calculated and the red points are the borders.The inliers provided on each side of the roof model are the blue and magenta points.

Figure 14
Figure 14 shows the midpoints of each set of RANSAC inliers in S2.The actual plane's midpoint is shown in green, and the midpoint is shown with the magenta colour.The true planes are shown with red points denoting the true points of the true plane and are coloured orange.The mid-point of the RANSAC plane are always relatively higher than the mid-point of the true plane.The positions for a few of the RANSAC midpoints are shifted left (north).

Figure 15 .
Figure 15.RANSAC mid-point and true plane mid-point on true plane session 2, where north-difference, east-difference, and height difference are indicated in red, green, and blue, respectively.
done as a Master's Thesis and as part of the Master's degree programme in Engineering & ICT in the topic of Geomatics at the Norwegian University of Science & Technology (NTNU) in Trondheim.The topic of the thesis was developed in partnership with the Norwegian Public Road Administration (NPRA) and the research project for Geo-hazard Survey From Air (GEOSFAIR), which is financed by the NPRA and the Norwegian Research Council.The study looks into the usage of unmanned aerial systems for avalanche monitoring and the risk of avalanches damaging Norwegian public roadways.

Table 1 .
Sessions of experimental UAV flight

Table 2 .
Expected collected point density for each UAV flight session

Table 3 .
Measured location of the tripods in[m]

Table 5 .
Points detected on each LCP in each session

Table 6
. Actual points collected in each flight session relative to the predicted points per LCP

Table 7 .
Number of points from the point cloud at each session above (A) and below (B) the true plane

Table 8 .
The true plane's actual upper and lower limits for the perpendicular distance.The perpendicular distances' median and MAD

Table 9 .
RANSAC algorithm's estimate of the number of inliers

Table 10 .
RANSAC inlier points per LCP at each session, above (A) and below (B), the true plane

Table 11 .
Plane equations from RANSAC inliers in S2 for each side of the roof

Table 12 .
The minimum, maximum, median, and MAD of the distance between inliers above and below the true plane

Table 13 .
Angle [°]between two planes created through the RANSAC algorithm

Table 14 .
Volume [m 3] below each roof model created at each session using the BBox bounds and RANSAC inliers