Point Cloud Generation from sUAS-Mounted iPhone Imagery: Performance Analysis

The rapidly growing use of sUAS technology and fast sensor developments continuously inspire mapping professionals to experiment with low-cost airborne systems. Smartphones has all the sensors used in modern airborne surveying systems, including GPS, IMU, camera, etc. Of course, the performance level of the sensors differs by orders, yet it is intriguing to assess the potential of using inexpensive sensors installed on sUAS systems for topographic applications. This paper focuses on the quality analysis of point clouds generated based on overlapping images acquired by an iPhone 5s mounted on a sUAS platform. To support the investigation, test data was acquired over an area with complex topography and varying vegetation. In addition, extensive ground control, including GCPs and transects were collected with GSP and traditional geodetic surveying methods. The statistical and visual analysis is based on a comparison of the UAS data and reference dataset. The results with the evaluation provide a realistic measure of data acquisition system performance. The paper also gives a recommendation for data processing workflow to achieve the best quality of the final products: the digital terrain model and orthophoto mosaic. After a successful data collection the main question is always the reliability and the accuracy of the georeferenced data.


sUAS and sensor hardware
The sensor platform was an IRIS, four-rotor small UAS made by 3DRobotics, California.Its maximum payload is 400 grams and the flight duration is about 10-15 minutes with fully charged battery.The imaging sensor was the camera of an iPhone 5s mounted with a vibration absorbing frame; the image size is 3,264 x 2,448 with 1.5 µm pixel size.The 4.1 mm focal length lens has f=2.2 aperture.

The test field
Two important aspects were taken into account when choosing the suitable test field: • High topographical variety of terrain • Field with vegetation challenges A small, ca.5,000 square meters, old, abandoned mine outcrop was selected to meet the objective of this study.Steep slopes provide the topographical variety, and parts of the area are covered by grass and small bushes.

Ground control points
Nine ground control points (GCPs) were surveyed for accurate georeferencing by using RTK GPS.The horizontal and vertical accuracy are about 2 cm and 2cm, respectively.WGS-84 UTM Zone N10 coordinate system was chosen.

Point cloud processing software
For point cloud generation, the Pix4D software was used.The Pix4D software has been widely used in the sUAS community to process huge number of images into geo-referenced point cloud, digital surface model, and orthophoto mosaic.

Processing workflow
Figure 1 shows the typical input and output data of the Pix4D software.Since the user has no control on the filtering function in Pix4D, the raw "densified" point cloud was used in this investigation.Next, the noise filtering and key pointing were done in the TerraScan software.The key pointing was set to 20 cm.

Figure 1. Inputs and outputs in Pix4D
Statistics on the final point cloud is shown in the

Height differences
The statistical analysis is shown in Table 3, a summary of the height differences between the 141 ground points and their projection on the TIN01 model.The projection was done in TerraModeler; note the interpolation method is not specified in the manual of the software.

Height differences vs. absolute height
Figure 4 shows the height differences with respect to absolute height.Note the trend in the plot.

Volume difference
The volume calculation was done on the intersection of the TIN02 and TIN03 surfaces, using AutoCAD Civil3D software environment.The results are listed in Table 4.
Table 4. Volume calculation, TIN03 -TIN02 Figure 5 is a visualization of the height differences between the points measured with RTK GPS and their vertical projection on the TIN01 model.Figure 6 shows height differences between the GRID02 and GRID03.

The mean value
The mean value of height difference on all points shows a ca.7 cm below zero height offset.This number itself would mean that the model based on Pix4D points is shifted above the reference model.

"Height difference vs. absolute height" interpretation
Almost all those points with positive value on height difference, in other words the height of the points projected to the TIN01 are lower than the originally measured points, are at the lower level of the test area.The regression line, marked by dotted blue in Figure 4, clearly shows this tendency; the Pix4D points at higher level are more likely above the reference model, while the Pix4D points at lower level are below the reference model.
Drawing a conclusion only from this statistical analysis would probably produce a false result, such as the model based on the Pix4D point cloud may have a vertical stretch.However, there are other environmental factors, such as vegetation, which must be considered too.

Interpretation of the "cut" and "fill" volume results
For being able to interpret the volume values given by the differences between the TIN02 and TIN03 surfaces, the reliability of the volume calculation against the geodetic measured model must be known.Let's consider the geodetic model (TIN03) as a reference model.A good approach is to calculate a "reliability volume" based on the confidence interval of many-repeated aerial Z measurement on a single triangle of the reference TIN model.A plane of a triangle (a triangle is considered as a plane for this model) is determined by 3 measurements.Thus, dividing the number Z measurements by three will result in the number of independent determinations of a triangle plane.This number can be considered also as manyrepeated Z measurements on each single point.Using this number of independent measurements and the STD value of a single Z measurement, the 99% confidence level can be calculated.In worst-case scenario, the 3 points move in the same Z direction to the edge of the confidence interval giving the maximum volume deviation.This way, the XY projected area of a triangle and the confidence interval determine the "reliability volume" for a single triangle.Summarizing all the "triangle volumes" gives the "reliability volume" for the whole test area.Note that this second approach is a very conservative, since it is very unlikely to have all the points moved in the same direction; more precisely, the chance is one to the half of the two of the power of the number of the points, which is 1:256 in this case.
Comparing the reliability volume and the "Net" volume (Table 4) given by the variety of the TIN02 and TIN03 models, we can conclude that the modeling based on UAS measurement give us, at least, as reliable volume calculation as the traditional method.However, looking at the "cut" and "fill" numbers, it is clearly seen that the UAS data describes the variety of the terrain more accurately (more detailed) than the geodetic method.

Visual interpretation of Figure 5
Visualizing the points on the orthophoto of the test area makes the interpretation easier.The following statements and conclusions can be drawn based on Figure 5: • Most part of the upper level ground is covered by grass.The grass has a significant impact on the height of the point cloud acquired by using imagery, clearly explaining the offset between the two method's data and reduces the presumption of the stretching model theory.
• Most of the points with positive height difference (blue dots) are really on the lowest level of the area, where there is no grass.The points without the impact of grass coverage have better chance to have positive height difference; this fact, again, reduces the presumption of the stretching model theory.

•
The area with the points on the lower level with significant negative values (big red dots) is also covered by grass.The significant positive values on the low level points would reduce the presumption of the stretching model theory, so to a less extent.
• Most of the points with positive height difference are concentrated on a small area (on the south of the test area), and might be explained with a local anomaly, directing attention to another type of possible error of this method.

CONCLUSIONS
This paper analysed the performance of a low-cost sensor and platform based solution for aerial topographic data acquisition.
An iPhone installed on a sUAS platform acquired imagery over complex terrain, and the Pix4D software produced the point cloud for the analysis.The test area was extensively surveyed to provide absolute control for the investigation.
The results of the performance evaluation clearly indicate that the imagery acquired by this inexpensive system is able to provide derived products that in accuracy are comparable to traditional airborne surveying performance.Thus, except for productivity, this simple system is surprisingly competitive to conventional high-performance sensor equipped airborne systems.
Despite the lower quality of the imaging sensor, due to the high redundancy of measurements, the system is capable for acquiring high detailed topographic data; in fact, the final results appear to be, at least, as reliable as the geodetic methodbased reference.
During this study, various data processing methods were used, and based on this experience, the workflow, shown in Figure 6, is recommend to achieve the most reliable products, such as digital terrain model and ortho photo mosaic.
Figure 2. TIN03 model: based on point cloud from UAS

Figure 5 .
Figure 5. Surveyed points' height vs. TIN01 model Thematic maps help visual analysis of the height difference between the two datasets collected with different methods.Figure5is a visualization of the height differences between the points measured with RTK GPS and their vertical projection on the TIN01 model.Figure6shows height differences between the GRID02 and GRID03.

Figure 6
Figure6visualizes the differences of the topographic variety based on the GRID02 and GRID03 surfaces.Note the high height differences at the cliff walls, which are the result of the more detailed aerial survey against the rough generalization of the geodetic survey.The statistical values of this comparison are shown on Table7.Note the small mean value of the height differences; it is within the accuracy level of the GCP measurement.Due to the rough generalization of the geodetic measurements, the minimum, the maximum, and the standard deviation values are relatively high.

Table 1
. Flight plan settings 258 of 1,308 exposures were used for the subsequent calculations.

Table 2 .
Statistics of final point cloud

Table 3 .
Statistics on height differences

Table 5 .
Table5shows the results of the "reliability volume" calculation."Reliabilityvolume"calculationUsingthis 99% confidence level may be too optimistic, since the RMS of the Z error at the GPCs in the Pix4D report is given as 0.03 m.Also, our statistical analysis shown in Table3gives a 0.05 m STD value at the GCPs.Using our STD value results 257 cu m as "reliability volume", see Table6.

Table 6 .
"Reliability volume" calculation based on STD value of GCPs

Table 7 .
Note the small mean value of the height differences; it is within the accuracy level of the GCP measurement.Due to the rough generalization of the geodetic measurements, the minimum, the maximum, and the standard deviation values are relatively high.