AERIAL IMAGES FROM AN UAV SYSTEM: 3D MODELING AND TREE SPECIES CLASSIFICATION IN A PARK AREA

The use of aerial imagery acquired by Unmanned Aerial Vehicles (UAVs) is scheduled within the FoGLIE project (Fruition of Goods Landscape in Interactive Environment): it starts from the need to enhance the natural, artistic and cultural heritage, to produce a better usability of it by employing audiovisual movable systems of 3D reconstruction and to improve monitoring procedures, by using new media for integrating the fruition phase with the preservation ones. The pilot project focus on a test area, Parco Adda Nord, which encloses various goods' types (small buildings, agricultural fields and different tree species and bushes). Multispectral high resolution images were taken by two digital compact cameras: a Pentax Optio A40 for RGB photos and a Sigma DP1 modified to acquire the NIR band. Then, some tests were performed in order to analyze the UAV images' quality with both photogrammetric and photo-interpretation purposes, to validate the vector-sensor system, the image block geometry and to study the feasibility of tree species classification. Many pre-signalized Control Points were surveyed through GPS to allow accuracy analysis. Aerial Triangulations (ATs) were carried out with photogrammetric commercial software, Leica Photogrammetry Suite (LPS) and PhotoModeler, with manual or automatic selection of Tie Points, to pick out pros and cons of each package in managing non conventional aerial imagery as well as the differences in the modeling approach. Further analysis were done on the differences between the EO parameters and the corresponding data coming from the on board UAV navigation system.


INTRODUCTION
Lombardy, as well as the nation which it belongs to, namely Italy, is characterized by the presence of many places of both landscape and cultural value, such as museums or areas with heterogeneous and distributed goods.However, the management of these areas is typically more complex and it's mainly realized through routes which are predetermined but not binding on the users, usually by leading to a more dispersive visit.The FoGLIE project (Fruition of Goods Landscape in Interactive Environment), funded by Regione Lombardia, began in October 2010 and will be completed in the following two years; it involves some Lombardy firms and Politecnico di Milano, for scientific and technical support in the photogrammetric surveys: these are indeed necessary to produce augmented reality scenes and bird's eye views, besides the widely used 3D visualization.Hence, FoGLIE starts from the need to enhance the natural, artistic and cultural heritage, widespread on the Lombardy's territory, and to produce a better usability of it by employing audiovisual movable systems of 3D reconstruction.At the same time, the project comes also from the need to improve monitoring procedures by making use of new media for integrating the fruition phase with the documentation and preservation ones.By keeping in mind these aims, the UAVs surely represent an innovative and flexible technology, able to collect a big amount of very high resolution information for both geometric and descriptive purposes (generation of DSM/DTM, forest landscape assessment, analysis of critical environmental situations and evaluation of not accessible or not properly enhanced areas, and so on).The results of this test phase will be used as guidelines for next applications on the whole park and for the monitoring of natural and partially urbanized areas through UAVs.This paper, after briefly summarizing the characteristics of the data set, describes the test performed for optimizing the UAV survey planning and for validating the vector-sensor system regarding both the photogrammetric typical aim (3D modeling) and the photo-interpretative one (vegetation classification).Indeed, since the pilot project is assessed on a park area, "Parco Adda Nord", another goal to reach is the identification of nonnative weed plants by means of automatic classification techniques, so that selective cutting down policies can be applied in order to protect the autochthonous tree species.

The test area
As already stated, the FoGLIE project makes use of "Parco Adda Nord" as test area: it's to the south of Lake Como in Lombardy, an Italian region and one of the most urbanized parts of Europe.Following the course of Adda River, the park has a narrow and long shape for a total length of roughly 55 km, with "Parco Adda Sud" (located in Po Valley) as southern limit.It was chosen due to the presence of various goods' types inside it: indeed, it's a suitable site not only for its natural treasures but also for the historical ones, such as the Leonardo's lock, the Paderno bridge and the Crespi working class village, property of UNESCO.The extended forests, which until the end of the 19th century covered the territory surrounding the Adda River, were later strongly reduced by the increasing human communities.Despite this, the park still consists of wetlands and copse woods rich in native vegetation, i.e. alder (Alnus glutinosa), plane tree (Platanus spp.), poplar (Populus spp.), birch (Betula spp.), removing the hot mirror and by setting a filter which allows the radiation, with wavelength greater than 830 nm, to go through it.This change led to yield monochromatic NIR pictures in the blue layer since the other two were saturated.Moreover, the battery was taken away for reducing the Sigma's weight (250 g): hence, the camera was directly connected to the MD4-200 battery, thereby shortening the autonomy of the latter.Other details can be found in Table 2.
2.3: Data generation 2.3.1.UAV survey planning: Although it was tried to apply as much as possible criteria similar to those adopted for traditional photogrammetric flight plans, it was necessary to take into account the differences in platform and sensors.First, only one camera at a time was employed since their concurrent use was not possible due to the MD4-200 low payload: this required the development of a subsequent co-registration procedure in order to work with multispectral information for the vegetation classification.Then, the flight height was kept constant at roughly 55 m (indeed it's better not to go beyond the 60 m because of security issues), thereby determining different Ground Sampling Distances (GSD).Lastly, the low UAV flight range (approximately up to 20 minutes) entailed the need to perform a single flight for each strip, by setting down the MD4-200 and changing its battery.All these elements contributed to planning the flight, characterized, at the end, by three strips with 80% as longitudinal overlapping and 30% as transversal one (for more parameters see Table 3).

Data acquisition and selection:
Data was acquired on July 6, 2011 in a sunny day by performing six separate flights, i.e. 3 for each camera, in the South-North direction; the MD4-200 flew autonomously on the testing area thanks to the automatic pilot function, while the take-off and the landing were remotely controlled by the operator.The UAV position at each shutter release was given by the supplied positioning system and was then recorded in order to provide a first approximation of the camera projection centers and the attitude angles.
For what concerns the RGB dataset, the overall result was satisfactory although some images' attitude was not the usual nadiral one; this probably could be traced back to the following factors: • the MD4-200 is lightweight (585 g), hence the presence of breeze or wind, at the flight height, can produce vibrations or movements; • the control of the camera's attitude, at the moment of the shutter release, seemed sometimes to be problematic and it led to obtain images with inclinations not scheduled within the flight planning.Instead, regarding the NIR data, the images were more blurred than the RGB ones: indeed, the tilt controlled camera mount was specifically built by the owner to carry the modified Sigma DP1 (thus, it's not that originally provided with the MD4-200).Moreover, the blurring effect could also be linked to the long exposure time, chosen in order to capture infrared wavelengths.
Another aspect to be underlined, but concerning both images sets, is that the illumination was not uniform within the single block, because of the time elapsed between the first and the third strips of each of them.Lastly, 42 Ground Control Points (GCPs), pre-signalized and natural (like manhole covers), were evenly placed and their coordinates were measured through GPS receiver (Trimble 5700, L1/L2, in RTK survey) with a precision of roughly 3 cm in planimetry and 5 cm in altimetry.In addition to the UAV images, also some pictures of a calibration panel, placed on the ground, were taken by both cameras: it was a b/w standard square panel, with fiducial marks printed on it and automatically recognized by Photomodeler.The RGB images were masked to remove the background since it slowed down the process, whereas a white balance was applied to the NIR images before starting the calibration procedure.The RGB dataset was composed by 3 strips for a total of 43 images (4000x3000 pixels); among them, 35 were selected on the basis of two criteria: maintaining an overlapping as homogeneous as possible and maximizing the number of visible ground surveyed points.Indeed, the actual overlapping along strip is highly variable with respect to the planned 80%.The same method was also used for choosing the NIR images which, from the original 40 (2640x1760 pixels in 3 strips), were reduced to 19.

First test: PhotoModeler vs. LPS
A first test was carried out by processing the images in PhotoModeler with 16 GCPs, 13 Check Points (CPs) and 218 Tie Points (TPs), all manually selected; this set of TPs was then exported and used also in ERDAS LPS (Leica Photogrammetry Suite) in order to compare the results coming from both close range and aerial photogrammetric software.Residuals on ground coordinates of CPs were compared: even if centimetric accuracies were obtained by both software, better results came from LPS regarding the height coordinate (Tab.4).

Second test:
As second test, it was decided to apply a refined preprocessing of images before triangulating the block in LPS, to enhance both the manual TPs search and the automatic one.First of all, previous experiences (Gini, 2010) proved that, in RGB images created by CCD sensors, the green layer represents the best compromise in terms of contrast, blurring, and object distortion and it was confirmed by some inspections on the current dataset.Thus, each green layer was separated and then used in the different steps which made up the radiometric preprocessing: • the noise evaluation was done on samples, manually chosen in homogeneous areas of each picture; • the noise reduction is a necessary step to be done prior to the contrast enhancement, otherwise noise amplification occurs in parallel to the signal one (Pateraki, 2005).In particular, the method "adaptive edge preserving smoothing" was employed since it removes noise along edges and in flat areas, while maintaining subtle details; • the contrast enhancement acts on less textured zones, thus improving the features definition; the homemade scientific software, developed at Università degli Studi di Parma, implemented Wallis filter, an algorithm which modifies images histogram by changing mean and standard deviation.The gray values' distribution is centered as much as possible with respect to the usual range (0-255), often generating saturation on both sides: of course, the level considered acceptable is subjective but it should not be too much, otherwise objects disappear.Different TPs dataset were employed for subsequent comparison on triangulation results: 1) 272 homologous points extracted by EyeDEA, software developed by the DICATeA department of University of Parma (Roncella et al., 2011a(Roncella et al., , 2011b) ) and based on SURF feature descriptor (Bay et al., 2008, Fischler et al.,1981); 2) 181 TPs manually selected, modified (if needed) and verified on the Wallis-filtered images: they derived from the original set of 218 TPs identified in PhotoModeler (see sec. 3.1).Lastly, also the automatic TPs search modulus of LPS was employed but it turned out to be unsuitable for photos taken on vegetation: indeed, too many points were identified on moving elements such as small leaves, branches or shadows' edges; moreover, random checks showed the presence of a high number of false matching.On the contrary, the EyeDea dataset was not affected by these drawbacks: however, it should be stated that the generation procedure was followed by a decimation one, carried out according the criteria of homogeneous distribution throughout the block and high point multiplicity.

Triangulation and comparisons:
Therefore, the images block was triangulated twice, with 21 GCPs, 18 CPs, 272 and 181 TPs (respectively, the EyeDea dataset and the manual one).Furthermore, an automatic gross error detection algorithm, implemented in LPS, was selected in both cases in order to remove gross errors in TPs and GCPs.Comparisons were performed on the CPs residuals (Tab.6), on the accuracies of TPs ground coordinates (Tab.7) and on the EO parameters' accuracies (Tab.8): the results were good for both methodologies even if, obviously, the combination of the radiometric preprocessing and of the subsequent proper automatic TPs extraction, turned out to be less time consuming.Another remark concerns the non nadiral position of the camera, which can be deduced from the strongly non rectangular shape of the ground footprint of most of the images (Fig. 9), thus giving evidence of the difficulties in accurately reproduce the flight plan.Therefore, the software yielded similar results with the highest differences in Y 0 (along strip axes), and ω (corresponding to pitch), thus bearing out it's not possible to obtain an accurate position of the camera by the on board navigation system.As last experiment, the same RGB image set, with increasing numbers of TPs (935/2790/7764) automatically extracted by EyeDea on the Wallis filtered images, was oriented by employing the homemade scientific software developed at Politecnico di Milano (Calge), and the results confirmed the achievable accuracies of TPs ground coordinates and EO parameters'.

The co-registration procedure
In order to obtain a multispectral dataset usable in a classification process, RGB and NIR images needed to be coregistered: thus, a surface model of the area was required to produce both RGB and NIR orthophotos, thereby to be employed as a four-channel image.Thanks to the dedicated modulus of LPS, a Digital Surface Model (DSM) was created by using the oriented block of RGB images, with a high ground resolution of 0.10x0.10m 2 .Besides outliers near the borders, it's possible to highlight the existence of isolated spikes in the vegetation whose canopy, nevertheless, was generally quite well modeled.A wrong interpolation can be found in the middle of the road (Fig. 11), where the presence of gravel maybe prevented the algorithm from modeling a flat surface, like actually it is.A similar result, in the upper part of the study area, could be ascribed to shrubs, herbaceous plants and overlap of only few images: however, this part was excluded during the mosaicking process since it wasn't represented in the NIR images.Therefore, only some editing were necessary with respect to the purpose, i.e. the vegetation classification.The procedure employed for the RGB dataset was repeated for the corresponding NIR images: due to their low quality, some difficulties were encountered in pointing GCPs and CPs and only few TPs were added.However, the AT was run with the same options of the RGB block (see sec.3.2.3)thus obtaining enough satisfactory results (CPs residuals, accuracies of TPs ground coordinates and of estimated EO parameters), as visible below in Tab. 12, Tab. 13 and Tab.14.After the orientation phase, 35 RGB and 19 NIR orthophotos were generated with ground resolution of 0.05x0.05m 2 by using the DSM previously analyzed.Hence, the orthophotos of both dataset were mosaicked separately thanks to the proper tool of LPS: although it can create seamlines automatically, a manual correction of them was needed to minimize as much as possible any sharp illumination changes among images and strips.Furthermore, it was decided to avoid altering the radiometry by means of implemented methods like the histogram matching or the color balancing: rather it was preferred to keep these variations, provided they were restricted to areas not so interesting for the vegetation classification (e.g. the road).At the end, the two overall orthophotos (RGB and NIR) were resized for having equal extension of roughly 100x85 m 2 (1982x1687 pixels).

Variables creation
Once the original 4 layers (Red, Green, Blue and NIR) were separated, different types of input quantities -let's call them "variables"-were derived from them, in order to improve the tree species' classification, as already done in previous experiences (Gini, 2010).At the moment, a first quick procedure was carried out by computing typical indices (as NDVI) or operations like band ratios, both present in literature in a long time.Certainly, there's the intention of deepening this part, by studying the spectral signatures of the foreign invasive plants: for example, the presence of similar signatures could lead to a misclassification and their separability should be maximized as much as possible.Indeed, through this statistical measure of the distance between signatures for any combination of bands, it's possible to determine the best subset of variables usable for the classification.Another choice was the creation of only radiometric variables: at first, geometrical ones (such as Canopy Height Model, curvature, slope, aspect and so on) do not seem to have a crucial role in classifying trees species, unless these latter have a characteristic and well-defined shape (e.g.cypress) or a particular limited height.Unfortunately, this is not completely true for the species of interest.
Other variables that might be widely explored are those related to textural features because at least one of the species, i.e. tree of heaven (Ailanthus altissima), could be easily distinguishable by the structure of the leaves (as visible in Figure 12).
Figure 12.Leaves of tree of heaven (Ailanthus altissima) The following 6 spectral variables were generated either in ERDAS Imagine or QGIS: • NDVI: the Normalized Difference Vegetation Index is quite powerful in performing a qualitative classification thanks to its capacity to distinguish live vegetated areas from other surface types (water, soil, etc.).It is expressed as the difference between the NIR and the red bands normalized by the sum of these (Silleos et al., 2006); • RATIO: the Ratio Vegetation Index is simply calculated by dividing the reflectance values of the NIR band by those of the R one.High index values are produced by combinations of low red (because of absorption by chlorophyll) and high infrared reflectance (as a result of leaf structure) (Silleos et al., 2006); • Green and NIR ratios: they are obtained by dividing respectively the green and the NIR layer by the sum of all the four bands.Since they are ratios, problems of variable illumination as a result of topography are minimized; • IHS: it's possible to define a color space that uses Intensity (brightness), Hue (dominant color) and Saturation (color's purity) as the three positional parameters in place of R, G, B. After the transformation, the three parameters were separated and only Hue and Saturation were employed, as done in previous experiences (Gini, 2010).Since these different processes influenced both the values range and the data type, it was necessary to uniform all the variables to a common unsigned 8 bit [0-255]: data is stretched or compressed to meet the new interval, but original low and high values are respectively rescaled to new low and high values.This operation was necessary also to stack the 10 variables (the 6 derived and the original 4 layers) and to save them as a single file, without occurrence of merging.

Unsupervised classification
Lastly, it was decided to complete the first quick procedure of classification by running a classic unsupervised algorithm: the main reason was the extension of the study area, too small for taking training and validation samples on plants different from the ones to be classified -as it would have happened for the scrub of tree of heaven (Ailanthus altissima).This is definitely an issue linkable to the used UAV which, due to limited autonomy, payload and flight height, allowed the acquisition of a small area.Hence, the generated layer stack was classified in 10 land cover types by means of the ISOCLASS algorithm (in ERDAS ER Mapper), as visible in Figure 13.As stated in literature, it can be noted that only the classes with deeply different spectral signatures were recognized and separated (i.e. the road, the grass, the bare soil), while the algorithm misclassified all the classes belonging to the same land cover.Thus, the 4 classes assigned to the vegetation are widespread on it and they don't correspond to single species canopies: for instance, the scrub of tree of heaven (Ailanthus altissima) in the upper part, the isolated black locust (Robinia pseudoacacia) in the grass, the plane trees (Platanus spp.) near the small house are misclassified.Furthermore, it can be noted that the shadowed areas were not recognized as a single class but were divided into three different ones.

CONCLUSIONS
In this paper it was described both the geometric and the radiometric processing carried out on data acquired by an UAV.For what concerns the former, it was assessed the orientation quality of a dataset of RGB images by means of the Aerial Triangulation procedure.Then, the results were compared with the UAV position and attitude data, registered during the flight by the low-cost GPS/IMU system on board: even if the navigation unit is necessary to follow a preset flight planning, it is not able to allow a direct orientation.Besides this, other issues (such as the handling of the tilt controlled camera mount and the presence of wind) led to obtain some not nadiral or blurred images: lastly, the datasets were also affected by not uniform illumination because of the time elapsed among flights.Nevertheless, the final blocks were characterized by high resolution (with GSD equal to 1.3 cm and to 2.5 cm) and by enough overlapping to be employed in the subsequent steps.
Both PhotoModeler and ERDAS LPS performed the triangulation of the not conventional photos, by producing good accuracies of the CPs ground coordinates.The standard deviation values of the CPs residuals were of the order of few pixels in planimetry and slightly higher in altimetry due to a disadvantageous ratio between the base and the flight height.At the end, it can be stated that the images acquired by cameras mounted on UAVs came out to be suitable for the 3D modeling purposes, thereby allowing the generation of high resolution DSM.In the radiometric process, the co-registration and the classification procedures required the demanding intervention of the operator to process data, by highlighting the need of automating as much as possible the workflow.Moreover the classification outcomes pointed out how new textural and spectral variables should be derived, with respect to the characteristics of the trees' species of interest; a new dataset, consistent with the first, should be also acquired by UAV to enlarge the study areas.Further efforts will be done to achieve a more effective classification through the maximization of the spectral signatures and the collection of ground truth data in order to use supervised algorithms which are known to provide more accurate results.

Figure 9 .
Figure 9. Triangulated RGB block, with GCPs (triangles), CPs (stars) and the 181 manual TPs (squares)Lastly, the Exterior Orientation parameters were analyzed with respect to the UAV position and attitude data, registered during the flight by the low-cost navigation system on board.As expected, these measurements can be considered only a rough approximation of the actual camera projection centers (indeed the position acquired by GPS code and the attitude carried out from low cost IMU were affected by errors not acceptable for photogrammetric purposes).The EO parameters, estimated both by PhotoModeler and LPS, departed appreciably from the recorded values, even if inside the expected range for the low accuracies of the positioning system (Tab.10).

: The vector-sensor system
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B1, 2012 XXII ISPRS Congress, 25 August -01 September 2012, Melbourne, Australia willow (Salix spp.), oak (Quercus spp.), hornbeam (Carpinus betulus), chestnut (Castanea sativa) and hazel tree (Corylus avellana).Foreign invasive species were then introduced but are now so widespread that periodic selective cuttings should be planned by the Park Forest authority, in order to protect the local ecosystem.These alien plants are: tree of heaven (Ailanthus

Table 5
. Statistics of differences in EO parameters (lengths in meters, angles in degrees)

Table 7 .
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B1, 2012 XXII ISPRS Congress, 25 August -01 September 2012, Melbourne, Australia Statistics of accuracies of TPs ground coordinates [m]

Table 8
. Statistics of accuracies of EO parameters in LPS (lengths in meters, angles in degrees)

Table 12 .
Statistics of CPs residuals [m]

Table 13 .
Statistics of accuracies of TPs ground coordinates [m]

Table 14 .
Statistics of accuracies of EO parameters (lengths in meters, angles in degrees)