GEOMETRIC AND REFLECTANCE SIGNATURE CHARACTERIZATION OF COMPLEX CANOPIES USING HYPERSPECTRAL STEREOSCOPIC IMAGES FROM UAV AND TERRESTRIAL PLATFORMS

Light-weight hyperspectral frame cameras represent novel developments in remote sensing technology. With frame camera technology, when capturing images with stereoscopic overlaps, it is possible to derive 3D hyperspectral reflectance information and 3D geometric data of targets of interest, which enables detailed geometric and radiometric characterization of the object. These technologies are expected to provide efficient tools in various environmental remote sensing applications, such as canopy classification, canopy stress analysis, precision agriculture, and urban material classification. Furthermore, these data sets enable advanced quantitative, physical based retrieval of biophysical and biochemical parameters by model inversion technologies. Objective of this investigation was to study the aspects of capturing hyperspectral reflectance data from unmanned airborne vehicle (UAV) and terrestrial platform with novel hyperspectral frame cameras in complex, forested environment.


INTRODUCTION
Recent progress in miniturized hyperspectral imaging technology has provided the markets with hyperspectral cameras operating with frame imaging principle (Mäkynen et al., 2011;Saari et al., 2011;Honkavaara et al., 2013;Aasen et al., 2015).With frame camera technology it is possible to derive 3D hyperspectral reflectance point clouds and 3D geometric data of targets of interest, when capturing images with stereoscopic overlaps.This will allow detailed geometric and radiometric characterization of the object.
Novel hyperspectral imaging technology based on a variable air gap Fabry-Perot interferometer (FPI) was used in this investigation.The FPI technology makes it possible to manufacture lightweight, frame format hyperspectral imager operating in the time-sequential principle.The first prototypes of the FPI-based cameras were operating in the visible to nearinfrared spectral range (500-900 nm; VNIR) (Saari et al., 2011;Mäkynen et al., 2011;Honkavaara et al., 2013).The FPI technology is also commercially available in the VNIR range (http://www.rikola.fi).Similar to conventional cameras, these sensors can be operated from terrestrial and airborne platforms using photogrammetric principles, capturing image blocks with stereoscopic overlaps.Efficient and accurate data postprocessing is required to transform these hundreds and thousands of images into products that allow the objects' geometric and spectral characteristics to be interpreted on a quantitative geometric and radiometric basis.The modern computer vision and photogrammetric techniques based on structure-from-motion image orientation techniques (Wu et al., 2013) and dense digital matching generating accurate 3D point clouds and digital surface models (DSM) (Leberl et al., 2010) offer efficient tools to process the data sets.This paper presents new instrumentation and processing chain for 3D geometric and multiangular reflectance characterization of complex forest canopies; the research augments the recent developments by Hakala et al. (2013), Honkavaara et al. (2013;2014;2016), Tommaselli et al. (2014) and Liang et al. (2015).The instrumentation can be operated from UAV and terrestrial platforms.Our objective is to study the aspects of extracting hyperspectral reflectance data with the novel instrumentation in forested environment, as well as the uncertainty modeling.

Multiangular 3D reflectance data capture using FPI camera technology
The hyperspectral frame camera technology based on the use of multiple orders of a variable air gap FPI is used in this study (Saari et al., 2011;Mäkynen et al., 2011).When the FPI is placed in front of the sensor, the wavelength of the light passing the FPI is a function of the interferometer air gap.By changing the air gap, it is possible to acquire a new set of wavelengths.During data collection, a predefined sequence of air gap values is applied to capture the full spectral range.The hyperspectral data cube is thus formed in the time-sequential imaging principle.When using this technology on a moving platform, each band in the data cube exposed to a different air gap value has a slightly different position and orientation, which has to be taken into account in the post-processing phase (Honkavaara et al., 2013).
The FPI camera prototype 2012b was used to capture hyperspectral images in the range of 500-900 nm (Honkavaara et al., 2013).It is equipped with custom optics having a focal length of 10.9 mm and an f-number of 2.8.The camera has a CMOSIS CMV4000 RGB image sensor.The time difference between adjacent exposures is 0.075 s, and between the first and last exposures in a data cube with 24 bands is 1.8 s.The image size is 1024 x 648 pixels with a pixel size of 11 μm.The field of view (FOV) is ±18° in the flight direction, ±27° in the crossflight direction, and ±31° at the format corner.The entire camera system weighs less than 700 g.
The full system consisted of the following components.An onboard GPS (RasPiGNSS, NV08C-CSM) was used for collecting camera position trajectory for georeferencing purposes.Ocean optics irradiance spectrometer USB 2000+ (OOU2000) with cosine collector optics was used for the measurement of incident irradiance; the cosine collector was stabilized to prevent tilting of the sensor affecting the results of the irradiance measurement.Additionally Raspberry Pi2 on-board computer was used for collecting timing data for all devices, logging the GPS and for saving the OOU2000 spectra.Furthermore, the FPI camera was equipped with an irradiance sensor based on the Intersil ISL29004 photodetector.The sensor was not calibrated to measure in W/m 2 and only relative, broadband irradiance intensity values were obtained (Hakala et al., 2013).In UAV operation a RGB camera (Samsung NX 300) was used to support the geometric processing.
In the airborne operation, the UAV was a hexacopter with Tarot 960 foldable frame and it was equipped with a tilting gimbal, allowing capture of vertical and oblique images around the object of interest (Figure 1).Autopilot was Pixhawk equipped with Arducopter 3.15 firmware.Payload of the system is 3-4 kg and flight time 15-30 min (depending on payload, battery and conditions).In the terrestrial operation, the FPI camera and the irradiance sensor were mounted on a pole with adjustable height and tilting, allowing images to be captured from multiple views and altitudes of tree canopies and the forest floor (Figure 2).Using the developed system, hyperspectral point clouds can thus be captured from vertical, horizontal and any other oblique perspective between these.The ground station (Figure 3) was composed of reflectance reference panels and ASD Field Spec Pro spectrometer to record irradiance at the clearing next to the measurement location.Ground control points (GCPs) were installed in the area to support the geometric processing.

Processing of airborne data
The data processing is described briefly in the following.More details of the approach are given by Hakala et al., (2013) and Honkavaara et al., (2013Honkavaara et al., ( , 2014Honkavaara et al., ( , 2016)).

Georeferencing data.
The study area included six GCPs, which locations were measured using Leica System 500 receiver with Leica AT502 antenna.The measurements were approximately 45 min static measurements with 1 Hz data acquisition rate.The measurement data was post-processed using RTKPOST2.4.2-software (http://www.rtklib.com/).The solutions were calculated using Precise Point Positioning method ("PPP Static").The precise orbit and clock data were downloaded from International GNSS Service (https://igscb.jpl.nasa.gov/components/prods_cb.html).
The navigation file for the day was received from NERC British Isles continuous GNSS Facility (BIGF, http://www.bigf.ac.uk/).
The flight trajectory of the UAV was measured using single frequency GNSS receiver mounted on the UAV.The data acquisition rate was 2 Hz.The post-processing of the flight trajectory was calculated using RTKLIB Kinematic method using 1 Hz base station observation data from Ordnance Survey Oxford reference station (approximately 6 km from the study site).The flight trajectories were further interpolated to exact image acquisition time stamps.

Photogrammetric processing. Agisoft Photoscan
Professional commercial software (AgiSoft LLC, St. Petersburg, Russia) were used for the image orientations and DSMs.An integrated geometric processing was carried out for the RGB images and one band of the FPI images for the morning flights (f1, f2).Geometric processing for afternoon flights (f5, f6) was performed for three bands of FPI images simultaneously and then the results of orientations were merged and optimized with RGB images orientations.In the integrated orientation processing, there were a total of 881 images for the morning block (f1, f2) and 2078 images for the afternoon block.In order to transform the image orientations to the WGS84 coordinate system, the 6 GCPs and GPS-trajectory of the flight were used.The outputs of the process were the image exterior orientations and the camera calibrations in the object coordinate system.Dense point clouds were generated using RGB images to acquire as high as possible point density.Final point clouds were produced with 10 cm point interval.Orientations of the remaining FPI bands were optimized to the reference bands.

Radiometric processing.
Empirical line method based on reflectance panels and direct method based on at-sensor radiance and irradiance at object could be used to transform image DNs to reflectance.The two methods can also be integrated; panels determine the absolute level and the irradiance observations support monitoring the changes in illumination.(Hakala et al., 2013;Honkavaara et al., 2014).
Six reflectance reference targets were installed in a small opening in the test area (Figure 3).Panels were of size of 1 m x 1 m and had reflectance of 0.53 @ 540nm (white), 0.10 @ 540nm (grey) and 0.03 @ 540nm (black).A set of three panels, one of each target panel type, were positioned in a sun illumination and another set in the shadow of the canopy.The reflectance factors of the radiometric targets were measured in field the ASD with 18 degree optics, by referencing to the Spectralon reference panel.Finally, the spectra were sampled for the spectral bands of the images.
The ASD Field Spec Pro spectrometer (Analytical Devices) was used to capture irradiance on the ground level next to the radiometric targets during the flights.The irradiance was measured using cosine collector optics and each spectrum had a GPS timestamp for time synchronization with other measurements.Spectra were collected with 5 s time interval during the flights.The OOU2000 spectrometer was onboard the UAV with cosine collector optics to collect the irradiance above canopy.The optics were mounted to a stabilizing gimbal that was supposed to keep the cosine collector pointing directly towards zenith, however the performance of the gimbal was sufficient.This caused the irradiance data to have intensity offsets relative to the actual angle between the cosine collector and solar direction.This was corrected by calculating a level shifting correction factor using the ground irradiance data.
In this investigation, the reflectance calibration of the reference image in the first flight f1 was calculated using three panels by the empirical line method.Other images and flights were registered to the flight f1 using the irradiance measurements captured during the flight; for flights f1, f2 and f5 the ASD irradiance was used; the ASD was in shadow during the flight f6 thus the wideband Intersil irradiance was used.
Finally, reflectance signatures, point clouds and mosaics, and hemispherical directional reflectance factors (HDRF) were calculated using the parameters produced in previous phases.

Terrestrial data
Principles in processing of the terrestrial data were similar to the UAV data processing described in Section 2.4.
For the georeferencing, six metal tags with four-square pattern (Figure 6a) on tripods were placed around the tree; they were used for georeferencing and as tie points to register individual pole data.The locations of three reference tags were measured using static 45 min GPS measurement sessions.The measurement and post-processing procedures were the same as with the UAV GCPs.The image orientations and point clouds were calculated using the PhotoScan software.
Four-color Spectralon reference panel was imaged in the beginning of every imaging set (at each pole location) (Figure 6b); this was used to calculate the transformation from image DNs to reflectance with the empirical line method.OOU2000 spectrometer with cosine collector optics was attached to the FPI camera to collect the at sensor irradiance and the ASD was measuring irradiance at the clearing.Outputs of the processing were reflectance point clouds.a) Figure 6.a) Geometric reference tags present in the measurements.b) A color-infrared (NIR, R, G) band composite of single FPI image frame with a 4-color Spectralon panel.

Geometric processing
Photogrammetric processing of UAV data sets by the PhotoScan was automatic excluding the interactive GCP measurement and quality control.The RMS reprojection error was less than 0.55 pixels (Table 4).In the terrestrial data sets, the images collected in each pole position could be processed automatically by the PhotoScan, but manual support was necessary to integrate datasets captured in different pole positions.Coded targets were measured manually as tie points and used as error free matches in the alignment process.Due to different viewing angles and too low overlaps in some areas, three pole locations (1, 3 and 4) out of five could be combined to produce single combined point cloud.The average reprojection error was 0.676 pixels.Figure 9 shows examples of irradiance spectra on ground measured by the ASD in a small forest opening and in the UAV by the OOU2000.Before calibration there was difference in the irradiance level (Figure 9a) which could be eliminated by a calibration correction (Figure 9b).The impact of surrounding vegetation is visible the spectrums (Figure 9a, b): the UAV irradiance is similar to ASD irradiance on ground level.When the UAV rises above canopy, the irradiance decreases in the NIR range; this is expected result and caused by the light scattering from the surrounding vegetation.
The empirical line method provides simple approach for reflectance calibration in close range applications.However, in forested environment finding suitable position for reflectance panels can be difficult.Physically based approach utilizing irradiance measurements offers more efficient approach.However, the accurate calibration of components of the system becomes crucial; poor calibration of sensors or instability will cause significant bias in measured reflectance values.In the terrestrial measurement the illumination setup is extremely complicated.We will continue the research to develop efficient methods for reflectance characterization in complex environments.

Radiometric outputs
Previous results show the challenges in radiometric data capture.The spectral data presented here is still preliminary.
Examples of preliminary reflectance spectra of individual trees show different levels of spectra for different tree species (Figure 10).Hyperspectral mosaics of the flights f1 and f6 are presented in Figure 11.They are of good quality because of the uniformity of the imaging conditions.In Figure 12 examples of UAV and terrestrial spectral point clouds is shown.
Figure 13 presents preliminary HDRFs sampled over the entire test area for the flight f1 for band at 794 nm using a spatial resolution of 5 m x 5 m (see approach in Honkavaara et al., 2014).The maximum view zenith angles were 65º.The preliminary HDRFs show increase in reflectance in the backward scattering direction, and the decrease in the forward scattering direction; this is expected behavior for forests.The HDRFs show great variability because of the natural variations in the scene; the fact that tree species was not taken into account increases the variability.In Figure 14 the HDRFs of two individual trees sampled using a spatial resolution of 1 m x 1 m show differences in HDRFs of different species.In the further work it is necessary to improve the processing.For example, tree species specific HDRFs can be sampled to increase the number of samples in each class.Other improvements include better reflectance calibration, outlier detection and uncertainty estimation.

CONCLUSION
We presented preliminary results of reflectance and 3D characterization of a Wytham Woods, Oxford, UK, forest area using hyperspectral multiview/stereoscopic imagery captured using novel hyperspectral frame camera system.The system was operated from UAV platform and terrestrially.The new technology enables spectral reflectivity characterization of complex canopies.In the future we will further improve the reflectance characterization and study the uncertainty aspects.Furthermore, we will study comprehensive analysis of forest characteristics using the 3D hyperspectral data.

Figure 1 .
Figure 1.The instrumentation used in the UAV measurements

Figure 3 .
Figure 3. Terrestrial reference data during the UAV flights.

Figure 4 .
Figure 4.The block structure.Vertical image block composes of 10 flight lines in North-South direction.The oblique block consists of 4 bi-directional flight lines.

Figure 5 .
Figure 5. Approximate positions of the terrestrial stereoscopic imaging by FPI camera.

3. 2 Figure 7 .
Figure 7. Empirical line parameters Refl=a*DN+b a) for flight f1 and b) for terrestrial pole position t3.Figures 8 and 9 illustrate the characteristics of irradiance measurement in forested scenes.Irradiance measurements during the UAV image capture by the ASD on ground (at 794 nm) and by the ISL wide band sensor are shown in Figure 8.The ASD measurements indicated stable irradiance during the flights f1 and f2, during the last half of the flight f5 and f6 the irradiance spectrometer was in partial or full shadow, which is seen as variable level of irradiance measurements.The wideband irradiance measured by the ISL sensor in UAV indicated that the illumination conditions were stable during all flights; the tilting of the sensor when the UAV was flying in different flight directions explains the changing level of irradiance measurements.

Figure 8 .
Figure 8. Irradiance measured for each image a) on ground level by ASD at 794 nm and b) in the UAV by the Intersil wideband irradiance sensor.a) b)

Figure 13 .Figure 14 .
Figure 13.Preliminary HDRFs from the morning data set: the raw observations (left) and the fitted model (right).

Table 1
. Paramters of image blocks.Dist: Nominal camera to object distance.V: vertical, O: oblique; T: terrestrial.

Table 2 .
Solar angles, imaging time and exposure information.