EVALUATION OF A METRIC CAMERA SYSTEM TAILORED FOR HIGH PRECISION UAV APPLICATIONS

In this paper we present the further evaluation of DLR’s modular airborne camera system MACS-Micro for small unmanned aerial vehicle (UAV). The main focus is on standardized calibration procedures and on photogrammetric workflows. The current prototype consists of an industrial grade frame imaging camera with 12 megapixel resolutions and a compact GNSS/IMU solution which are operated by an embedded computing unit (CPU). The camera was calibrated once pre-flight and several times post-flight over a period of 5 month using a three dimensional test field. The verification of the radiometric quality of the acquired images has been done under controlled static conditions and kinematic conditions testing different demosaicing methods. The validation of MACSMicro is done by comparing a traditional photogrammetric evaluation with the workflows of Agisoft Photoscan and Pix4D Mapper. The analyses are based on an aerial survey of an urban environment using precise ground control points and acquired GNSS observations. Aerial triangulations with different configuratrions of ground control points (GCP’s) had been calculated, comparing the results of using a camera self-calibration and introducing fixed interior orientation parameters for Agisoft and Pix4D. The results are promising concerning the metric characteristics of the used camera and achieved accuracies in this test case. Further aspects have to be evaluated by further expanded test scenarios.


INTRODUTCION
The increasing number of small UAV's for survey activities led to a demand for appropriate aerial camera systems.Using customized consumer cameras in combination with computer vision based software solutions have been repeatedly proven by several working groups (Nex et al., 2015) and show impressive results for mapping and object reconstruction purposes (Gerke and Przybilla, 2016;Przybilla et al., 2015;Eling et al., 2015;Cramer et al., 2013;Harwin and Lucieer, 2012).However, only a few aerial camera systems were presented for UAV's which enable a photogrammetric image processing in a traditional manner (Martin et al., 2014;Brauchle et al., 2014).This implies to determine explicit values for the interior (IO) and exterior orientation (EO) parameters like it is common for conventional aerial mapping cameras.Due to national law policies the maximum take-off weight (MTOW) of UAV's is often restricted which will lead to limitations as well for the payloads.Using carriers with an MTOW less 5kg is quite popular in Germany (German Federal Ministry of Transport and Digital Infrastructure, 2014) and other European countries which was the starting point of developing a metric and lightweight aerial camera for those carriers (Kraft et al., 2016).A first prototype had been developed based on the concept of DLR's modular airborne camera system (MACS) (Lehmann et al., 2011) with the intention to verify the system by a traditional photogrammetric evaluation.A special focus is as well on analyzing the long-term stability of the IO parameters.Therefore an industrial grade frame imaging camera was calibrated several times over a period of five month, using two different three dimensional test fields.
To verify the overall concept an aerial survey was done by using 39 ground control points, as well as precise GNSS observations.Based on this data set several aerial triangulations have been calculated.Those will be compared to the results of processing the data set with the structure-from-motion based systems Agisoft Photoscan and Pix4D Mapper by using the same preconditions for IO and EO parameters.

AERIAL CAMERA SYSTEM
The frame imaging camera consists of an industrial grade camera module with a 12 megapixel CMOS chip (ON Semiconductor KAC-12040C with Bayer pattern) and an industrial C-Mount lens (Schneider Kreuznach APO-Xenoplan 2.0/24), Fig. 1.The aperture is fixed to f4.0 and the focus is fixed to a focal range of 20-105m.All mechanical connections are glued to enable a stable interior orientation.
Figure 1: Industrial grade camera components For determining exterior orientation parameters a Dual-Frequency GNSS receiver (Novatel OEM615) is used in combination with an industrial grade IMU (Sensonor STIM 300).The solution is capable to post-process the flight trajectory and every image can be assigned with a precise GNSS time and position.Therefore the signal of the sensor readout time is registered to the GNSS time.Due to a continuous syncing of all time counters each image can be assigned with a precision less than 1µs.The image acquisition, time synchronization, trigger control and the co-registration of all data is done by a small embedded CPU (Intel Atom Dual Core, 4GB RAM, Linux OS).The camera is operated in global shutter mode by using a GigE-Vision Interface.The embedded CPU has the ability to acquire raw images at a frequency of 4 Hz and all data is recorded to a swappable Cfast 2.0 Card in raw format (up to 256GB).The Software is fully developed by DLR's institute of Optical Sensor Systems (OS) which is responsible for DLR's modular airborne camera system (MACS).The configuration of the camera system is done prior and post flight using a connected notebook or tablet.The current prototype shown in Figure 2 is optimized for aerial flights with unmanned multirotor aircrafts and could be considered as autonomous payload (external power supply presumed).The current weight is about 1620g (including camera, stabilized gimbal, embedded CPU, GNSS receiver and antenna, IMU and power management).Due to the use of a stabilized gimbal (modified HiSight SLR gimbal by MikroKopter) the dimensions are 165mm in width, 200mm in length and 265mm in height.Especially for the calculation of the EO parameters a variable lever arm between the IMU and GNSS antenna has to be considered because of occurring gimbal movements during flight.For this case the dual frequency GNSS antenna was directly mounted to the aircraft's frame with lowest distance to the IMU (Figure 8).The resulting positioning error cannot be measured directly but it can be quantified by analysing the motions of the aircraft under the assumption that the stabilized gimbal is compensating the motions during flight continuously.

GEOMETRIC CALIBRATION
The frame imaging camera was calibrated several times (Table 1) using two different three dimensional test fields located at Beuth University of Applied Sciences, Berlin (Figure 3) and Bochum University of Applied Sciences, Germany.On both sites several coded markers are attached on two right-angled facades (Figure 3) which serve as ground control points (GCP's).All markers have been measured with a tachymeter followed by a net adjustment resulting in a standard deviation of 1mm in each dimension.
The image acquisition was done at different locations, distances, line of sights and height levels in relation to the calibration target.At each location four images with four different rotations were captured (0°, 90°, 180° and 270° around the line of sight axis) considering a homogeneous distribution of markers for each image.The parameters of the IO and EO were calculated during a bundle block adjustment using Technet Pictran for the first two calibrations and Aicon 3D-Studio for all following calibrations.The point measurement was done automatically due to the use of coded markers.
The significantly estimated parameters are:  Due to the fact that the deviations are less than the pixel pitch of the sensor (4.7µm) the IO can be considered as constant in this case (Figure 4, 5 and 6).Thus the chosen industrial grade frame imaging camera shows features of a metric camera and a selfcalibration step during flight might not be necessary.This hypothesis will be proved in section 6.

AERIAL SURVEY
The maiden flight of the aerial camera system was performed on 30.10.15 at DLR campus in Berlin-Adlershof (Figure 7).A building complex and its surroundings were chosen as test site.
Thus 39 GCP's were marked within an area of 80x70m placing 19 GCP's on the ground and 20 GCP's on different height levels (7-14m).All GCP's were surveyed with precise geodetic devices.Due to the variable lever arm between IMU and GNSS antenna (section 2) a positioning error has to be considered.The resulting error cannot be measured directly but it can be quantified by analysing the motions of the aircraft under the assumption that the stabilized gimbal is compensating the motions during flight continuously.By analysing 870 measurements of the aircrafts flight recorder following values could have been calculated for this flight: Pitch 1.0° (mean) 2.4° (std.dev.) -7.7° / 8.0° (min./max.)Roll -1.3° (mean) 2.9° (std.dev.) -14.0° / 6.4° (min./max.) By computing the resulting GNSS antenna offset for each measurement a mean value of 15,1mm for XY (Max.57,0mm and Std.Dev. of 12,3mm) exists for the applied setup (Figure 8).This will decrease the accuracy of the calculated GNSS positions and has to be taking into account for the start conditions of the aerial triangulation.The refined GNSS positions were assigned to their corresponding images which serve as input for further data processing in section 6. 29.Jan.

GEOMETRIC RESOLUTION
The geometric resolution of an image acquisition system depends not only on the used hardware components but on the applied image pre-processing methods (e.g.demosaicing), too.Therefore the image resolution has been determined under controlled, static conditions on the one hand and under kinematic conditions during flight on the other hand.
The quantification of the image resolution has been done by use of the method described by (Becker et al., 2005 andHonkavaara et al., 2006).Therefore the modulation transfer function (MTF) and point spread function (PSF) have been calculated for images with the designated test pattern (Siemens star).The MTF is approximated by a Gaussian shape function. (1) There are two criterions for the resolving power of the camera.
The width of the PSF at half the height of the maximum is one criterion (full width half maximum FWHM).The second criterion is the spatial frequency where the MTF reaches a certain (minimal-) value (MTF10).(Honkavaara et al., 2006) proved that the frequency belonging to the point of the MTF where it's at 10% of the maximum contrast level is closest to the Rayleigh criterion.The reciprocal of this frequency is the approximation for the number of the smallest line per pixel.
In addition to this quantification five different demosaicing methods have been used to obtain RGB images.A (common) bilinear interpolation method, the adaptive homogeneitydirected demosaicing algorithm (AHD) described by (Hirakawa & Parks, 2005) where colour artefacts and aliasing effects are being considered in terms of interpolation directions, the highquality linear interpolation method (MHC) described by (Malvar, He, Cutler, 2004) where luminance changes (especially at colour edges) are being strongly considered and the method (VNG) described by (Chang et al., 1999) where based on a threshold variable numbers of gradients are selected and interpolated in the 5x5 neighbourhood.The fifth method (DCB) is an open source implementation by (Góźdź, J., 2009) which is not described in detail but yields promising results.The analysis is done for the different demosaicing methods applied during image pre-processing.Please notice for Figure 11, that the resolution of the blue channel of the bilinear demosaicing method under kinematic conditions could not be determined correctly and is not given here.
Comparing the results from both scenarios, a clear difference between the static and the kinematic environment is obvious, but there are also differences in the different demosaicing methods.In both cases the bilinear method delivers the worse results.The other four methods perform quite similar, whereas the DCB might be seen as the one performing best in static and kinematic scenario.In order to compare the resolution to the nominal GSD the MTF10 values per demosaicing method are averaged for all three channels.Under static conditions the average of VNG for example is 7.17mm.That results in a factor of 1.24 in relation to the nominal GSD, whereas the factor for DCB is 1.08 (average 6.23mm).Under kinematic conditions the resulting factor for VNG is 1.77 (average 18.23mm) and for DCB 1.61 (average 16.6mm).Obviously the nominal GSD could closely be resolved in static environment which is not the case for the airborne scenario.These differences probably are a lumped sum of turbulences and vibrations of the carrier and ground smear due to the continuous flight during image acquisition (flight speed of ~20km/h).These issues will be deepened in a further study (preferably with different cameras and different resolutions under laboratory conditions).

DATA SET PROCESSING
In a first step five aerial triangulations were calculated including the determined pre-flight interior orientation (by 22.10.15), the improved navigation data and a subset of 155 post-processed images by using a traditional photogrammetric software environment (Trimble Match-AT).The point correspondences were determined with Least-Square Matching algorithm (LSM) and all GCP's were measured manually followed by a refinement with LSM.
The final boresight alignment between IMU and camera was initially determined by the first adjustment (Configuration I in Table 2) resulting in -0.9200° for Omega, -0.0442° for Phi and +1.3950° for Kappa.The EO parameters have been corrected by those values to enable the boresight alignment as well for the structure from motion software solutions.
As shown in Table 2, different configurations will be considered and identical start conditions have been used for each aerial triangulation (Table 3).Especially the start conditions for the EO parameters were chosen by considering the expected error of the variable lever arm between IMU and GNSS antenna.In a next step a processing of the data set was done with Agisoft Photoscan and Pix4DMapper taking into account the same IO (Hastedt and Luhmann, 2015) and measured EO parameters (RTK positions).A processing was done as well by estimating the IO parameters within a self-calibration configuration (*).
The following tables will show the 3D RMSE results of the several configurations (Table 2).In addition the value is normalized to the calculated mean GSD of 17mm (section 5).

DISCUSSION
Following measurement accuracies has to be considered for the discussion: - The aerial triangulations with fixed IO within configuration I (Figure 12), II (Figure 13) and III (Figure 14) show similar results due to the cross flight pattern with high overlaps.For configuration I and II accuracies are reached in sub-centimeter level no matter which software solution is used.These accuracies are a result of the metric camera, the captured overlaps, used flight altitude and the introduced ground control points with tachymetric precision.In configuration III all software solutions achieve similar results.The standard deviations may differ slightly but they have to be declared as equal by considering the resulting measurement accuracies for the exterior orientation and bundle block adjustment.The check point RMSE values shows accuracies of traditional GNSS surveys with RTK precision without introducing any control point in the aerial triangulation.Focusing on configuration IV (Figure 15) und V (Figure 16) Match AT still deliver higher accuracies than Agisoft and Pix4D.This behaviour has to be investigated by further test flights/ datasets.The results for using a camera self-calibration cannot be clearly rated.We have proven the camera's metric but Agisoft shows a systematic offset for the principal point and is adjusting the focal length significantly for configuration IV and V while Pix4D shows minimal deviations for the principal point with a clear outlier in configuration V (Figure 16).Due to the fact that both solutions deliver less detailed statistics for the camera calibration a final conclusion cannot be given.

CONCLUSIONS
As shown in section 3 the use of industrial grade imaging components can lead to similar IO parameters thru several calibrations.The deviations of the IO parameters are less than the pixel pitch thus the chosen frame imaging camera shows features of a metric camera.Exterior orientation parameters can be obtained precisely for every image (section 2) and the aerial camera system can be carried by unmanned aircrafts less 5kg MTOW.The maiden aerial survey (section 4) could be evaluated by using a traditional photogrammetric workflow only (section 6) and shows reliable results.An evaluation was done by Trimble Match-AT as well as Agisoft Photoscan and Pix4D Mapper under consideration of using the same IO and EO parameters.The achieved results by different aerial triangulations show similar RMSE values at the check points for the cross flight configurations (I, II and III in section 6) and reach accuracies of traditional GNSS surveys with RTK precision.A self-calibration doesn't achieve higher accuracies in all configurations with cross-flight pattern and hence is not necessary while using a metric camera.
The results for configuration IV and V are promising using a traditional photogrammetric workflow (Trimble Match-AT) but have to be verified with further test flights.The computed results by Pix4D and Agisoft do not allow reliable conclusions due to missing statistical quality parameters about IO and EO residuals.
A final assessment of the introduced aerial camera system is not possible in this test case, but further test flights are planned for summer 2016 at the area of Zeche Zollern (Dortmund, Germany), which has been used already for an ISPRS Benchmark (Nex et al., 2015).Several aspects need a more detailed investigation by recording aerial images of larger areas with different overlaps, in different flight levels and by using different unmanned aircrafts (multirotor and fixed wing).By further use of a stabilized gimbal a solution has to be found to measure the motions of the GNSS antenna in relation to the IMU to minimize the resulting EO error.

Figure 2 :
Figure 2: CAD model of current aerial camera system without wiring and dual frequency GNSS antenna

Figure 4 :Figure 5 :Figure 6 :
Figure 4: Motion of focal length (period of 5 month) The standard deviation was calculated to 3mm in planimetry and 2mm in height.The aerial survey was done at 11:50 a.m. with following parameters in reference to the groundin an altitude over ground of 55m with a flight strip distance of 15m.The aerial camera was carried by a remotely piloted octocopter with a total takeoff weight of 4890g.Due to cloudy weather conditions an exposure time of 1.5ms was chosen to avoid ground smear and to prevent over-and under-saturations.A Siemens star and a 1951 USAF resolution test chart were placed on the ground to estimate the true resolution.The flight was done by following alternating flight lines automatically.

Figure 7 :
Figure 7: Post-processed trajectory of maiden flight (left) and GCP overview (right) with height above ground level scale The post-processing of the raw images was done under consideration of the parameters for dark signal non-uniformity (DSNU) and photo response non-uniformity (PRNU) in bilinear debayering mode.The recorded raw GNSS data were postprocessed with Novatel Inertial Explorer including the differential correction data from SAPOS (a satellite positioning service of German land survey).By using a forward and backward calculation a root mean square error (RMSE) of 9mm was achieved for the recorded GNSS positions during flight.

Figure 8 :
Figure 8: Current setup of the mounted aerial camera system

Figure 9 :
Figure 9: Resolution test charts in-flight (original resolution) The resolving power has been determined pre-flight (static conditions) and in-flight (kinematic conditions) with the same pattern.A Siemens star and an USAF resolution target have been used for the evaluation, whereas the estimated MTF10 values are derived from the Siemens star only.The UASF pattern only serves as sort of comparison.Figure 9 shows one image from the flight.The MTF50 circle is overlaid to the Siemens star.A mean resolution (MTF10 value) of 17.4 mm from Siemens star here would correspond to about 18 mm from the USAF target, depending on which person reads the later from the target.Thus, only the quantified image resolutions MTF10 from Siemens star analysis are shown in the Figure 10 and 11 for the static and kinematic environment.

Figure 13 :Figure 14 :Figure 15 :Figure 16 :
Figure 13: RMSE results at 20 check points (configuration II) II Configuration III Configuration VI Configuration V Pixel Ck xH yH

Table 3 :
Start conditions of aerial triangulations