AUTOMATIC EXTERIOR ORIENTATION PROCEDURE FOR LOW-COST UAV PHOTOGRAMMETRY USING VIDEO IMAGE TRACKING TECHNIQUE AND GPS INFORMATION

In this paper, we propose the automatic exterior or ientation procedure for the Low-cost Unmanned Aeria l Vehicle (UAV) photogrammetry. The Low-cost UAV has become useful too for low altitude photogrammetry from the rapid ly increasing of automatic control technique in past several years. However, automatic exterior orientation is still im portant issue. The most important point of automatic exterior orientation i s the automatic detection of corresponding points i each still image. For this goal, the authors have developed the automatic correspond ing point detection technique using both video Imag e nd still images. The video image of this investigation is obtained with still images simultaneously. Therefore, the trackin g result of video image gives robust correspondence information between each stil l image. Moreover, the exterior orientation procedu r sing GPS information from UAV and minimum ground control points have bee n investigated. This paper is structured as follows . At the first we describe the over view of automatic exterior orientation pro cedure in this investigation. As the next, the deta ils of tracking of common feature points and robust bundle adjustment are described. Also, accuracy assessment of this automatic exterio r orientation procedure is described. Finally some conclusion is given. * Corresponding author.


INTRODUCTION
The aerial triangulation in photogrammetry using mass images taken by the Low-cost UAV has become possible easily by rapidly increasing of automatic flight control technique of Lowcost UAV in These past several years.The Low-cost UAV which integrated GPS and IMU usually has ability to perform autonomous flight and automatic navigation along planned waypoints.These waypoints are usually planned beforehand easily by dedicated software as considering the camera parameter, area of interest, overlap of image, altitude and so on.Moreover, still images at planned waypoints are acquired automatically by digital still camera mounted on UAV.Therefore, Low-cost UAV system is widely used as the useful platform in the low altitude application field of photogrammetry such as agriculture, archaeology, traffic monitoring, and disaster area surveying (Remondino, 2011).However, in order to perform the fully automatic aerial triangulation using mass still image in low altitude application field, some problems have to be resolved.The automatic detection of corresponding point from common features visible in each still image is the most important problem.In the case of low altitude application using UAV, this problem becomes complex issue because the geometry of images is similarly to close-range photogrammetry.In the case of standard altitude application of automatic aerial triangulation, direct geo-referencing approach using GPS and IMU on UAV is often considered.But, direct geo-referencing approach using GPS and IMU on Low-cost UAV is difficult in the case of low altitude application because the GPS observation is more affected by environment than standard case of aerial triangulation.Therefore, the automatic detection of corresponding point between each still image has to be performed by image based approach.The automatic detection of corresponding points using coded target is one of the popular approaches.However, the photogrammetry using Low-cost UAV is often used under the difficult situation for direct ground surveying such as disaster area or cultural heritage.Consequently, only minimum targets can locate in area of interest, but the correspondence of all images is not provided by these targets.Another important problem is the robust exterior orientation using automatic detected corresponding points, GPS information at waypoints from UAV, minimum ground control points (GCP).Because the corresponding points from common feature are often including error and also the positioning information from Low-cost UAV often includes outlier, robust exterior orientation has to be investigated.On the other hand, the authors have been concentrating to investigate the Structure from Motion (SfM) technique using both video image and GPS in late years (Anai, 2010).In these investigations, we have proposed the robust feature point tracking method based on the "Orientation Code" (OC) image processing (Ullah 2001).Also, bundle adjustment method for video image that uses both SfM technique and GPS data with considering error in GPS observation.If both video image and still images are obtained from UAV, the tracking information of video image gives the robust corresponding information between each still image.From these circumstances mentioned above, this paper proposes about automatic corresponding point detection method using both video images and high resolution still images.Also, the automatic exterior orientation procedure using corresponding points, GPS and minimum GCP is also described.

AUTOMATIC EXTERIOR ORIENTATION USING VIDEO IMAGE AND STILL IMAGES
This section describe about the automatic exterior orientation process in this investigation.
In order to perform automatic detection of corresponding points from discontinuous high resolution still images without using coded target or manual operation, the keypoint detector and descriptor such as SIFT (D. G. Lowe. 2004.) or SURF (H.Bay. 2006.) are widely used now.By the way, if assume that the video image between discontinuous still images are obtained, video image sequences gives information about the motion of corresponding points.Therefore, the authors developed the procedure of automatic detection of corresponding points using video image sequences and high resolution still image.Moreover, robust exterior orientation that using corresponding points, GPS on UAV and minimum ground control point was also investigated.

Integrated Devices on UAV
The authors have developed the UAV system which has ability of acquiring video image and still image simultaneously.Figure1 shows the UAV system of this investigation.This system is based on "AscTec Falcon8" which developed by "Ascending Technologies GmbH".Table1 shows the details of integrated devices of this UAV system.The finder image of digital still camera RICHO GX200 is connected to video recorder by NTSC video line.Therefore, high resolution still images (12 mega-pixels) at each waypoint and the video image (VGA size) during a flight are obtained.The synchronization between video image and still images is performed by detection of shuttered frame from video image sequences.Also, the positioning results at each waypoint are obtained by KML format file as the function of "AscTec Falcon 8".Table1.Details of Integrated Devices

Flow of Automatic Exterior Orientation procedure
Figure 2 shows the flow of automatic exterior orientation procedure in this investigation.The waypoint planning and flight control are performed by "AscTec autopilot control software" provided by Ascending Technologies GmbH.High resolution still images at waypoints, video image during flight, and also positioning information of each waypoint are obtained as the result of a flight.At the first of this processing, the detection of common feature points from high resolution still image and tracking of detected feature points on video image sequences are performed using robust feature point tracking method based on the OC image processing.The tracking result of feature points are obtained in the image coordinate on high resolution still image.As the next step, sub-pixel matching of each feature point is performed.The good results of this subpixel matching are used as candidate of correct pass points.In order to perform tie point matching, the exterior orientation using these candidate pass points is performed.The result of exterior orientation in this step gives the geometry of candidate pass points and still images in arbitrarily position and scale.Finally, registration in global coordinate using the positioning information of waypoints and minimum ground control points is performed by using bundle adjustment with robust regression.These photogrammetric procedures are developed based on "TOPCON Image Master".Therefore, the 3D modelling from still image also can be performed easily. (1) In this equation, ∆I x and ∆I y show horizontal and vertical gradient of pixel (x, y).N is the quantization level of direction.
N is set to 16 typically (Figure 3).The "γ" is the threshold value for the suppressing of small gradient pixel.( )  The matching of common feature points between each video frame is performed by OCM in this procedure.Therefore, almost points are matched correctly, but generally include some error as outlier in main transformation between each video frames.Therefore, detection of outlier in common feature points is necessary to perform.In this procedure, authors assume that the transformation between each video frame is approximated with affine transformation as following equation, ( Where a 1 ~a6 :affine parameter, (u, v):image coordinate at frame i.
The affine parameters a 1 ~a6 are calculated using LMedS method and the outlier of matching is eliminated automatically.Finally, in order to perform the sub-pixel matching, Least Square Matching (LSM) of each common feature points on each still image is performed.In this LSM processing, forward and backward matching is performed for the checking of LSM result.This means that the result of forward (e.g. from left image to right image) matching and backward (e.g. from right image to left image) matching has large difference, the matching results of this point is removed as error correspondence.The good results of this sub-pixel matching procedure are used as the candidate of correct pass points in following procedures.

Tie Point Matching
In order to perform adjusting between each flight line, tie point matching using candidate pass points is performed.Figure 8 shows the flow of tie point matching procedure.At the first, exterior orientation only using candidate pass points is performed.

Robust Bundle Adjustment in Global Coordinate
As the final step of exterior orientation, we perform bundle adjustment in global coordinate system which using the corresponding points of still image, the GPS observation of UAV at waypoint and the minimum GCP.The candidate pass points and tie points detected automatically as mentioned before generally include mismatching.Also, the GPS observation often includes outlier.Therefore, the bundle adjustment of this investigation is performed using robust regression method based on M-estimator.The M-estimator is defined as the least-square method that the each weighting coefficient is decided to suppress the outlier.Weighting coefficient of M-estimator w eff is obtained from following equation.
[ ] where w: weighting coefficient of least-square method v: residual error of each observation equation σ: RMS of each observation equation.z: normalized residual error ψ: influence function The influence function ψ of this investigation is Tukey's Biweight that defined as follows. ( (7) The constant C in equation ( 7) is selected as 5~9.
Therefore, the robust bundle adjustment using M-estimator is performed by minimizing the following error function.

∑ ∑ ∑∑
where x fp : image coordinate of point p on the f-th image x' fp : re-projected image coordinate of point p on the f-th image G f' : 3-D coordinates of GPS at the f'-th waypoint G' f' : Computed 3-D coordinates of GPS at the f'-th waypoint P g : 3-Dcoordintaes at the g-th GCP.P' g : Computed 3-Dcoordintaes at the g-th GCP.w fp eff : Weighting coefficient of M-estimator for re-projection error w f' eff : Weighting coefficient of M-estimator for GPS w g' eff : Weighting coefficient of M-estimator for GCP From this procedure, exterior orientation parameter of each still image and 3D coordinate of each common feature point are obtained in global coordinate system, and also the influence of error observation such as mismatching points or outlier positioning of GPS at waypoint is suppressed by this procedure.

EXPERIMENT
In order to evaluate the effectiveness of proposed automatic corresponding point detection method and robust exterior orientation method, some experiments were performed.These experiments performed at the embankment of Arakawa river side located near the TOKYO.The area of interest was about 80m × 40m.In this area, 11 check points for accuracy evaluation were constructed before by VRS-GPS surveying.
In this experiment, two kinds of following flights were planned.The first flight type was planed as photography from constant altitude like conventional aerial photogrammetry.The second flight was photography from constant height from the ground surface.

Constant Altitude Flight
Figure 9 shows the result of the first flight experiment.In this case, 33 still images were acquired by UAV.The height from UAV to ground was changing from about 10m to 20m due to the altitude of this flight was constant.
The positioning data of each still image were obtained by DGPS (SBAS) on UAV.From still images, 1029 corresponding points were obtained automatically with our proposed method using video image and still image.The exterior orientation of this experiment was performed by proposed robust bundle adjustment using still image position and only 3 ground control points.The outlier in still image position from GPS observation and miss matched points were suppressed automatically with our proposed robust bundle adjustment.The RMSE of 3D measurement for 11 check points from aerial triangulation in this experiment was 16mm in XY axis and 22 mm in Z axis.
Figure 10 shows the 3D texture model which generated from automatic corresponding points.

Topography Trace Flight
Figure 11 shows the result of second flight.The height from UAV to ground surface was about 15m and UAV flied as trace the topography of embankment.In this case, 33 still images were obtained and 1489 common feature points were obtained automatically.The RMSE of 3D measurement for 11 check points was 20mm in XY axis and 24 mm in Z axis. Figure 12 shows the 3D texture model of second flight.From these experiments, it is confirmed that the proposed automatic corresponding point detection method and robust exterior orientation method have enough ability for automatic aerial triangulation using Low-cost UAV in low altitude photogrammetry.

CONCLUSION
In order to perform automatic aerial triangulation using Lowcost UAV in low altitude application field, we propose automatic corresponding point detection method and robust exterior orientation method in this investigation.
The automatic corresponding point detection method of this investigation utilize video image and still image.The video image can observe the movement of common feature points continuously by the robust common feature point tracking method based on OC image processing.Also, the LSM on still image gives fine accuracy image coordinate.Therefore, the automatic corresponding point detection method of this investigation gives many robust corresponding points from conventional features in each still image automatically.
On the other hand, the proposed exterior orientation procedure using robust bundle adjustment base on M-estimator have ability to suppress outlier of GPS observation or miss matched corresponding points.Therefore, the exterior orientation parameter and 3D coordinate from this method is obtained in global coordinate system by using not accurate GPS and minimum global control points.However, the estimation of the reliability of proposed method is still not enough.Also, the process speed of proposed method has to be more increased.Moreover, in order to perform more accurate aerial triangulation without using global control points, utilization of RTK-GPS system has to be considered.Therefore, our next motivation will be the development of low altitude automatic aerial triangulation system using UAV that have RTK-GPS.

Figure
Figure 1.UAV of This Investigation

Figure 2 .
Figure 2. Flow of Automatic Exterior Orientation2.3OC Image ProcessingIn this investigation, the detection of common feature point and tracking procedure are performed by "Orientation Code" (OC) image processing.The most unique point of OC image processing is that the coded images are used instead of the original gray-scale images.The OC means a quantization of the maximal intensity change direction around interest pixel.The OC of each pixel is defined as following equation:

Figure 3 .
Figure 3. Orientation Code (N = 16) In order to extract common feature points from OC image, we use Orientation Code Richness (OCR) that extracts the pixels which have high entropy of OC.The entropy of the local limited area of the pixel size M-by-M region at the interest pixel (x, y) is calculated as follows, ( ) Figure4shows the example of OC image.From this image, it is understood that the OC image is independent from the changing brightness and also OC image describe the important feature of image.Figure5shows the result of OCR.The bright pixel on OCR means the high entropy of OC.Figure6shows the example of OCM.From the robustness of OCM, almost points are not influenced by the changing of brightness or noise of image.

:
Figure 9. Result of First Flight