A DIGITAL-MAP AIDED TARGET LOCATION IN AN AERIAL IMAGE

Target location in an aerial image has become a critical issue with the wide application of Unmanned Aerial Vehicle (UAV) in the fields of target tracking and reconnaissance. This paper proposes a pipeline of target location based on the existing digital map and Digital Elevation Model (DEM). It utilizes the corresponding points between the aerial image and the digital map to build a mapping model, and calculate the position of target from this model. The pipeline can provide not only an approximate position obtained by the Direct Linear Transformation (DLT), but also an accurate value by implementing a few of iterations. The related error analysis is conducted to verify the positioning accuracy of this strategy. It can successfully avoid the dependence of the performance of GPS and POS, and ensure the high positioning precision simultaneously, which is attractive to low-cost surveying applications of small UAVs.


INTRODUCTION
In conventional methods of target location, the projection model are constructed from the sensors' measurements, including the position and attitude information provided by inertial Position and Orientation System (POS) and Global Positioning Satellites (GPS), in order to achieve a target location for the camera with a known orientation (Hongjian, 1998).The expensive high-stability platform and highprecision POS are required to indicate the exact orientation (Shaoli, 2005).However, UAV, as an economical and convenient means of aerial photography, is seldom equipped with these expensive sensors.It is necessary to find a target location strategy suitable for UAVs, in which the positioning precision does not rely on the performance of GPS and POS.As a mature surveying and mapping product, the digital map integrates the ground mapping, aerial mapping, and aerospace mapping data, which is helpful to aid the target location as a reference due to its rich and accurate data (DongGyu, 2002).This paper addresses a pipeline of target location based on the existing digital map and Digital Elevation Model (DEM).It utilizes the corresponding points between the aerial image and the digital map to build a mapping model, and calculate the position of target from this model.Since it makes full use of the high-quality reference, like digital map and DEM, its positioning accuracy is not affected by the performance of GPS and POS.It theoretically keeps the positioning deviation level nearly approximate to the quantization error of digital map or DEM, as long as exact corresponding points.Therefore, the correspondences are assumed to be correct in this paper, which can be given manually or automatically.According to the deviation of unknown model parameters, the mapping model can be respectively represented as the following three forms: 1) A full-parameter model.It consists of 11 perspective parameters and 2 radial distortion parameters, providing at least 7 corresponding points.It is suitable for the complex terrain very well.2) A simplified model under some constraints.The constraint is derived from prior knowledge which is known beforehand, like focal length, aspect ratio, even an interior orientation, etc.Thus, a full-parameter model is simplified by the given constraints to a concise model.3) A homography model mapping from a plane to another plane.It is considerably simple, fitting the flat terrain or the undulating slower terrain relative to the FOV of camera.4 non-collinear corresponding points are adequate to solve the model, not referring to any DEM data simultaneously.For the above models, the pipeline can yield not only an approximate position obtained by the Direct Linear Transformation (DLT), but also an accurate value by implementing a few of iterations.For the case requiring DEM data, the position of target could be confirmed by intersecting the ray determined by the principal point and the target image point with the terrain surface.Therefore, before running this location pipeline, the terrain surface fitting should be performed by interpolating the existing DEM data.Besides, this paper gives the error analysis for each mapping model, reflecting the impact of various error sources on the model precision and positioning accuracy.The error sources take into account the quantization errors of digital map and DEM.This analysis provides a valuable reference to select the proper-resolution map and DEM for the assigned location accuracy.Meanwhile, some experiments have been made to testify the positioning accuracy of each mapping model, under different levels of error.It is proved that it is feasible to carry out an accurate target location for aerial images, which are captured by UAV without the expensive stabilized platform and high-precision POS.

MAPPING MODELS
The mapping models reflect the transformation from the unknown parameters to the measurements.The measurements include the image coordinates and their corresponding object coordinates.Besides different forms of projection parameters, the unknown parameters also involve the true object coordinates, in light of the both measurement errors of image points and corresponding object points.And the number of unknown parameters varies with the different mapping models.

 
, , is the coordinates of object point in the geographical coordinate system, which is assigned as a reference system.

 
is the coordinates of image point in the image coordinate system.X  and x  are respectively their homogeneous coordinates.It is available to get the transformation between the geographical coordinate system and the map coordinate system, and all the ground control points on the map could easily transformed into the reference system to calculate mapping models.

Full-parameter Model
In this model, the mapping relationship covers the whole imaging process, particularly the image motion caused by radial distortion.Hence, the distortion corrected image coordinates c x  represent the imaging point of an object point through the pinhole projection.For simplicity, the distortion correction process can be written as: where 1 k and 2 k are the unknown parameters in terms of radial distortion.Then, the projection matrix between the object point and the corrected image point is written in homogeneous coordinates as: where i p is the i-th row of projection matrix 3 4  P .It may be expressed in terms of the vector cross product as Then for each correspondence j j  x X , the following two equations are derived, where p is a 12-vector made up of the entries of the matrix 3 4  P .Thus, we may obtain a 2 12 n  matrix A by stacking up the two equations for each correspondence (Richard, 2003).
The unit singular vector corresponding to the smallest singular value of A is the solution of the projection parameters p , as the result of direct linear transformation (DLT).This linear estimate could be a starting point minimizing the reprojection error.The reprojection error is to minimize the squared Mahalanobis distance: , , ,  , , , , , This cost function is minimized over all 3 14 n  unknown parameters (forming a vector u ) for n correspondences, and at least 7 correspondences (forming a vector b ) are required to establish these parameters. is the covariance matrix of measurements.The nonlinear function is usually computed by iterative techniques.Since it is usually vulnerable and sensitive to the starting point, optimization may converge to a local minimum instead of diverging from the true solution.The solution of DLT is a reasonable initial estimation and the Levenberg-Marquardt (LM) iteration method could be used to provide fast convergence and regularization in the case of this over-parameterized problem (William, 1988).It is underlined to normalize the measurements before applying the DLT algorithm.Normalization can effectively reduce the condition number of the set of DLT equations, improving the converging property (Richard, 1997a).Let the normalization matrix for image points be a similar transformation T and normalization matrix for corresponding object points be another similar transformation U , the relationship between the projection matrixes before and after the normalization could be given by: -1 norm where norm P is the projection matrix of normalized data.

Simplified Model under Some Constraints
1. Without the radial distortion If the corrected parameters related to radial distortion are measured in advance, the corrected image points c x may be estimated and the remaining 12 projection parameters can be accurately computed according to Eq.(3).

With the given intrinsic matrix
If the intrinsic matrix K is known through camera calibration, the projection matrix can be written in terms of matrix multiplication as where f is the focal length,  is the skew,   , x y c c is the principal point.R is a 3 3  rotation matrix representing the orientation of the camera coordinate frame, and t represents the coordinates of the camera center in the world coordinate frame.R and t are called the exterior orientation.The rotation matrix can be expressed as the Rodrigues formula: 8)   where v is a unit 3-vector in the direction of the axis and  is the angle of rotation.Hence, the number of unknown projection parameters is 7.Under this constraint, at least 4 correspondences are required to iteratively solve the nonlinear problem by LM method.The starting point of 7 parameters can be solved according to the calculated projection matrix of DLT.

Homography Model
Homography transformation reflects the mapping relationship between two 2-dimension planes, only relying on 4 correspondences.Therefore, the 3-dimension geographical coordinate system is simplified to a concise 2-dimension planar coordinate system composed of the longitude and latitude.
Considering the limited covering area of one frame, the angular information of longitude and latitude may be approximated by the linear interpolation.The  x  is a homogeneous vector consisting in the interpolated longitude and latitude.The similar equations are given by: where i h is the i-th row of the homography matrix 3 3  H .It can be solved by the DLT algorithm and the normalization should be implemented in advance.Because the altitude is not used here, i.e., the DEM is not demanded, this model is suitable for the flat terrain or the undulating slower terrain relative to the FOV of camera.

Residual Error
The RMS (root-mean-squared) residual error is the average difference between the measurements and their estimations.It reflects how well the computed unknown parameters match the input data, and explains the accuracy of the estimation procedure to an extent.Suppose the measurements are subject to any Gaussian distribution, with covariance matrix  .The residual error is the expected Mahalanobis distance

Estimation Error
Since there is still a deviation between the noise-free data and noisy measurements, the model may not give an approximation to the true noise-free values, even when the residual error is zero.However, estimation error is the distance from the estimated value to the true result, which could evaluate the performance of an estimation algorithm.It usually decreases in inverse proportion to the number of measurements.It is only used for synthetic data, or at least data for which the true measurements are known.Under the assumption that the surface consisting of valid measurements is locally planar, the estimation error may be written as where u is the true value, and û is the estimated value.
If the estimation correctly converges to a global true result, both sides of this equation are basically equal.Otherwise, the right-hand-side of Eq. ( 10) is likely to be much larger than the left-hand-side.

Covariance Matrix
There are many factors affecting the uncertainty of the estimated parameters, including the number of correspondences, the accuracy of measurements, as well as the configuration of correspondences.Especially, a degenerate configuration may lead to a wrong estimation (Newsam,1996).The uncertainty of the computed transformation is conveniently captured from the covariance matrix of the transformation.1. Full-parameter model The parameter vector 1 2 ( , , , ) k k u p X has 3 14 n  entries , where 14 parameters describe the transformation matrix, distortion corrected parameters, and 3n parameters represent estimates of n object points.The Jacobian matrix splits up into two parts as   |  J D B where D and B are the derivatives with respect to the camera parameters and the object points respectively.The covariance matrix is written as where u J is the Jacobian matrix evaluated at a parameter vector u .
(2 4 )( ) where, ( , ) T is the element of ( , ) p is expressed as Eq. ( 16), and cj   x X is written as Eq. ( 17).Then,  u can be calculated by Eq. ( 11). 2. Simplified model under some constraints Without the radial distortion For the parameter vector ( , ) u p X , the Jacobian matrix can be written as Then the covariance matrix  u can be gained by Eq. ( 11).With the given intrinsic matrix For the parameter vector ( , , , ) T T  u v t X , the Jacobian matrix can be written as ) The derivative ( , , ) t is a 12 7  matrix which could be computed from the projection matrix.Then the covariance matrix  u can be gained by Eq. ( 11) in the same way.

Homography model
If the image points are accurate, and the covariance matrix of map points is   x , then the covariance matrix of a parameter vector h constructed from homography transformation can be written as 1 ( )

Location Error
The location error could be evaluated by the covariance matrix in point transfer.For the projection model, the covariance of transferred point is expressed as 1 ( ) where    p J x p could be given by Eq. ( 16), and x X could be given by Eq. ( 14) or Eq. ( 17).In Eq. ( 22), it is mandatory to keep no cross-correlation between X and p , so X should be new points not used in the computation of the transformation p .For homography model, the covariance of transferred point is similarly expressed as where    h J x h could be gained by Eq. ( 21), and x x can be computed by Eq. ( 24)

EXPERIMENTS
In order to test our pipeline, we design a set of synthetic data, the true values of correspondences  x X completely satisfying the projection transformation in Eq. ( 2), where x are the coordinates of image points and X are the geographical coordinates of matching object points.These image points are evenly distributed in order to provide a good configuration, reducing the risk of no convergence.The focal length of a virtual camera is 20mm, the size of image is 512×512, and the pixel size is 6 m  × 6 m  .The intrinsic matrix K is written as 20 0 1.536 0 20 1.536 0 0 1 The coordinate reference is fixed on the camera, then the projective matrix is expressed as 20 0 1.536 0 0 20 1.536 0 0 0 1 0

P
We add some Gaussian noise on X respectively with 40dB signal noise ratio (SNR) and 20dB.So the covariance of X is computed by , where I is an identity matrix.
For the distortion of lens, 1 k is assigned to be -0.2535 and 2 k is 0.1187.The measurements of X is illustrated as Figure 1.If it is attempted to locate the corner of one image, whose accurate position is (-36.72,-36.72, 480) (meter).Then, the deduced positions from four models are listed in Table 1 reflects the absolute deviation from the true value.This contribution has been peer-reviewed.

CONCLUSIONS
This paper presents a pipeline of target location, in which the mapping model fits the transformation of correspondences, respectively from the aerial image and digital map.According to the diversity of unknown parameters, three forms of mapping models are discussed in detail, which could be suitable for different types of terrains.The associated error analysis is also elaborated, involving the location error of transferred target point.From the results of synthetic experiments, the positioning error is considerably small.Hence, this target-location strategy based on rich and high-precision data of digital maps is effective and economical for low-cost UAVs, even not equipped with high-performance platforms and POS.

Figure 1 .
Figure1.The measurements of noisy ground points X .If it is attempted to locate the corner of one image, whose accurate position is(-36.72,-36.72,480) (meter).Then, the deduced positions from four models are listed in Table1.( l X ,


are the square roots of three diagonals of  X , which are the metric of uncertainty

Figure 2 .
Figure 2. The appearance of location program.
. From Table1, the location precision of SNR 40dB is usually higher than that of SNR 20dB, which explains the impact of measurement errors on the location precision.The fourth model, (homography model) can give a similar result as that of the second model (simplified model without radial distortion).

Table 1 .
The deduced positions from four mapping models and their location errors International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-1/W2, 2013UAV-g2013, 4 -6 September 2013, Rostock, Germany