ALIGNMENT OF POINT CLOUD DSMs FROM TLS AND UAV PLATFORMS

The co-registration of 3D point clouds has received considerable attention from various communities, particularly those in photogrammetry, computer graphics and computer vision. Although significant progress has been made, various challenges such as coarse alignment using multi-sensory data with different point densities and minimal overlap still exist. There is a need to address such data integration issues, particularly with the advent of new data collection platforms such as the unmanned aerial vehicles (UAVs). In this study, we propose an approach to align 3D point clouds derived photogrammetrically from UAV approximately vertical images with point clouds measured by terrestrial laser scanners (TLS). The method begins by automatically extracting 3D surface keypoints on both point cloud datasets. Afterwards, regions of interest around each keypoint are established to facilitate the establishment of scale-invariant descriptors for each of them. We use the popular SURF descriptor for matching the keypoints. In our experiments, we report the accuracies of the automatically derived transformation parameters in comparison to manually-derived reference parameter data. * Corresponding author


INTRODUCTION
Terrestrial laser scanners (TLS) point clouds capture prominent ground details such as building facades, whereas vertical UAV imagery enables us to generate models of roof structures and other missing details which the TLS are unable to collect.Combination of their datasets has potential use in applications such as automatic 3D building modelling, accident and crimescene reconstruction/analysis and open-pit mining.In this paper, we present an approach to align photogrammetricallyderived UAV 3D point clouds with TLS points.
The UAV and TLS data used in this work have different point densities.The 3D UAV point clouds were generated from downwards-looking camera using a Structure from Motion approach, thus the UAV 3D data are in a local 3D photographic coordinate system.We assume no prior information such as GPS or IMU data is available.The TLS 3D data are referenced in a rectangular Universal Transverse Mercator (UTM) / orthometric heights reference frame.Thus both data sets are in different coordinate systems.To map the UAV point clouds into UTM system, the 3D conformal transformation parameters (1 scale factor, 3 rotation angles and 3 translational components) must be estimated based on corresponding points / features between the two data sets.To determine these corresponding points our approach begins by detecting 3D keypoints on both the TLS and UAV point clouds.Our keypoint extraction method utilizes curvature extremas to identify significant point surface landmarks, for example, buildings corners and general areas of sharp changes in slope.To estimate the 3D transformation parameters, we require a minimum of 3 keypoint correspondences.To find the matching keypoints we use the SURF descriptors (Bay et al. 2008) characterizing the height surface around each keypoint.The descriptors are feature vectors which encode the keypoint's 'DNA', i.e., they uniquely characterize each individual keypoint.Since keypoints on two 3D surfaces vary due to conformal surface changes, the matching of descriptors permits the identification of common keypoints on both the source and target 3D point surfaces to ensure successful feature point matching.Figure 1 illustrates our methodological framework.

RELATED WORKS
There is some works which integrate both airborne and terrestrial laser scanning datasets (ALS and TLS).Cheng et al. (2013) presented a semi-automated building corner matching approach to co-register airborne and terrestrial laser scanning data.Their method uses an extension and intersection of building boundaries to form 3D structural corners on both the TLS and ALS.The corners are matched using a random sampling and verification strategy.An alternative TLS and ALS matching technique using linear features was proposed by von Hansen et al (2008).Firstly, lines are extracted from the raw point clouds.This is done using the edge points from point cloud planes are established using the eigenvector of the covariance matrix of each point cloud.Edge points with the similar eigenvectors are considered to be members of the same linear feature and are linked together to form the 3D line.Afterwards the rotation and translation between the TLS and ALS datasets are sequentially estimated.The rotation is computed by correlating the individual orientation histograms formed using ALS and TLS lines.The translation is then computed using a 'generate and test' approach, where the quality of all combinations of line matches are assessed using the closeness of matching ALS and TLS line midpoints.However, the method lacks the capability to cope with potential scale differences between datasets.
There is a dearth of relevant work in the alignment of photogrammetrically-derived UAV point clouds and LIDAR data.From the reviewed literature, Yang and Chen (2015) has presented a sensor-driven method to address the aforementioned registration problem.Imagery and 3D LIDAR points are collected simultaneously from a rotor-wing mini UAV with a Canon 5D Mark II camera and Riegl LMS-Q160 scanner respectively.They begin by extracting building outlines from LIDAR data and back-projecting these outlines onto the UAV imagery.The back-projection relies on prior position and orientation (POS) data acquired by an on-board Novatel Span technology.To improve this initial alignment, dense 3D point clouds are generated from the UAV images.Then, the refined transformation parameters are acquired by applying the Iterative Closest Point (ICP) algorithm on the 3D UAV points and the LIDAR points.In this work, our proposed alignment approach is entirely data-driven and is independent of any expensive POS sensor data.

METHOD
To automatically co-register the UAV and TLS surface models, we employ a point feature matching approach.Firstly, surface points are automatically detected using a 3D keypoint extractor.
Afterwards, scale and rotation invariant descriptors must be generated for all keypoints to handle these transformation differences between the point cloud DSMs.We explore and SURF (Speeded up robust features) descriptor (Bay et al. 2008) method for this application.

Extraction of scale-invariant DSM points
Curvature is a representation of terrain roughness.Regions of high curvedness will intuitively be more distinct than areas of lower curvature.A planar region is defined by surface points which share similar elevation.Rougher, more complex portions of the DEM on the other hand have points which share low height similarities.To mathematically indicate the degree of complexity of the local neighbourhood based on inter-height similarities, we analyse its corresponding local covariance matrix.Shape variation was extracted by applying Principal Component Analysis on the covariance measure of the point clouds around each point.The covariance was computed using the points neighbouring the keypoint candidate.
Eigen-analysis of the covariance gives 3 eigenvectors e u and its corresponding eigenvalues λ u , shown in Eq. 1.In a physical sense, the e u defines a local 3D orthogonal reference frame for the neighbourhood, whereas, λ u defines the size of each e u .The size of λ u represents the dispersion of the point's neighbours in each of the 3 orthogonal axes defined by the local reference frame.where; u=(1,2,3); u is in ascending order of λ magnitude.
The minimum λ 0 of λ u , is the surface normal of the keypoint candidate, i.e., the direction of minimal variation with respect to the local tangent plane on the 3D surface.The λ 1 and λ 2 are indicative of the deviation of points on the tangent plane.We use these eigenvalues to quantify an approximate surface curvature measure for the local DSM point cloud surface.Hence, we define surface curvature as the ratio of variance from the surface tangent plane, λ 0 , to the total variance,  

Keypoint scales estimation
The surface curvature of every point (all of which are considered to be keypoint candidates at this stage) are computed using their neighbouring points.However, the extents of the neighbouring points as determined by a radius are undetermined.This is so since the scale factor between the UAV and TLS DSMs is an unknown parameter (along with the rotation and translation parameters).It is also critical to determine the appropriate set of neighbouring points around a keypoint when generating its descriptor for the latter matching Apply 3D conformal transformation using corresponding keypoints phase.Thus, scale-invariant keypoints are very important.As illustrated in Figure 2, we address this problem by firstly estimating the curvature for all point clouds on a DSM.This is done at various radii values (i.e.various 'scales').Radii scales range from 0.1 to 0.5 with 0.05 increments on a normalized point cloud coordinate system between 0 to 1).The appropriate radius is the value at which the maximum curvature is attained.A query point is then deemed to be a keypoint if its curvature value is larger than those of all its neighbouring point clouds.
Figure 2 illustrates the keypoint detection framework.

Point description and matching
Following the keypoint detection phase, we utilize the SURF descriptor (Bay et al. 2008) at each keypoint to facilitate the matching of the point features.Therefore, for every keypoint, a SURF descriptor is computed on the DSM height map.Similarly, existing descriptor implementation like the SIFT algorithm (Lowe, 2004) could also be alternatively used in such a case.The SURF descriptor is based on Haar wavelet filter statistics in the x and y image directions and can be a 64 dimensional or 128 dimensional descriptor.
The match for each keypoint is established by utilizing the Euclidean distances for their 1st and 2nd nearest neighbours in the descriptor feature space.This is referred to as the nearest neighbour distance ratio (NNDR) (Szeliski, 2010).If the NNDR tends to a value of 1, then the match is not very reliable since the disparity between the first best and second best matches are not very distinct.A small NNDR value represents a possible matching candidate.If NNDR is less than a set threshold then a match is accepted (we use 0.3 in our experiments).To confirm the initial point correspondences, we proceed to apply the RANSAC method (Fischler and Bolles, 1981) to filter outlying matches.
Afterwards, a 3D conformal transformation, solving for the scale factor, rotational angles and translational shift between the UAV and TLS point cloud surfaces are computed.

Study area and datasets
We test the presented method on a study area situated in North York, Ontario, Canada.The test site contained a single building surrounded by parking lot, vegetation and bare land.Vertical photography of the test site was collected by a geo-X8000 UAV and the terrestrial laser scans via an Optech ILRIS long range scanner (Figure 3).To generate the dense point clouds from the UAV imagery, we used the Agisoft software (Agisoft, 2015).The method was prototyped in MATLAB programming language.To minimize the lengthy processing time, we downsampled the original point cloud density acquired from both the TLS and the UAV systems.

Results
Tables 1 and 2 show results achieved by the presented approach and those from of the reference parameters, respectively.The seven transformation parameters were estimated by manually selecting 4 point correspondences.The presented approach on the other hand has automatically determined 8 point correspondences (where the UAV DSM contained 13 keypoints and the TLS DSM had 17 keypoints).The derived parameters based on these 8 points closely resemble the reference parameters.Figure 4c shows the co-registration results of the UAV points (Figure 4a) with the TLS points (Figure 4b).The RMSE of the residuals from the automated method is 0.20 m, whilst that of the reference parameters is 0.27 m.The results of the automatically derived parameters can also be further improved by applying the Iterative Closest Point algorithm (Besl and McKay, 1992).

CONCLUSIONS AND OUTLOOK
We presented an automated co-registration approach to integrate UAV and TLS DSMs point cloud.We begin by firstly extracting scale-invariant keypoints.Then apply the SURF descriptor to match the detected salient points.Our results are comparable with the reference parameters.Future work will look at developing novel descriptors for matching point features with the intention to improve the overall accuracies in the estimated rigid body transformation parameters.

Figure 1 .
Figure 1.Proposed approach for UAV & TLS point cloud DSM alignment e.g.SIFT, SURF) for the point cloud UAV & TLS DSMs Match similar keypoint descriptors

Table 1 .
Transformation parameters -Automated selection of corresponding points

Table 2 .
Transformation parameters -Manual selection of corresponding points Parameters