QUALIFICATION OF POINT CLOUDS MEASURED BY SFM SOFTWARE

This paper proposes a qualification method of a point cloud created by SfM (Structure-from-Motion) software. Recently, SfM software is popular for creating point clouds. Point clouds created by SfM Software seems to be correct, but in many cases, the result does not have correct scale, or does not have correct coordinates in reference coordinate system, and in these cases it is hard to evaluate the quality of the point clouds. To evaluate this correctness of the point clouds, we propose to use the difference between point clouds with different source of images. If the shape of the point clouds with different source of images is correct, two shapes of different source might be almost same. To compare the two or more shapes of point cloud, iterative-closest-point (ICP) is implemented. Transformation parameters (rotation and translation) are iteratively calculated so as to minimize sum of squares of distances. This paper describes the procedure of the evaluation and some test results.


INTRODUCTION
Recently, SfM (Structure-from-Motion) software is popular for 3D reconstruction and point cloud generation.SfM applications, such as Smart3DCaputure, PhotoScan, and Pix4D, are convenient for non-professional operator of photogrammetry, because these systems only require sequence of photos to generate point clouds with colour index which corresponds to the colour of original image pixel where the each point is projected.If the condition of capturing image is well-done, the result seems to be quite accurate.However, in many cases, the result is not constructed with correct scale or correct coordinates in reference coordinate system.
Basically, the quality of the point clouds created by dense image matching of SfM software should estimate by comparing true point cloud or more precise point cloud.In some cases, point cloud data measured by a laser scanner are adopted.But in many cases, laser scanner data do not have sufficient precision for, such as small objects.If the objects are on a cliff and hard to be accessed, measuring by TLS (Terrestrial Laser Scanner), might be impossible.
We focused on the evaluation of correctness of shape of the point cloud created by SfM.Even if it does not have correct scale, the shape of the object might be correct.To evaluate this correctness, we propose the difference of point clouds with different source of images.It the shapes of the point clouds with different source of images is correct, two shapes of different source might be almost same.
To compare the two or more shapes of point cloud, iterativeclosest-point (ICP) is implemented by Besl et al., 1992 andTakai et al., 2013. Transformation parameters (scale, rotation, andtranslation) are iteratively calculated so as to minimize sum of squares of distances.If the shape of the point cloud is correct, the distances of corresponding points between the two point clouds are expected to be small.Distance in ICP should be determined as the distance between a point of one point cloud and its nearest face of the other point cloud.The distances of the two point clouds reflect the error of the two point clouds' shapes.This method can be applied for point clouds without correct scale.This evaluation cannot be applied for some cases.One is the case that the object is isotropic like a sphere, or the case that the object is planar.This means that the method should check the anisotropy of the object by statistical analysis of distribution of normal vectors.
This ICP optimization and error estimation can also be used for extracting the part of deformation of the shape.

QUALIFIATION METHOD
The qualification method follows the procedure shown in Figure 1.

Capturing Two Groups of Images
Two groups of images are collected: Image set A and Image set B. Two groups of images should be captured under almost same condition, but it should not be same, because the errors in the point clouds created from these image groups should not have same tendency in systematic error caused by aerial triangulation and dense image matching.

Creating point clouds
Two point clouds of each image groups are generated by SfM software.The two of these point clouds should be almost the same coordinate system.This can be attained by creating GCPs in the point clouds of one group of images and execute bundle adjustment in the other group of images.To avoid the deformation caused by systematic error of bundle adjustment, the errors of GCP coordinates should be large in bundle adjustment.Point-to-plane and point-to-point distance minimization problem is solved using the method of Low (Low, 2004).Pointto-point distance (Dpt_pt) and point-to-plane distance (Dpt_pl) is defined in following equation: where is a transformation matrix in homogeneous coordinate system is a point in source point cloud, is the matching point in target point cloud, and is the normal vector of point calculated by PCA.T is described as following where are rotation angles about x, y, z axis ( ) and are translation.
CCICP minimizes sum of square difference of corresponding coordinates of points.Detail of CCICP algorithm is described in the paper of Takai et al (2013).

Quality Evaluation
CCICP outputs the mean value of square distances E. Root of E value includes systematic error of both point cloud.Therefore, root mean square error of each point cloud is smaller than root of E.

Test Object
The test object is a lava stone of Izu Oshima Island.The size of the lava stone is about 10 cm width and 5 cm height.Surface of the lava stone is partly rough with small holes, and partly smooth.The colour of the lava stone is almost black.Also, the stone object is set on a map-printed cloth (Figure 3).

Image capturing
The cloth and the stone had been set on a rotary chair (Figure 4) and images had been captured with SONY Cyber-shot DSC-WX200 (Figure 5).Two set of images, (image set A and image set B) had been captured.For each set of images, more than hundred of images had been captured all around the test object, with two different angles of depression (Table 1).

Generation of Point Clouds
Two set of images had been processed with SfM software, named Pix4Dmapper of Pix4D, for the generation of point cloud.
Image set A had been processed without GCP, therefore the image coordinates of point cloud of image set A is arbitrary and the unit of the coordinate system is approximately 1 mm (Figure 6).111 images had been processed and all of the images had been adopted for point cloud generation.The point cloud is dense around at stone and sparse at surrounding ( Figure 7.).In this paper, we refer to 1 unit length as 1mm.
Image set B had been processed with 125 images (Figure 8).Six GCPs measured in image set A (Figure 9) had been used in bundle adjustment procedure in Pix4D.To avoid the systematic deformation with bundle adjustment, the accuracy of GCPs had been set to 20 mm.(2) A GCP on the stone (3) A GCP on the cloth Figure 9. GCPs in point cloud Point clouds with reduced images had been generated with image set A and B respectively.We refer to point clouds generated from 1/1, 1/2, 1/4, 1/8 of image set A as A-1, A-2, A-3 and A-4 respectively, and similarly, B-1, B-2, B-3 and B-4 respectively.The number of points tends to decrease in both point cloud data set of both image set (Figure 10, Table 2).Table 3 shows the number of matching pairs for CCICP.Matching points were more than 90% of sample points of CCICP.It is considered that eliminated pairs in CCICP include points with big errors, pairs with different classification, or sparse point cloud, but their numbers were relatively small.Table 3 also shows that more than 90% of CCICP pairs were planar points.Figure 11 shows result of PCA classification of A-1.(1) Point cloud with real colour Figure 12 shows a sample profile of point clouds before and after CCICP registration, and Table 4 shows mean distances of point clouds before and after CCICP registration.In all cases, mean distances are about pixel size (0.03mm) or less.This shows that these setting of images and number of images do not affect precision of measurement so much, while it greatly affects number of measured points.This method requires redundant image capturing, but it is easy for small objects with large overlapping configuration.Numerical relationship between mean distance of CCICP and errors in point cloud generation has not been theoretically discussed in this paper.We are planning to clarify the relationship by analyzing the result of CCICP with simulated errors of point clouds.

Figure 1 .
Figure 1.Procedure of the Qualification 2.3 Point Cloud Registration Registration of two point clouds is executed with ICP algorithm.ICP iteratively try to minimize the distance of two point clouds.One point cloud, the source, is moved to fit the other point cloud, the target, by rigid transformation which is the combination of translation and rotation.We adopt CCICP (Classification and Combined ICP) algorithm.The CCICP algorithm minimizes point-to-plane, point-to-point distances, simultaneously, and also reject incorrect correspondences based on point classification by PCA (Principle Component Analysis) (Takai et al 2013).The points in the local point clouds are classified into linear points, planar points and scatter points depending on the results of the PCA which is shown Figure 2 (Demanke et al 2011).

Figure 3 .
Figure 3.The test object: lava stone of Izu Oshima Island of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-4/W5, 2015 Indoor-Outdoor Seamless Modelling, Mapping and Navigation, 21-22 May 2015, Tokyo, Japan

Figure 6 .
Figure 6.Point cloud A generated from image set A and estimated camera position and rotation

Figure 10 .
Figure 10.Point Clouds with reduced imagesTable 2. Data set name list of point clouds Point clouds with image set A Point clouds with image set B Data set name Number of image Data set name Number of images A-1 111 B-1 125 A-2 56 B-2 63 A-3 28 B-3 32 A-4 14 B-4 16 Figure 11.PCA classification of A-1

( 1 )
Figure13shows 3D error (distance of two point clouds) for registration between of A-1 and A-2.This shows that big errors (more than 0.09mm) cluster to some parts, while error distribution of other part have no eminent tendency.This means this qualification method can access local matching quality in a point cloud.

Table 1 .
Two set of images

Table 3 .
Numbers of matching pairs for CCICP

Table 4 .
Mean distance before/after CCICP registration