CONTOURS BASED APPROACH FOR THERMAL IMAGE AND TERRESTRIAL POINT CLOUD REGISTRATION

Building energetic performances strongly depend on the the rmal insulation. However the performance of the insulation materials tends to decrease over time which necessitates the continuous mon itoring of the building in order to detect and repair the anom alous zones. In this paper, it is proposed to couple 2D infrared images rep resenting the surface temperature of the building with 3D po int clouds acquired with Terrestrial Laser Scanner (TLS) resulting in a semi-automatic approach allowing the texturation of TLS d ata with infrared image of buildings. A contour-based algorithm is proposed w hose main features are : 1) the extraction of high level primi tive s not required 2) the use of projective transform allows to handle perspective effects 3) a point matching refinement procedur e allows to cope with approximate control point selection. The procedure is applied to test modules aiming at investigating the thermal properties of material.


INTRODUCTION
In a context of energy resource scarcity and greenhouse gas reduction, the energetic building efficiency improvement is one of the major challenge of the next years.Most of the energy consumed by a building is devoted to heating and air conditioning.Thus, to reduce the energy consumption, there is a need to develop efficient wall insulation materials and building renovation techniques (Net, 2009).As the infrared camera yields images representing the surface temperature, inspection of building facade using infrared camera provides a very appealing technique for localization and time monitoring of thermal anomalous zones.Over the last decade, a number of approaches were proposed to register point clouds with photographic images, thus allowing the texturation of 3D building models (Stamos, 2002, Pu and Vosselman, 2009, Banno and Ikeuchi, 2010, Huhle et al., 2010).However, the methods devoted to texturation of point cloud with thermal infrared images remain limited.(Laguela et al., 2013) proposes a 2D-3D automatic registration method.It consists in registering 3D line edges extracted from the point cloud and edge lines extracted from the infrared image using a line detector algorithm (von Gioi et al., 2010).For the point cloud, 3D edge line detection is achieved by segmenting the point cloud into planar surfaces using a region growing algorithm and then determining their intersections.The lines from both data are registered using RANSAC (Random Sampling Consensus) algorithm (Fischler and Bolles, 1981), the procedure consists in finding the best subset of lines that maximize the number of co-linearity between lines from both data sets.The idea of using RANSAC for the registration is quite appealing but this method strongly depends on the estimated lines which can be very imprecise since building edges do not necessarily result in abrupt changes in the infrared images.Also, in the proposed method, only affine transformations are considered while the use of low focal distance lenses is likely to result in perspective effect for which a projective transformation is required.(Gonzalez-Aguilera et al., 2012) proposes to merge both sensors data using a 2D-2D registration method.After having corrected the geometric aberration induced by camera lenses, the point cloud is projected onto a plan parallel to the infrared image plan which results in the so-called range image.The registration is performed using an affine transformation and the control points from both images are selected using Harris operator (Harris and Stephens, 1988).To reduce the sensitivity of Harris operator to the point cloud density, the authors propose to perform an interpolation prior to the control point selection.Another 2D-2D registration approach is proposed by (Pelagotti et al., 2009); it consists in registering multispectral images (infrared, X-rays and ultraviolet fluorescence) of cultural heritage object and range image to build a textured model.First the range image is transformed to an intensity image, then each image is registered with the intensity image using the algorithm presented in (Maes et al., 1997) based on mutual information maximization.Another method proposed by (Lagela et al., 2011), consists in calibrating the camera, then both sensors data and intrinsic camera parameters are imported to RISCAN software witch compute the transformation parameters using manually selected control points.The main contribution of this paper is to present a contour based approach for 3D point cloud and infrared image registration which does not depend on the shape of the edge points and which takes into account the perspective effect by considering projective transformations.To account for the imprecise edge location and manual control point selection errors, an edge point matching refinement procedure is also proposed.The paper is organized as follows: sections 2 and 3 present respectively the edge extraction of the point cloud and the infrared image.The contour based registration approach including the point matching refinement algorithm is presented in section 4. Throughout the paper, the different steps of the method are illustrated on data corresponding to test module aiming at studying the thermal properties of insulation materials.The infrared images are provided by a FLIR B-620 camera and the point cloud by a TLS LEICA C10.The conclusion and future work are finally presented in section 5.The approach was evaluated on test module, conceived by CRITTBOIS and dedicated to the investigation of insulation and thermal properties of materials (figure 1).

TLS data segmentation
A building is mainly composed of planar surfaces (facade, doors, roofing) and the idea of the method is to register the infrared images to these planar surfaces which are extracted from the point cloud using a segmentation procedure.
The geometric accuracy of a 3D points depends on many factors such as surface characteristics (i.e.color, materials), sensor noise (Boulaassal et al., 2007).Thus, points belonging to a planar surface are not truly co-planar.As segmentation by region growing method is known to be sensitive to this geometrical error, we rather propose to use a Ransac based segmentation approach which exhibits very nice robustness properties with respect to this type of error (Vosselman et al., 2004).The resulting plane is a section of the point cloud and point belonging to another facade element or to the adjacent facade may be considered as belonging to the plan.We use the method proposed by (Awwad et al., 2010) which overcomes this problem by comparing the normal vector of the candidate model with the normal vector of the potential consistent point.The algorithm consists in the following three steps: 1. Neighboring point selection using FDN (Fixed Distance Neighbor) and computation of the normal vector of the point using least squares method.
2. Clustering of the point cloud using the orientation of normal vector.Two points belong to the same cluster if the angle between the normal vectors of the two points is less than a threshold determined by the user.
3. Segmentation of points cloud using a modified version of RANSAC.Firstly, a subset of 3 points are randomly selected from the cluster.Secondly, model parameters are computed to fit the sample.Finally, the model is ranked by the number of the consistent points.A point is said to be consistent if the orthogonal distance to the model is lower than the threshold δ th and angle between normal vector of the point and the normal vector of the proposed plan is lower than a threshold Rθ th .
The number of iterations is fixed by calculating the probability of finding a better model.Indeed, the probability of selecting a non-contaminated sample is given by: where NI is the proportion of inliers and N is the total number of points.Thus (1−Q) represents the probability of selecting a contaminated sample and the probability of selecting κ contaminated samples is given by: from which we get the number of iteration: In practice ǫ and NI are design parameters chosen by the user.
The algorithm is iteratively applied to extract all the planar surfaces of the point cloud.

Edge point extraction
To extract the edge points of each planar patch, we use a property of the Delaunay triangulation which states that the edge points correspond to the longest vertices of the Delaunay triangles (Boulaassal, 2010).To apply this approach, it is first necessary to project the points on the estimated plan.The procedure consists in the following steps: 1. Delaunay triangulation of projected points 2. Computation of the distance between the triangle vertices 3. Distances ordering 4. Selecting vertices of triangle segments for which distance is greater than λ th .The threshold λ th corresponds to the change of curvature in the ordered distance curve.
Figure 4 presents the 3D edge points extracted from the whole point cloud.This is achieved by successively extracting the edge points from the different 2D planar surface.

INFRARED IMAGE EDGE EXTRACTION
We propose a contour based registration approach; so it is necessary to extract the edges of the infrared image.The edge detection consists in identifying areas of a digital image that correspond to abrupt changes in light intensity.These changes in image properties generally reflect important events.In particular, for the infrared image, they reflect abrupt changes in temperature which are often present at the junction of facade elements (wall-roofing, window-wall, etc ...).A Canny filter (Canny, 1986) is used for the edge detection of the thermal image (figure 5).The results is presented in figure 6.

REGISTRATION
The registration consists in matching two or more images in order to compare or combine their respective information.It aims at finding a transformation T such that each control point of the reference image pi of coordinates (xi, yi) is transformed into the corresponding point of the target image p To handle the perspective effect, it is necessary to consider projective transformation.Considering the homogeneous coordinate system, the projective transformation can be written as : Noting a jk = h jk /h33, the coordinate x ′ i and y ′ i can be written as: The transformation is fully defined by the eight parameters a jk .So, at least four control points are needed to estimate the eight parameters since each control point yields two coordinates.It is worth mentioning that the accuracy of the registration transformation estimation is strongly related to the quality of the control points selection.Also, the registration of the two images has to face two other difficulties: the edge points of both data does not have necessarily the same cardinality and some of the edge points present in an image may be not present in the other and vice versa.
In order to reduce the impact of these problems on the registration quality, a method is proposed allowing to move the control points of the original reference image to get a maximum number of common points in both edge images.The advantage of using this type of criterion is twofold.Firstly, it is well adapted to the binary character of the edge images.Also, it is well suited to edge images not having the same cardinality.Indeed, while it can be difficult and time consuming to evaluate a distance between the edge images not having the same number of points, counting the number of common points yields no difficulty.The optimization algorithm is based on a greedy principle: it consists in successively modifying the position of each control points, each point being allowed to evolve in a neighborhood around the current position.The complete algorithm is given in table 4. The notations used are the following: R (0) is the set of control points of the initial reference image, C is the set of fixed control points in the target image, T (R, C) is the transformation corresponding to the control points C and R, N (R i } and finally J (R) is the criterion which corresponds to the maximum number of common points between the two images when the transformation T (R, C) is applied to the reference image.
The criterion considered for the registration is neither concave nor uni-modal.It is even possible to find examples where it does not admit a unique global maximum.The initial control points must be chosen manually and not too far from the optimum (provided it exists and is unique).If this is not the case, the method

CONCLUSIONS AND FUTURE WORK
In this paper, a registration algorithm allowing a TLS point cloud texturation with infrared image approach is presented.Fisrtly, the point cloud is segmented into planar surfaces using a modified The edge points are registered using a projective transform and a control point matching refinement procedure is proposed to cope with imprecise control point selection.The effectiveness of the proposed approach is demonstrated on real data corresponding to test modules.It is worth mentioning that the proposed approach can be straightforwardly applied to the texturation of TLS point cloud with photographic images.The case of color images can simply be handled by transforming it into a grey scaled image.However, we believe that jointly registering the three (R,G, B) images with the point cloud is likely to provide a much more effective approach.Also, it is possible to envisage a fully automatic control point selection approach for which a consensus based approach is expected to provide an efficient way to determine the maximum number of common edge points.

Figure 1 :
Figure 1: Test modules image 2 POINT CLOUD EDGE EXTRACTION

Figure ( 3
Figure (3) shows the segmentation process result of the method to the point cloud in the figure (2).

Figure 2 :
Figure 2: Point cloud of the material test module

Figure 5 :
Figure 5: Infrared image of the test module

Figure 8 :
Figure 8: Evolution of the cost function (number of common points)