THE RESEARCH OF LINE MATCHING ALGORITHM UNDER THE IMPROVED HOMOGRAPH MATRIX CONSTRAINT CONDITION

Focusing on the mismatching problems in line matching, this paper integrates the radiation information and the geometry information of the imagery as the multi-constraint conditions, and presents an improved line matching algorithm based on the improved homograph matrix constraint condition. This algorithm firstly obtains the homologous points by feature matching, and for each line to be matched, it calculates the homograph matrix with the homologous points in the neighbourhood of this line. And then it projects the line to be matched line in target image to the search image by the homograph matrix, and determines the candidate lines according to the distance between the central points of lines and the distance between two lines; In these candidate lines, this algorithm further determines the possible homologous lines according to the similarity constraints of the line angles, the distance from the origin of image to the lines, and the overlap of lines; Finally, the epipolar constraint is adopted to find out the overlap segments between homologous lines,and the real homologous line will be determined by the gray similarity constraint. This paper adopts the unmanned aerial vehicle images and UCX digital aerial images to carry on the experiments, and verifies the validity of the algorithm in this paper. * Corresponding author: Weixi Wang, Ph.D, E-mail: measurer@163.com.


INTRODUCTION
Linear features are the important features of imagery, and also the important outlines of objects for their 3D reconstruction.Different with the point features, the linear features have richer image information, and are less affected by noise.Line Matching is simply finding the corresponding images of the same 3D line across two or multiple images of a scene.It is often the first step in the reconstruction of scenes such as an urban scene (Mosaddegh, 2008).Comparing with the point matching, the line matching has the following main advantages: ⑴ more geometry constraints of linear features are used, and the matching results will be more reliable and in high accuracy; ⑵linear features are not sensitive to noise in the extraction, and less affected by the geometric distortion and gray deformation of image.Linear features will effectively improve the accuracy of matching; ⑶the number of line feature is far lower than the number of feature points, and the stereo matching based on linear features will greatly improve the efficiency of matching; ⑷the linear features are easier for extraction and description.However, line matching is much more difficult to obtain a reliable matching result by single constraint.Mainly due to the following reasons: linear features commonly found in the edges of objects.In different view points, the linear features may appear occlusion, fracture, etc., and cause the image textures on both sides of lines would be different; for the same ground object, the direction of its lines projected into different images will also be different, so the candidate lines to be matched may appear the results of "one-to-null", "one-to-one","one-tomultiple" and even "multiple-to-multiple"; simultaneously, because of the incompleteness of linear feature extraction and the inconsistency of homologous line endpoints, the direction, length and texture features of lines cannot directly be used as the primitives in line matching.For now, the existing line matching algorithms can be divided into two categories: one is based on the structure information of linear features: mainly considering the geometry attributes of the line (length, degree of overlap, gradient, direction, location) in the matching process; The other is based on the dominant points of line: according to the dominant points, a line is divided into a number of discrete points, and the matching of feature line is achieved by matching these dominant points in it.Each algorithm has its own advantages and disadvantages at the same time.Due to the influence of various factors in the imaging process, as well as the complexity of line matching, it deserves to research a matching algorithm having high accuracy, excellent applicability, and good robustness.Fu Dan (2008) proposes a linear matching method based on the polar constraint and the RANSAC algorithm, and effectively solves the matching problem of partly occluded lines in the image . Li Tao (2008) proposes a robust and fast line matching algorithm based on the supporting region of line, which enhances the ability of this algorithm to adapt to the noise.However, these algorithms lack effective geometric constraints, and bear not only the complexity of their own, but also the low success rate of line matching.In this case, this paper integrates the radiation information and the geometry information of the imagery as the multi-constraint conditions, and presents an improved line matching algorithm based on the improved homograph matrix constraint condition.This algorithm firstly obtains the homologous points by feature matching, and for each line to be matched, it calculates the homograph matrix with the homologous points in the neighbourhood of this line.And then it projects the line to be matched line in target image to the search image by the homograph matrix, and determines the candidate lines according to the distance between the central points of lines and the distance between two lines; In these candidate lines, this algorithm further determines the possible homologous lines according to the similarity constraints of the line angles, the distance from the origin of image to the lines, and the overlap of lines; Finally, the epipolar constraint is adopted to find out the overlap segments between homologous lines，and the real homologous line will be determined by the gray similarity constraint.

The Principle of Homograph Matrix
Homograph matrix is a mathematical concept, it defines the relationship between two images that any point in one image can be find the corresponding point in another image, and the corresponding point is unique, and vice versa (Wu Fuchao,2002).The homograph matrix can determine the correspondence relationship between images, and transfer the features from one image to the other.Through the location constraint of two line segments sets, the homograph matrix can realize the collection of matching lines.Let Where T i h (i = 1, 2, 3) is the vector i1 i2 i3 (h , h , h ) .By the matrix H, the corresponding point of point a in the right image can be expressed as: In fact, the point a is corresponding with the point b in the right image, and ( , ,1) T b x y ′ ′ = . Then the following equation can be drawn: All of the homologous points in the stereopair will obtain the above equations, and merges all the equations into a matrix expressing: Where: n is the group number of corresponding points.To ensure that the equations have a solution, there must be at least 5 groups corresponding points, and then using the least square algorithm to calculate the image transformation matrix having minimum error, i.e., the homograph matrix H .

Constraint of Candidate Lines Based on the Modified Homograph Matrix
In the field of computer vision, the homograph matrix only be applied to transfer the features between two images.This paper introduces the principle of homograph matrix in the line matching algorithm, and realizes the effective location constraint for the line segments sets of the left and right images.
For aerial images, according to the complex surface relief especially in the urban areas, if only adopt one homograph matrix in the matching process, the offset of homologous lines will be very large after the projection of homograph matrix.In order to avoid this situation, this paper modifies the homograph matrix algorithm.For each line to be matched, it utilizes the homologous points in the neighbourhood of line to calculate the homograph matrix.Firstly, it determines the existing homologous points in the neighbourhood of line to be matched, and calculates the homograph matrix with them.Then, it projects the line to be matched to the right image based on the homograph matrix, and determines the possible candidate lines in the right image according to the distance between the central points of lines and the distance from central point to other lines.The principle is shown as Figure 2:

Epipolar Constraint
If a pair of matching lines satisfies all the above similarity constraints, then the epipolar constraint will be used to find out the corresponding overlap segments between the two lines (Wu Bo, 2012).For example, for a pair of matching lines AC and BD in Fig. 3(a) and (b), the epipolar lines of the end points of AC and BD can be derived as illustrated using dashed lines in Fig. 3.By intersecting these epipolar lines with the lines AC and BD, the overlap segments between these two lines can be obtained, which is AD' and A'D.
Figure 3. Using epipolar constraint to find corresponding overlap segments for line matching

Brightness Contrast Constraint
After the epipolar constraint, it obtains the overlap segments of homologous lines.The brightness contrast in a local buffering region along both sides of the matching lines can be used to further disambiguate the line matching.Then limits a rectangular area with the central axis is L and the width is r 2 , and this rectangular area will be defined as the linear feature supporting region of line L .As shown in Figure 4 (b), the supporting region can be decomposed into 1 2 + r parallel line segments with equal length.L and the left r line segments are defines as the left linear feature supporting region, also L and the right r line segments are defines as the right linear feature supporting region.The gray value of the point j in the line i will be marked as j i g .Arranges the gray values of ( 1) r n + × points in the left supporting region can be arranged as a matrix form, and then the gray value matrix of the left linear feature supporting region can be obtained.Simultaneously, the gray value matrix of the right linear feature supporting region also can be obtained.On both sides of the tobe-matched lines, the Normalized Cross Correlation (NCC) values will be calculated separately between the image gray values within the supporting region.The larger one is taken as the final NCC value for this line.Then the correlation coefficients of the linear feature supporting regions can be calculated.

EXPERIMENTAL ANALYSIS
This paper adopts the unmanned aerial vehicle images and UCX digital aerial images to carry on the experiments of line matching.

Experiment 1
The experiment data are two images cut from the stereopair imaged by unmanned aerial vehicle, and the image sizes both are 512 ×512 pixels. of two line sets are shown as Fig. 5(f).This paper determines the candidate lines according to the distances between the lines to be matched, and fixes the homologous lines using other constraint conditions, then obtains the matching results are shown as Fig. 5(p) and Fig. 5(q).Through the visual interpretation, the " one-to-multiple " phenomenon can be found in the matching results, which is due to the broken lines in the extraction, and belongs to the correct matching results.From this experiment it can be found that the homograph matrix carries on the effective constraint to the line matching, reduces the complexity of matching algorithm, and improves the accuracy rate of matching.

Figure 1 .
Figure 1.The flowchart of line matching based on multiconstraint conditions coordinates of the point on the right image.Then the transform from point a to point b by homograph matrix H will be described as b Ha = , where H is a matrix of 3 3 × size, and defines the one by one relationship between the points of two image points.H is defined as following (

Figure 2 .⑵ρ
Figure 2. Constraint of candidate lines based on the modified homograph matrix

(
Figure 4. Linear feature supporting region and decomposition correlation coefficients of the left and right supporting regions for corresponding lines.

⑵
Fig. 5(a) is the target image, and Fig. 5(b) is the searching image.⑴ Computation of the homograph matrix.In this step, it firstly realizes the image matching based on feature points, and the succeed matched corners in the stereopair images are shown as Fig. 5(c).Then substitutes the matched points to the equation group 0 LH = , obtains the coefficient matrix L , and computes the matrix T L L .Finally it solves the homograph matrix H through the Singular Line extraction and matching This paper adopts the Canny edge detection operator to carry on the edge detection of image, and gets the binarization edge image.Then it extracts the lines from the binarization edge image using the improved Hough Transform.By setting the threshold, it avoid the over connection problem for long-distance points, and filters out some short straight lines.The line extraction results are shown as Fig. 5(d) and Fig. 5(e).Using the computed homograph matrix H , it projects the line set in Fig. 5(d) to the image coordinate system defined by the searching image, and the overlap results Figure 5.The original images and the results of post-processing