AN INTEGRATED RANSAC AND GRAPH BASED MISMATCH ELIMINATION APPROACH FOR WIDE-BASELINE IMAGE MATCHING

: In this paper we propose an integrated approach in order to increase the precision of feature point matching. Many di ﬀ erent algorithms have been developed as to optimizing the short-baseline image matching while because of illumination differences and viewpoints changes, wide-baseline image matching is so difficult to handle. Fortunately, the recent developments in the automatic extraction of local invariant features make wide-baseline image matching possible. The matching algorithms which are based on local feature similarity principle, using feature descriptor as to establish correspondence between feature point sets. To date, the most remarkable descriptor is the scale-invariant feature transform (SIFT) descriptor , which is invariant to image rotation and scale, and it remains robust across a substantial range of affine distortion, presence of noise, and changes in illumination. The epipolar constraint based on RANSAC (random sample consensus) method is a conventional model for mismatch elimination, particularly in computer vision. Because only the distance from the epipolar line is considered, there are a few false matches in the selected matching results based on epipolar geometry and RANSAC. Aguilariu et al. proposed Graph Transformation Matching (GTM) algorithm to remove outliers which has some difficulties when the mismatched points surrounded by the same local neighbor structure. In this study to overcome these limitations, which mentioned above, a new three step matching scheme is presented where the SIFT algorithm is used to obtain initial corresponding point sets. In the second step, in order to reduce the outliers, RANSAC algorithm is applied. Finally, to remove the remained mismatches, based on the adjacent K-NN graph, the GTM is implemented. Four different close range image datasets with changes in viewpoint are utilized to evaluate the performance of the proposed method and the experimental results indicate its robustness and capability


INTRODUCTION
Image matching, which is a process to detect corresponding points in the overlapping area of a stereo pair, has still remained as an important and challenging issue in digital photogrammetry, remote sensing and computer vision.Therefore it is not a surprising fact that many image matching algorithms have been developed during last years.These algorithms can be categorized into two area-based and featurebased methods.Area-based approaches use pixel intensities and matching is done by measuring the correlation between windows of predefined size.The main disadvantage of these methods is that they are sensitive to the intensity changes introduced by noise, different viewpoints or by different sensor types (Jogelkar and Gedam, 2012).Instead of pixel values, feature-based methods attempt to extract local shapes and structures, such as edges and corners, and match them using feature descriptors.The most well-known feature-based approach is the Scale Invariant Feature Transform (SIFT) method (Lowe, 1999 and2004), which it is invariant to image rotation and scale, and it remains robust across a substantial range of affine distortion, presence of noise and changes in illumination.
Even the best algorithms for image matching make some mistakes and output some mismatches (Adam et al., 2001).Due to this, different methods for outlier removal have been described recently (Fishier and Boles, 1981;Aguilar et al., 2009;Liu et al., 2012;Zhao et al., 2013;Wu et al., 2015).Among various mismatch elimination methods, the epipolar constraint based on RANSAC (RANdom SAmpling Consensus) is the most widely used robust estimator, particularly in computer vision.But the algorithm does not output good results when there are more than 50% outliers.Another simple and popular approach for outlier detection is the Graph Transformation Matching (GTM).The algorithm depends on finding a consensus nearest-neighbor graph from initial matches.In the other word, GTM uses local structures to find the inliers, so not enough global information is considered in this method.According to the methods mentioned above, generally, there are a few false matches in the selected matching results based on these approaches.Furthermore, image matching is much more difficult for widebase line images due to the large geometric transformation and illumination changes.However, because of constructing a strong geometry, wide-baseline image matching yields more accurate results than its short-baseline counterpart.Motivated by these facts, it seems that there is a need to search for finding solutions to achieve more reliable corresponding points in the wide-baseline image matching.In addition some applications, such as image registration, are very sensitive to outliers and sometimes existence of one mismatch in the corresponding set yields bad results.Therefore, to tackle the difficult task of widebaseline image matching and obtain an accurate pair of matched points, an improvement is proposed in this study which it incorporates previous approaches.In this paper, we use advantages of two well-known RANSAC and GTM approaches to eliminate outliers and obtain a high accuracy set of matched points.To do so, a three step matching scheme is presented.First, the SIFT algorithm is used to obtain initial corresponding point sets.In the second step, in order to reduce the outliers, RANSAC algorithm is applied.At the end, to remove the remained mismatches, based on local adjacent relation, the GTM algorithm is implemented.Indeed, both spatial relations and local structure are considered.

INTEGRATED APPROACH
In order to obtain reliable corresponding point sets, three steps are considered here.Each step would be mentioned in the following.

Scale Invariant Feature Transform
A scale invariant interest point detector and a descriptor based on the gradient distribution in the detected local region are incorporated in the SIFT algorithm.The interest points are detected based on local 3D extrema in the scale-space pyramid built with Difference-of-Gaussian (DoG) filters, which is invariant over a wide set of transformation, especially scale change (Lowe, 1999).In the SIFT descriptor, a vector is constructed for each keypoint by measuring the local image gradients in the detected local region and point matching is done by computing the similarity between each pair of vectors.To date, SIFT method is the most remarkable image matching algorithm which is used widely in different applications.However it outputs some mismatches, especially in case of wide-baseline image matching, as demonstrated in figure 1.
Figure 1.Result of SIFT algorithm in case of the wide-baseline images

RANSAC eliminator
As mentioned before, in this study the RANSAC method is used to reduce the outliers.In this step, an instance of samples from the SIFT matching result are randomly selected and based on them, a fundamental matrix is calculated.The, due to the fundamental matrix, epipolar lines are constructed and number of inliers are determined.Based on desired probability, the process is repeated and finally the fundamental matrix with the largest number of inlier is chosen.Although the epipolar line based on RANSAC is one of the most popular methods for outlier rejection, it has some limitations.For instance, as illustrated in figure 2, when two false matches be close enough to the respective epipolar line, they would be labelled as inliers in the RANSAC procedure.
Figure 2.An instance of remaining outliers by using RANSAC on the wide-baseline images In order to overcome this limitation, we use the GTM algorithm in the third step which would be mentioned in the following.

Graph Transformation Matching
In the GTM algorithm, based on neighbourhood relationships, a K-nearest-neighbor graph is constructed, respectively for each image.Then according to a graph-similarity measure one outlying correspondence is eliminated.After each iteration, due to changes in point sets, the graphs are updated and the process is repeated until the consensus graph is obtained.In most cases, the method has been shown to be able to deal with difficult cases such as similar patterns and non-rigid deformations of objects.However, it has some difficulties when the mismatched points surrounded by the same local neighbor structure (figure 3) or the real matched points have different neighbor structures (Liu et al., 2012).
Figure 3. Mismatches outputs from GTM due to similar local neighbor structure Our focus in this paper is on precession of final matched points.
Due to that, we integrated two mismatch eliminators, i.e., RANSAC and GTM methods.Results show that the combination of these two, can outputs more reliable matched points than using them alone.As described before, RANSAC cannot determine mismatches which are really close to their respected epipolar line, while they can be eliminated by using the GTM method.On the other hand, GTM has difficulties in outputting desirable results when the amount of outliers is large.So using RANSAC algorithm before GTM, reduces the outliers and makes the GTM more robust to remove the remained false matches.This three step matching strategy is illustrated is figure 4.
Figure 4. Steps of the presented matching strategy

THE DATA USED AND RESULTS
In order to evaluate the performance of the proposed method, we used four different sets of close range images with widely separated viewpoints.Three datasets, provided by Strecha et al. (Strecha et al., 2008), include perspective transformations of non-planar geometry, namely Herz-JesuP8, Castle-P19 and Fountain-P11.For the case of planar geometry we used the Oxford graffiti dataset from Mikolajczyk et al. (Mikolajczyk et al., 2005) as the fourth dataset.Results of the proposed matching strategy for the two datasets of Castle-P19 and graffiti are shown in figure 5 and figure 6.According to the figures, it is clear that the integrated approach can effectively remove the mismatches and output a reliable result.In the following the precision of the final matched points are compared against the RANSAC and GTM algorithms.In order to do so, the tolerance threshold for the epipolar constraint is empirically set to 0.001.The maximum number of iterations is chosen to be 1177, corresponding to a 99% probability of correct estimation with 50% of outliers (Aguilar et al., 2009).Also we used the implementation of RANSAC in MATLAB.In the case of GTM, a K-NN graph was created with an empirically chosen value of K=5. Figure 7 shows the ability of the integrated algorithm to achieve a reliable and accurate result.
Figure 7. Performance of integrated algorithm on the four datasets Figure 7 shows that the integrated method presents the highest precision among all considered methods.Although in case of Fountain-P11 dataset, where the precision values of the RANSAC and GTM are lower than 80%, the integrated method could not output a perfect result, it outperformed both RANSAC and GTM.

CONCLUSION
In this paper, an integrated outlier detection method is proposed to reduce ambiguity in matching feature points.The algorithm uses two well-known mismatch eliminator i.e.RANSAC and GTM methods.Although the two approaches are shown to be able to effectively detect the outliers, they have some limitations in dealing with wide-baseline images since the different viewpoints makes stereo image matching difficult.Results show that the integration of the mentioned methods can refine their performance.Successfully tested with four close range image pairs, the proposed method has shown its excellent ability to remove the outliers and output a high precision result.Despite of the high precision matched points that the proposed method produces, low number of corresponding points, due to using two mismatch eliminators, is the disadvantage of the approach.By this, the algorithm can be used in some applications which the precision of the final matched points is much more important than the amount of them.

FUTURE WORK
In the future work it would be a suggestion to use more datasets with different transformation including scale, rotation and illumination changes.It also worth considering the effect of changes in parameters of each step in order to increase the number of final matched points.

Figure 5 .
Figure 5. Outlier rejection for the second datasetAs can be seen in figure5(b), the RANSAC algorithm keeps several of the false matches due to their very short distance to the epipolar line, while these outliers are removed in the third step by using the GTM algorithm.Same scenario is shown is figure6for the case of planar geometry.