EPIPOLAR RECTIFICATION FOR CARTOSAT-1 STEREO IMAGES USING SIFT AND RANSAC

Cartosat-1 provides stereo images of spatial resolution 2.5m with high fidelity of geometry. Stereo camera on the spacecraft has look angles of +26 degree and -5 degree respectively that yields effective along track stereo. Any DSM generation algorithm can use the stereo images for accurate 3D reconstruction and measurement of ground. Dense match points and pixel-wise matching are prerequisite in DSM generation to capture discontinuities and occlusions for accurate 3D modelling application. Epipolar image matching reduces the computational effort from two dimensional area searches to one dimensional. Thus, epipolar rectification is preferred as a pre-processing step for accurate DSM generation. In this paper we explore a method based on SIFT and RANSAC for epipolar rectification of cartosat-1 stereo images.


INTRODUCTION
There are several methodologies available on epipolar rectification using Rigorous Sensor Modelling (RSM), Rational Functional Model (RFM), etc.,.In this paper we exploit the fine characteristics of CARTOSAT-1 such as stable geometric fidelity and invariant perspective centre of linear CCD array across the entire image.Here we propose a method for constructing epipolar image pairs using SIFT (Scale Invariant Feature Transformation) based matching and filtering the wrong matches by RANSAC (RANdomSAmple Consensus) algorithm.SIFT match produces initial level correspondence between the stereo images to construct fundamental matrix.Epipolar line and the rotation matrices are derived from the fundamental matrix.Then the epipolar rectified images are generated by resampling stereo images with respect to the rotation matrices.RANSAC algorithm is used to weed out wrong match points resulted from the SIFT.The proposed method produces epipolar rectified images and achieves subpixel accuracy for better anaglyph viewing.This method was carried out on different image pairs of cartosat-1 and the results are presented.The paper is organized as follows first we present epipolar rectification followed by SIFT and then we outline RANSAC algorithm tuned for the present application.

EPIPOLAR RECTIFICATION
Epipolar geometry is the geometry of stereo pair of images.The general methods of solving epipolar rectification are by parallel projection and rigorous sensor model (RSM).The later one using RSM requires proper resampling technique to preserve consistent scan line difference between stereo pairs.Alternatively, we adopted affine transformation based solution to reduce scan line disparity between left and right images as all the epipolar lines are parallel [ref].Importantly, due to cartosat-1 platform stability, non-agile camera mount and consistent geometry of payload parameters, the affine based approach offers an alternate solution to the problem.Figure 1.epipolar geometry As a first step, we construct affine epipolar fundamental matrix using accurate match points obtained by SIFT and filtered by RANSAC.Fundamental matrix is the algebraic representation of epipolar geometry.An affine fundamental matrix differs from a regular fundamental matrix in the sense that its epipoles are at infinity.This makes the top left 2×2 area of the matrix zero.Only the remaining 5 elements contain information.Solving for those elements we force the constraint that: and where and are the corresponding (match) points on left and right image.
The epipoles of the left and right image are represented by the last row and column of the affine fundamental matrix ( .The parameters in the matrix are solved by finding the Eigen vector with the smallest singular value. Next, we use these results to solve for rotation matrices to flatten the epipolar lines, where In the next step, scale and offset for Y axis is computed by solving using least square method, where here the subscript j ( j = 0,1) denotes the element at position j after forming the product i.e., similarly, In the same way, all the transformations including skew are solved for X axis, Once again forming an equation of the form to solve for scale, skewness and off-set along x-direction.Finally affine matrix for left and right are constructed based on the scale, skew and offset values estimated from the above equations.Skewness applies to both left and right imagesequally to minimize distortion.Affine matrices for left and right image are given by, (1) where .

FEATURE DETECTION AND MATCHING
Efficient matching between images is obtained by first finding as a set of robust features which are invariant to scale, rotation and translation.SIFT (Scale Invariant Feature Transform) proposed by David Lowe (2004) belongs to the class of featurebased matching built on two main processing phaseskeypoint extraction and keypoint matching.
The main steps of SIFT are as follows (1) images and their sub-sampled images are convolved with Laplacian of Gaussian operator to generate images into corresponding scale-space.Extremum points are selected in scale-space as keypoint candidates, which are likely to generate the most stable image feature points.
(2) Keypoint candidates are located accurately and points are rejected that have low contrast or are poorly located along an edge.(3) Dominant orientations are computed and assigned to each keypoint.(4) The local image descriptors are constructed according to the gradient information in the area around the keypoints.
(5) The ratio of the distance of the closest neighbour and that of the second-closest neighbour serves as a criterion to determine whether two keypoints are matching.
SIFT approach transforms local image features relative to coordinate frames that are expected to be stable across multiple views of an object.To achieve the computational efficiency step (1) is approximated with a difference of Gaussian (DOG) eq.( 1) at different image pyramid level to detect the extreme values.
(2) Pixels with these extreme values are selected as keypoints, described by means of a feature descriptor defined by a 128 dimensional vector step (3).Then keypoint matching is to calculate the Euclidean distances from one keypoint descriptor on the left image to another keypoint descriptors on the right image.If the distance ratio (between the shortest Euclidean distance divided by the subsequent second shortest distance) is smaller than the threshold, then the keypoint is matched step (5).Thus, the one on the left image is matched to another one on the right image.Otherwise, the matching for this keypoint on the left image fails.The matching and searching operations will be done repeatedly until all keypoints are processed.The accuracy of generated match-points by SIFT are at sub-pixel level which also ensures the accuracy of epipolar rectification up to sub-pixel level.

AFFINE FITTING AND RANSAC
Matching by SIFT generates a large set of points which may contain wrong matches.To eliminate the wrongly matched points we use RANSAC.Let …, in the Left image and the corresponding matched points in Right image are …, .We fit an Affine transform (2) which takes care of rotation, scaling and shearing of the points.
(3) or in short e use RANSAC paradigm for estimation of affine transform parameters and .RANSAC is an iterative method to estimate parameters of a mathematical model from a set of observed data which contains outliers.It is a non-deterministic algorithm and can produce a reasonable result for parameters.The main steps of RANSAC algorithm are as follows Input: matched points and a known model(Affine Transform).Initialize :inlier_set = , percentage of inliers = 50% of data size, .(5) If the necessary number of iterations is enough, reestimate the model parameters using all the identified inliers in the current set and terminate.Else repeat step 1 through 5.

CURRENT PROCEDURE
The following figure [2] shows our workflow of the scheme adopted for epipolar rectification of Cartosat-1 images.Rotation matrices for left and right images are derived from fundamental matrix estimation.Then the matrices are corrected for scale, offset and skew/shear differences.

DATA SETS AND RESULTS
A description of the Cartosat-1 mission is given in Cartosat-1 Handbook (2006).We recall some of the satellite parameters that results quality stereo images and stable geometry.Cartosat-1 has a fore (F) and aft (A) panchromatic camera for along-track stereo, with a tilt in flight direction of and , respectively.Each sensor is comprised of 12,000 pixel CCDs with 7 microns pixel spacing.A roll steering of the satellite allows across-track pointing up to 23 degrees, increasing thus the revisit frequency.Dynamic changes, especially the continuous yaw steering and pitch bias (if applied during imaging) may affect both geometric stability and radiometric quality (image smearing).The stereo pairs with above geometry quality is well fitting into our methodology for generating high precision epipolar images.An image over Jammu and Kashmir contatining high terrain undulations was taken as to study the effect of this procedure.The result is show in Figure (3).The anaglyph is formed with this stereo image and a portion was shown in Figure (4).We noted that the procedure adopted produced best results and are visually verified by generating Anaglyph.

CONCLUSIONS
We have experimented a procedure based on SIFT and RANSAC for generating epipolar rectified images from Cartosat-1 stereo image pairs by fitting an affine transform.The results are producing the best epi-polar rectified images.
Randomly select a subset of data sufficient enough number of points to fit a model .[3 distinct points are sufficient to find 6 affine Transform coefficients] (2) Estimate the parameters of the model from the sample.[ ] (3) Calculate the number of inliers which support the hypothesis with a predefined tolerance [ count the number of points that satisfy ]. (4) If the number of inliers is greater than that of the inliers in the current set, keep this set of inliers as the new current set of inliers.[ if assign ].