AUTOMATIC IMAGE REGISTRATION USING FREE AND OPEN SOURCE SOFTWARE

Image registration is the most critical operation in remote sensing applications to enable location based referencing and analysis of earth features. This is the first step for any process involving identification, time series analysis or change detection using a large set of imagery over a region. Most of the reliable procedures involve time consuming and laborious manual methods of finding the corresponding matching features of the input image with respect to reference. Also the process, as it involves human interaction, does not converge with multiple operations at different times. Automated procedures rely on accurately determining the matching locations or points from both the images under comparison and the procedures are robust and consistent over time. Different algorithms are available to achieve this, based on pattern recognition, feature based detection, similarity techniques etc. In the present study and implementation, Correlation based methods have been used with a improvement over newly developed technique of identifying and pruning the false points of match. Free and Open Source Software (FOSS) have been used to develop the methodology to reach a wider audience, without any dependency on COTS (Commercially off the shelf) software. Standard deviation from foci of the ellipse of correlated points, is a statistical means of ensuring the best match of the points of interest based on both intensity values and location correspondence. The methodology is developed and standardised by enhancements to meet the registration requirements of remote sensing imagery. Results have shown a performance improvement, nearly matching the visual techniques and have been implemented in remote sensing operational projects. The main advantage of the proposed methodology is its viability in production mode environment. This paper also shows that the visualization capabilities of MapWinGIS, GDAL’s image handling abilities and OSSIM’s correlation facility can be efficiently integrated to effectively use in remote sensing based production environment.


INTRODUCTION
Image registration is an important operation in remote sensing applications that basically relates a set of images which are in different coordinate systems.The process of registration spatially aligns the set of images by considering one of the images as a reference and transforms the remaining images through correspondence between them (Le Moigne et al., 2011).Time series analysis in remote sensing based applications requires two or more scenes of the same geographical region acquired at different times.To infer the information from the stack of time-series data, pixel by pixel comparison is to be made, thus all the images in the stack are to be registered to a map coordinate system and therefore have its pixels addressable in terms of map coordinates (eastings& northings or latitudes & longitudes) rather than pixel and line numbers (Richards &Jia, 2006).
In general, the registration process is usually carried out in four steps.The first step consists of selection of features on the images.Next, each feature in one image is compared with potential corresponding features in the other one.A pair of points with similar attributes is accepted as matched correspondence and the set of homologous points are termed as tie points or control points.These three steps can be achieved by interest point detection and correlation process when used in The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-8, 2014ISPRS Technical Commission VIII Symposium, 09 -12 December 2014, Hyderabad, India conjunction.Finally the parameters for the best transformation which models the deformation between both images are estimated using these tie points.Implementation of automatic image registration process in the chain of production mode not only reduces the human intervention and but also eradicates the errors caused due to manual selection of tie points between the images.
In this paper, we describe the implementation of automatic registration of remote sensing data using Free and Open Source Software (FOSS).The advantage of the proposed image registration method is its viability of working in production mode.Similar tasks were reported by Ferguson et al., (2006), Lou et al., (2006), Eugenio and Marchell (2009), the fundamental difference between their approaches with the approach described in this article is the difference in the implementation of the software.FOSSdevelopment is emerging as an alternative approach for developing large software systems.FOSS offers new types and new kinds of processes to research, understand, improve and practice (Scacchiet al.,2006).
FOSS serves the collective interest of the global community to share and develop interoperable information infrastructure.Thus, the ability to reuse, reanalyse and reapply will be possible.The current article highlights the usage of FOSS for accomplishing the task of automatic image registration of satellite data to achieve a mean radial error of less than a pixel.FOSS was used to handle Images and to perform the correlation process.The integration and handshaking mechanism between various modules of FOSS software with a DotNet application were also described.
In recent times, facilities and functionalities of FOSS have important advances that enhanced access to image processing and GIS analysis (Lein, 2012).Open Source Geospatial Foundation (OSGeo) is very active in supporting and developing the quality software tools for geospatial tasks (Hall and Leahy, 2008).The GDAL/OGR library and associated utility programs provide widely used basic functionality on the FOSS platform (Jolma et al, 2008).GDAL provide a generalized application programming interface (API) for raster data and OGR does the same for vector data.GDAL can be used to read, manipulate and write most of the commercial and scientific raster formats.Some of the available open source remote sensing software includes GRASS, QGIS, Open Source Software Image Map (OSSIM) and Orfeo Toolbox (OTB) (Camps-Valls et al.,2011).OSSIM in particular is a high performance engine for remote sensing, image processing, geographical information systems and photogrammetry (Kala and Silori, 2013).Christophe and Inglada (2009) mentioned that OSSIM software has the best established libraries to work with remote sensing data.The OSSIM image processing software provides tools for registering the satellite images (Bunting et al., 2013;OTB-CNES, 2013).The redistributable open source software MapWindow is a mapping and GIS modelling system which is gaining popularity in geospatial industry because of its simplicity to use (Steiniger and Hay, 2009;Fonseca et al.2014).MapWindow's core, MapWinGIS component was developed to meet the need for an alternative to proprietary GIS components.MapWinGIS is an ActiveX control that gives full ability to display, query, manipulate the spatial data and was built to add GIS functionality to the end user application.
This paper shows that the visualization capabilities of MapWinGIS, GDAL's image handling abilities and OSSIM's correlation facility can be efficiently integrated in remote sensing based production environment.Thus, the chain of operations can be performed in serial manner by reducing the manual intervention.

Datasets
The proposed software has been implanted for the Advanced Very High Resolution Radiometer (AVHRR) is a radiation-detection imager having five channel scanner, sensing in the visible, near-infrared and thermal infrared portions of the electromagnetic spectrum and Advanced Wide Field Sensor (AWiFS) operating in the four spectral bands and 56 m resolution.For performing correlation between two images, a reference data of precise geometric quality is prerequisite.Usually manual methods have to be implanted by using the process of rectification using survey grade ground control points (GCPs).

Software implementation in detail
The registration module is designed in such a way that first it checks for cloud content in the pre-processed level-1B image with the help of cloud flag image.If the cloud content is more than the specified limit (the default limit is set to 80% of the scene, but can be changed as per project decision) then the software will be driven into a semi-automatic mode where manual intervention is required to mark the tie points with respect to the master data.The software is equipped with point editing tools for marking new points, deleting the points and moving the points.Editing tool will come into rescue if the process of correlation yields no tie points.If the cloud content is within the specified limit (less than 80%) then the process of registration will be automatic.Figure 1  Area based methods match measurable image quantities like brightness, absolute gradient or phase using digital correlation techniques and feature based methods match features extracted from the images like corners or junctions (Fitch et al., 2005).Forstner (1982) explained that the correlation can be used for the accumulation of tie points, further these points will be used in the process of rectification.If rotation and scale issues are minimal then normalized cross-correlation is an effective method as a similarity measure, which is invariant to linear brightness and contrast variations between two images (Zhao et al., 2006).Harris corner detector can be applied to extract high curvature point followed by correlation technique to establish matching between the two images.
In the process of image registration the major share of process is identification of tie points between the images which is time-consuming and tedious, thus there is a need for automatic tie point generation process.The      MapWinGIS ActiveX control used in this application confirms that it has merits that can be used for visualization of remote sensing data.On the other hand GDAL is proven to be a boon for image processing applications because of its feasibility to perform a range of raster manipulation functionalities.The reliability of the proposed automatic registration process enabled to incorporate the software in the production mode to generate the time series data.

Fig. 1 .
Fig.1.Architecture andGraphic user interface of proposed automatic registration software shows the graphic user interface (GUI) of registration module.The interface was developed using Microsoft Visual Basic (in DotNet environment) that contains two viewers to display false colour composite (FCC) for both the input image and reference image.These viewers are MapWinGIS ActiveX controls.The MapWinGIS ActiveX control supports MouseMoveEvent and MouseDownEvent, which can be used for interactivity with raster/vector data that is been displayed.The DotNet framework supports ProcessClass which provides access to local and remote process and enables to start and stop local system process.The hand-shaking mechanisms between the GDAL's and OSSIM's readily available resource programs wereimplemented by the Process Classwiththe help of methods like Process.StartInfo and Process.Start.The pre-processed image and master image will undergo a correlation operation which finds tie points between them.Tie points which represents the image correspondence can be detected using either or both of the two methods called area-based and feature-based.
registration module presented in this paper is based on OSSIM's command line applications as core.Command line tools provide added functionality mainly because of the much larger set of possible actions and the opportunity to make jobs in batch modes.The OSSIM's command line tool namely correl.exetakes input parameters like correlation settings, reference image and input image.Correlation settings control the density of points that are to be generated.The search parameters include template radius and pixel absolute error.A moderate value was fixed for density of points and correlation coefficient was kept as 0.85.During the correlation operation corners are detected in the master image using Harris corner operator and these corner features are spread uniformly on the master image.An image chip is extracted around each corner in the master image by considering the user defined radius.This feature in the chip will be searched in the pre-processed image around a proximity area with larger radius.The normal completion of the normalized cross correlation between master image and input image yields a set of tie points which indicates a similarity measure between them (OSGEO, 2013).The correlation process ends by creating an Extensible Markup Language (XML) file namely tiepts.xmlwhich is composed of header and information about tie points in a structure manner.Each record of tie points contains ground coordinate (i.e., map coordinates) of the reference image and its corresponding location in the input image (in pixel coordinate system i.e., row and column).Figure 2 shows the screen shot of this XML file.The latitude and longitude of the tie points (considered that of the master image) are attributes of the element <ground>.The correspondence of this tie points with the input image is the attribute of element <image>.Both the <ground> and <image> are child elements of <SimpleTiePoint>.The .NET Framework (version 4.5), provides a set of asynchronous APIs for XmlReader and XmlWriter, which are based on the Visual Studio Asynchronous Programming model.These APIs provide the ability to process XML asynchronously.The namespace System.Xml.Linq contains the classes for LINQ (Language-Integrate Query) to XML.LINQ to XML is an in-memory XML programming interface that enables to process XML documents efficiently and easily.The XDocument class contains the information necessary for a valid XML document that includes an XML declaration, processing instructions, and comments.The possibility of parsing tiepoint.xmlfile enables fetching the correspondence between reference image and input image.The MapWinGIS ActiveX control not only has the ability to display raster and vector files but also graphics.Graphics can be of type circle, point, line, polygons.The colours and line width can be specified while creating the graphics.Tie points can be shown to the user by overlaying them as graphics in the reference image and input image with unique ids as labels.Displaying the tie points in the reference image and input image enables the user to judge the accuracy of image correspondence.Figure 3 depicts the screenshot of tie points that are generated between master image and input image.

Fig. 2 .
Fig. 2.Figure showing the XML file containing the tie points resulted correlation process

Fig. 6
Fig. 6 Screenshot depicting part of geographically viewers showing registered image (left) and master image (right) at a part of central India.
functionalities that are required to perform most of the remote sensing based operations.The correlation functionality of OSSIM software has the potential to detect the similarity measure between two images.

Table 1 :
RMSE of x and y along with extreme deviations at check points computed for test cases.

/Scene Description RMSE (pixels) for AWiFS Quadrant Data
This automated procedure was tested with three years of AVHRR and AWiFS data.It was realized that the registration accuracy achieved is within the pixel.Freely available software tools like GDAL, OSSIM and MapWinGIS are equipped with all the features and