Remote Sensing Image Fusion Based on Discrete Fractional Random Transform for Modified IHS

The traditional IHS fusion, as a spatial domain method, mostly achieves the good spatial quality but the spectral distortion. The discrete fractional random transform (DFRNT) fusion, as a transform domain method, can maintain the high spectral information. Thus, in this paper, we propose a combined DFRNT-IHS approach to obtain good spectral quality and high spatial quality. Moreover, instead of fusion in spatial domain or transform domain respectively as in the conventional way, the proposed DFRNT-IHS approach is for joint spatial and transform domain. Real multi-spectral and panchromatic images are used to evaluate the effectiveness of the proposed method. The fused images are found to preserve both the spectral information of the multi-spectral image and the high spatial resolution information of the panchromatic image more effectively than several existing image fusion techniques. Spectrum distribution of DFRNT is random and dispersive, which guarantees low spectral distortion and certain robustness. * Corresponding author: Qing Guo, qguo@ceode.ac.cn


INTRODUCTION
In a remote sensing image fusion process, a high spectral and low spatial resolution multi-spectral (MS) image with a high spatial resolution panchromatic (PAN) image to produce a high spectral and high spatial resolution image (Chavez 1986, Ranchin and Wald 1993, Zhou et al. 1998, Aiazzi et al. 2002, Otazu et al. 2005, Thomas et al. 2008).Therefore, the major objective of image fusion is to generate a high spatial resolution MS image.With the sensor technology development, although the numbers of available high spatial resolution images are increasing, image fusion is still an important and popular method to interpret the image data to obtain a more suitable image for a variety of applications, such as visual interpretation, digital classification and etc.
Various fusion algorithms have been proposed.They can be broadly classified into space domain method and transform domain method according to the fusion performed domain.The space domain methods usually generate a fused image in which each space pixel is determined from a set of space pixels from the input sources, including intensity-hue-saturation (IHS) (Carper et al. 1990, Chavez et al. 1991), Brovey (Civco et al. 1995) and principle component analysis (PCA) (Chavez andKwarteng 1989, Guo andLiu 2011).The transform domain methods convert the input images into a common transform domain, such as wavelet domain (Garguet-Duport et al. 1996, Audicana et al. 2004), pyramid domain (Liu et al. 2001) and random transform domain (Guo and Liu 2010).Fusion is then applied by combining their transform coefficients.
IHS method transforms three MS bands from red-green-blue (RGB) colour space into IHS space to separate spatial information (I) from spectral components (H, S).After replacing the intensity component with PAN image, the merged result is converted back into RGB space.It is obvious that all the fusion procedures are performed in the spatial domain.Although IHS method can preserve high spatial resolution of the PAN image, it severely distorts the spectral information (Prasad et al. 2001), because I component actually still has some spectral information.
DFRNT method firstly transforms both MS and PAN images to DFRNT domain.In DFRNT domain, high amplitude spectrum (HAS) and low amplitude spectrum (LAS) components carry different information of original images and different fusion rules are adopted in HAS and LAS components, respectively.Here, performing fusion in a transformed domain is an indirect change of the original image simultaneously based on space image features and different spectrum distribution features.The fused image is observed to preserve both spectral information of MS and spatial information of PAN.Comparing to IHS method, DFRNT method can preserve better spectral information and lower spatial information.Spectrum distribution of DFRNT is random and dispersive, which guarantees that good information is reserved (Guo and Liu 2010).
In this paper, our motivation is to combine DFRNT and IHS to generate the fused image with good spectral quality and high spatial quality.Meantime, the proposed DFRNT-IHS method is for joint spatial and transform domain, instead of fusion in spatial domain or transform domain respectively as in the conventional way.
The rest of this paper is organized as follows.The proposed method is described in Section 2. Section 3 provides the experimental results including test data, fusion methods for comparison and performance analysis.The conclusion will be drawn in Section 4.

DFRNT and its Properties
DFRNT for two-dimensional signal can be expressed as International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-7/W1, 3rd ISPRS IWIDF 2013, 20 -22 August 2013, Antu, Jilin Province, PR China The kernel transform matrix R  is written as Where  = fractional order of DFRNT T = period of eigenvalues with respect to the fractional order The randomness of DFRNT comes from the matrix V, which is generated by the eigenvectors of a symmetric random matrix Q The matrix Q can be obtained from an NxN real random matrix P by ( )/2.
DFRNT has several good mathematical properties, such as linearity, unitarity, index additivity and periodicity.Moreover, the randomness and dispersive spectrum distribution are the most important properties (Liu 2005).Due to the special properties, DFRNT transformed spectrum with different amplitude values spreads out, and many amplitudes can play roles in the fusion.Thus, it has a lower influence on the samestrength changes in spectra comparing to the concentrated distributed spectrum.This guarantees low spectral distortion.Meantime, majority of fusion results in DFRNT domain are acceptable, even distortion occurring at any position.The dispersivity confirms certain robustness.

Fusion Scheme
In this paper, MS and PAN images are already registered before fusion.The proposed DFRNT-IHS fusion algorithm is described as follows.
Step 1: The low-resolution MS image is firstly re-sampled using cubic interpolation to the same size as the high resolution PAN image.
Step 2: Three MS bands are transformed from RGB colour space into IHS space to separate spatial information (I) from spectral components (H, S).In order to preserve spectral information of MS as much as possible, keeping H and S components of MS image untouched, the spatial component of MS image and PAN image are fused in DFRNT domain.
Step 3: In order to further minimize the spectral distortion, histogram matching (Morovic 2002) is applied to the PAN image to make its brightness and contrast best match that of I component from MS image.
Step 4: The histogram matched PAN image and I component of MS image are transformed to DFRNT domain.
Step 5: It is known that the spatial detailed information of PAN image is mostly carried by its high-frequency components, while the spectral information of MS image is mostly carried by its low-frequency components.If the high-frequency components of MS image are simply substituted by the high-frequency components of the PAN image, the spatial resolution is improved but with the loss of spectral information from the high-frequency components of MS image.Meanwhile, discarding the low-frequency component of PAN image results in loss of spatial information.Since I component of MS image still has some spectral information, we further decompose I into low-frequency and high-frequency components.Then in different frequency component, different fusion rules are performed.In DFRNT domain, the high amplitude spectrum (HAS) component is the nominal low-frequency component, and the low amplitude spectrum (LAS) component is the nominal high-frequency component.As a rule, majority of spectrum energy is carried in the low-frequency component.Thus, energy is calculated through sorted spectrum amplitudes in a descending order.Then, ratios of different parts of energy to the total energy are calculated to extract HAS/LAS components.The ratio with the best vision and evaluation results as a whole serves as the separation threshold to extract the HAS/LAS components.
Step 6: In HAS component, the main objective is to preserve spectral information of MS image as much as possible while adding spatial details of PAN image into the fused result.Therefore, weighted combination is used in HAS component.
When the contribution of PAN information over the total information is high, a large weight coefficient is employed to add more spatial information of PAN to improve the spatial quality.Otherwise, MS information is the main portion and less spatial information of PAN is added to preserve spectral information as much as possible.The formula expression can be expressed as .Step 7: In LAS component, the main goal is to improve spatial details of the fused image.Thus, LAS component of PAN is used as the LAS component of the fused result.

HAS
Step 8: Fused image can be obtained by taking first inverse DFRNT and then inverse IHS to RGB space.

Test data, fusion methods for comparison
The images used for this experiment was acquired by the IKONOS remote sensing satellite over Calgary City in the west of Canada.This dataset is composed of MS bands 1-4, which are blue (0.45-0.53 mm), green (0.52-0.61 mm), red (0.64-0.72 mm) and near-infrared (NIR) (0.77-0.88 mm) bands, respectively, of size 128x128 and a corresponding PAN band (0.45-0.90 mm) of size 512x512.The MS and PAN images have 4-m and 1-m spatial resolutions, respectively.The colour composite of MS bands 3, 2, and 1 in red, green and blue (321RGB) is selected for the illustration.All MS bands are linearly stretched which leads to an effective use of the whole  For comparative purposes, two separate fusion methods were applied to the test data, including IHS and DFRNT methods.All resulting fused images had a 1m spatial resolution and should be compared with a real 1 m MS image in order to assess their quality.Since the latter did not exist, spatially degraded images were used in this paper.The original MS and PAN images were thus degraded to 16 and 4 m spatial resolutions, respectively, to simulate the fusion of MS and PAN images with a spatial resolution ratio of 4:1 (Wald et al. 1997).
The fused image (4m) is obtained by fusing the degraded PAN (4m) and MS (16m) images.The result of each fusion method is evaluated by comparing the fused images with the true MS (4m) image.

Performance analysis
Figure 1(c)-(e) illustrates the fused images by the proposed method, DFRNT method and IHS method, respectively.From these figures, it is observed that the fused images using DFRNT-IHS preserve most of the spectral information of MS images and improve the spatial resolution simultaneously.As for the fused images by IHS, it is obvious that the spectral information is distorted most and the spatial information is improved most.The tonal variation in IHS fused image is noticeable.In contrast, DFRNT and DFRNT-IHS fused images have similar tonalities to that of the reference MS image.For the DFRNT results, the spatial quality is a little inferior to that of DFRNT-IHS, and the spectral quality is almost the same as that of DFRNT-IHS.Observing the figures carefully, one can find that the proposed method contains more details than DFRNT method.Some edges appear visually sharper and clearer in the DFRNT-IHS fused image than that in DFRNT, for instance, the high way, the river edge near the high way in the lower-left corner of the scene and the block boundaries.
Based on the above observations, one can conclude that, the proposed DFRNT-IHS method performs slightly better or equivalently to the other methods in this article.

CONCLUSIONS
We have introduced an image fusion method for remote sense images.The proposed method is for joint spatial and transform domain, instead of fusion in spatial domain or transform domain respectively as in the conventional way.Moreover, the method combines DFRNT and IHS approach to obtain good spectral quality and high spatial quality.In order to prove the effectiveness of the proposed method, the qualitative visual analyses are introduced.The experimental comparing results demonstrated that our method is generally better than several existing methods.
Random transform for image fusion is a new and developing problem.The proposed method is effective, but it also can be improved.We will further extend our model for more accurate fusions.
pixel value range.The re-sampled MS and PAN images are given in figure1(a) and (b), respectively.