A Robust Registration Method for Multi-view SAR Images based on Best Buddy Similarity

Due to the influence of imaging angle and terrain undulation, multi-view synthetic aperture radar (SAR) images are difficult to be directly registered by traditional methods. Although feature matching solves the issue of image rotation and maintains scale invariance, these methods often lead to non-uniformity of interest points and may not achieve subpixel accuracy. The traditional template matching method makes it difficult to generate correct matches for multi-view SAR oblique images. In this paper, a multi-view SAR image template matching method based on Best Buddy Similarity (BBS) is proposed to solve the traditional methods' problem. Firstly, the initial correspondences between images are established according to the Range-Doppler model of SAR images. Secondly, a sliding window search is performed on the established correspondence, the BBS is calculated, and the subpixel locations of the peaks on the similarity map are estimated to achieve a fine match. In the calculation process of BBS, the SAR-ROEWA operator is used to suppress the speckle noise of SAR images. The experiment demonstrated that SAR-BBS can accurately match SAR images with large rotation angle. The peak value on the search window is significant. The registration accuracy of SAR-BBS outperforms the other state-of-the-art methods.


Introduction
Synthetic aperture radar (SAR) image registration is a fundamental step required for many applications, and its quality directly affects the effectiveness of subsequent tasks.At present, there is a lot of research on SAR image registration algorithms, and they are applied to SAR interferometry (InSAR) deformation measurement (Zou et al., 2009), SAR offset tracking (Strozzi et al., 2002), and block adjustment tie-points matching (Huber et al., 2010).The raw SAR images are usually in the oblique direction, which is affected by the motion attitude of the satellite platform and the imaging angle (Xiang et al., 2022b).SAR images acquired on multi-view satellite platforms have various advantages, such as complementing each other with perspective shrinkage and overlapping problems that occur in undulating terrain areas (Jing et al., 2018).However, it also brings new challenges to SAR image registration algorithms.SAR image registration methods can be roughly divided into three categories, pixel space-based matching methods, feature description-based matching methods, and deep learning methods (Sommervold et al., 2023).Among the traditional methods, the matching method based on pixel space is the most commonly used SAR image registration method, which accurately estimates the offset information between images by selecting the template, searching for the offset (which can be carried out in the time domain or frequency domain), and fitting the subpixel position, which is simple in principle and has high matching accuracy of homologous images.Common search criteria include normalized cross-correlation (NCC) (Pratt, 1974), sum of squared differences (SSD) (Mahmood and Khan, 2012), structure similarity index measure (SSIM) (Viola and Wells III, 1997), mutual information (MI) (Oliveira and Tavares, 2014), and other methods.However, the shortcomings of the pixel space-based matching method are also obvious, and the initial correspondences need to be given in advance, and the calculation time complexity is high (Sommervold et al., 2023).The featurematching method needs to select interest points according to certain rules, use a set of descriptors, and find the correct matches through a certain similarity measure (Li et al., 2020).Most of the newly proposed feature matching methods are developed based on classical SIFT (Lowe, 2004), such as SAR-SIFT (Dellinger et al., 2015), SURF (Bay et al., 2008), PCA-SIFT (Ke and Sukthankar, 2004), ORB (Rublee et al., 2011), etc., which can achieve good matching effects for images with rotation, lighting, scale, etc., and images without prior information, but nonnegligible mismatching and unevenly distributed interest points bring challenges to the method based on feature description matching.As an emerging SAR image registration method, deep learning has significant advantages in automatic operation and high efficiency (Sommervold et al., 2023).From CNN (Alzubaidi et al., 2021), U-Net (Siddique et al., 2021) to GAN (Strite and Morkoç , 1992), most methods extract features from the network and evaluate feature similarity, and the results are better than traditional feature matching accuracy.However, due to the training requirements of large samples, deep learning methods are difficult to ensure the robustness and timeliness of matching results.Ghannadi et al. (Ghannadi et al., 2020) proposed to extract statistical, frequency, and polarized texture features before image registration, construct the second optimal texture image using the weighted average of extracted features, optimize the weight coefficient by using the cat swarm algorithm, and input the results into SURF for matching, which increases the number of real matches and improves uniformity.Pallotta et al. (Pallotta et al., 2022) proposed a constrained least squares method to eliminate outliers in multitemporal SAR registration.Xiang et al. (Xiang et al., 2022a) proposed a geometric perception method to mask the overlapping and shadowed areas of SAR images after geocoding to maintain matching accuracy.Huang et al. (Huang et al., 2022) proposed a registration algorithm for the segmentation concept to reduce the geometric distortion and grayscale inconsistency between dual-band SAR.Wang et al. (Wang et al., 2022) applied SAR-SIFT to the image-matching problem in the large-scale uncontrolled orthorectification method of GF-3 images in China, and parallelized the process of computation.
Although the SAR image registration method above has achieved good registration accuracy, the existing methods still face great challenges for multi-view SAR image registration.Geometric and radiometric differences caused by different-angle imaging are key issues in multi-angle image matching.Although the feature matching method can ensure the scale and rotation invariance to a certain extent, it is not possible to extract highquality interest points in all regions, and it is difficult to meet the uniformity requirements of the control points of image registration.The template matching method focuses on the grayscale properties of the image itself.The control points are evenly distributed and the matching accuracy is high.However, the existing template matching method still has great limitations for multi-view SAR image registration.
In this paper, a robust method for multi-view SAR image matching is proposed.Firstly, for two multi-view oblique SAR images, the initial correspondences are generated using the provided satellite position information.Secondly, SAR-BBS based on the SAR-ROEWA operator is defined and used for the refined search process of correspondences.which can separate homogeneous structures with the same distribution in the mixed mode of speckle noise and image signal.Finally, fitting by polynomial model can achieve accurate registration of two-scene multi-view SAR images.

Overall process
The registration method proposed in this paper consists of two stages.The overall process is shown in Figure 1.Secondly, the template matching based on SAR-BBS for each correspondence is proposed.A SAR-Harris map based on the SAR-ROEWA operator is generated.Then the template of the point is extracted and is used to calculate the number of best buddy point (BBP) for correspondence.With the template swiping on Sense Image, every BBP on the search space is calculated.BBS is the collection of BBP in the search space.Finally, we searched the correlation peak on the BBS to get the most appropriate offset.

Initial correspondences generation
For the two SAR images obtained in different orbits, the interest points are usually matched through the image orbit parameter file provided by the satellite transmitter.The original SAR image is in a radar coordinate system, also known as a radar-Doppler coordinate system (Luo et al., 2022).The position of ground point  on the radar-Doppler coordinate system is expressed as   ( 0 , ),  0 is the time to acquire the pixel, and  is the oblique range between the target and the radar.The position of ground point P on the Cartesian coordinate system of space is represented as   (, , ) .Based on the SAR imaging equation, the following relationship is obtained.
where  and  are the position and velocity vectors of the satellite in the space Cartesian coordinate system at  0 moment,  is the working wavelength of the SAR sensor,   is the Doppler frequency of the SAR sensor, and , , and ℎ represents the major and semi-major axes, short semi-axes and earth height of the reference ellipsoid, respectively.
In secondary image imaging, whenever the time  0 changed by ∆ relative to the imaging moment, it is iteratively calculated in the following equation.(3) Based on Eq. ( 1), (2), the position of  on the secondary image coordinate system   (  ,   ) can be calculated by where   is the current position of the satellite calculated in Eq.
(3),  0 is the smallest oblique value on the SAR Range-Doppler coordinate system, and ∆ is the progressive step of the oblique range.Based on the Range-Doppler model, a rough offset from the reference image point to the secondary image can be obtained.

Template matching process
After estimating the coarse offset, some correspondences can be extracted from the imagery.On this basis, the template matching process is carried out to refine the correspondence of homogeneous points.The traditional template matching method makes it difficult to achieve high accuracy for multi-view SAR images at this time.Therefore, we improve the similarity of BBS (Oron et al., 2018) and make it suitable for SAR image matching according to the statistical characteristics of SAR noise.The BBS method calculates the number of BBP in two pointsets.When the BBS application matches the sliding window template, the template content is considered to have the greatest similarity when the number of BBP in the two pointsets is the peak in the entire search area.BBS is defined as follows.
(7) (  , ) in Eq. ( 7) is a distance function similar to Euclidean distance, cos distance, etc.The distance function used by Dekel et al. is where  represents the pixel value of the point,  represents the location of the point, and for the constant .We use the suggested empirical value that  = 2. Since BBS is sensitive to noise, in SAR-BBS, we processed the contents of the points set  in Eq. ( 7).Since the noise model of SAR images is multiplicative, the ROEWA operator is used to filter the input points set in the horizontal and vertical directions (Xiang et al., 2018), respectively.ROEWA is expressed as where  ,  represent the number of points in the twodimensional points set,  is the spatial scale parameter, and (, ) represents the pixel value of the current point at the coordinates on the image.Based on this, the gradients in the horizontal and vertical directions are expressed as After achieving the gradient extraction and noise suppression of the points sets  and  , the down-sampling operation is performed spatially to reduce the computational overhead in the process of calculating the number of BBP by window sliding.

Experimental datasets and environments
In this paper, two TerraSAR images obtained at different angles and different orbits are used to verify the matching accuracy of the proposed SAR-BBS algorithm.The experiment is divided into two parts.The first is the template similarity experiment and analysis, which is compared with similarity measures such as NCC and MI.The second is the matching accuracy analysis, which is compared with two state-of-the-art multi-view image registration algorithms: SAR-SIFT and RIFT.The relationship between the coverage area and the overlap of the real SAR image for the matching experiment is shown in Figure 2.

Evaluation Criteria
In the experiments, correct matching ratio (CMR) and root mean square error (RMSE) were used to quantitatively evaluate the results of the registration method.Here, CMR is defined as where  represents the correct number of matches and  represents the total number of matches.The correct matches are calculated by the fast sample consensus (FSC) algorithm (Wu et al., 2015).

Comparison of template matching similarity
Firstly, we apply different template-matching similarity measures to some regions in the real SAR images to test the effectiveness of the SAR-BBS template matching proposed in this paper for SAR images with different viewing angles.The most widely used NCC and MI similarity measures were selected for comparative testing, and the results are shown in Figure .3.
According to the results shown in Figure .3, the selected template region has a large angular deformation and plane displacement, which brings great challenges to the similarity measure of template matching.For the template on the reference patch, a sliding window search is performed on the sensing patch, and the similarity information calculated on the search window forms a similarity map.The SAR-NCC map peaked on both test patches, which had the largest number of BBP in points sets and was considered the best match within the search area.NCC map performed poorly in search results on the first patch, did not show a unique peak, and was mixed with multiple low NCC value noise peaks, which means that the NCC method failed in this region.
The NCC method has two peaks on the second test patch, one is a relatively stable true peak, and the other peak is at the edge of the window, which may be due to the pseudo peak caused by the matching boundary.For the first true peak, it is easy to see how much distortion the image is observed, and it does not reflect the true offset position of the image.For the MI method, both test patches fail and the similarity map does not show an exact matching peak point.NCC and MI methods rely on the value of image pixels, and it is difficult to match the correct correspondence for images with rotation and local structural distortion.The SAR-BBS method uses the ROEWA operator to suppress SAR images with speckle noise, and adopts a more robust BBP expression for image structure information, which can successfully cope with image matching with large rotation angle and viewing angle distortion.

Comparison with state-of-the-art
The       2, it is obvious that SAR-BBS outperforms other methods at CMR and RMSE.As a template matching method, SAR-BBS can initially control the interest point according to the correspondence established by the Range-Doppler model, and then use the sliding window to search for the corresponding peak value, so as to refine the matches.BBS processes the template image as points set, reduces the dependence on the pixel position information in the image, extracts a subset with a statistically higher probability of the same distribution through the distance relationship between the points sets, and increases the robustness by sliding the window.However, the BBS method is very complicated in calculating the distance of the points set, which also brings great time consumption.

Conclusion
In this paper, a robust registration method is proposed for SAR images acquired under multi-viewing conditions, and the accuracy and effectiveness of the proposed method are highlighted by comparing it with state-of-the-art and commonly used methods.The contributions made in this article are as follows.SAR-BBS was proposed and applied to the registration of multi-view SAR oblique images.The ROEWA operator was used to suppress the interference of SAR spot noise on the matching template.Image templates are converted to points sets, and the effect of attenuating rotation on image matching.BBP is calculated to count the number of pixels with the same distribution form.The template-matched sliding window is used to increase the robustness of BBP to noise.By comparing with NCC and MI methods, the effectiveness of BBS on rotating templates is verified.The comparison with SAR-SIFT and RIFT shows that SAR-BBS has higher matching accuracy and a reliable number of points for multi-view SAR images.
Firstly, the initial correspondences on reference and sense image are generated based on the Range-Doppler model.The interest points on the reference image are generated by regular grid division, which is the same as most template matching methods.The interest point is connected to the counterpart on the sense image by the RD model which uses external DEM.

Figure 1 .
Figure 1.Processing of proposed method which consists of two stages.
and  are part of the points set of the template image and the search window, respectively, expressed as  = {  } =1  and  = {  } =1  ,  and  represent the number of elements of the points set  and  , respectively. is a pair of point {  ∈ ,   ∈ } satisfies the expression of the best buddy (  ,   , , ) = { 1 (  , ) =   ∧ (  , ) =   0 ℎ , qualitative results compared with SAR-SIFT and RIFT are shown in, the SAR-SIFT method is too focused on the extraction of interest points, which causes the local deformation information to be enlarged to the overall image area when the transformation model is fitted, resulting in unnecessary registration deformation.Figure.5 shows that the RIFT method is still challenging in matching SAR images obtained for different viewing angles, and the coarse matching results without error point checking have more errors.As shown in Figure.6, the SAR-BBS method has a uniformly distributed matching number because it regularly selects grid points distributed under the SAR image coordinate system.The matching results of SAR-SIFT and RIFT are coarse differences rejected by the FSC method.The transformation model is set as affine.The quantitative experimental results are shown in Table2.

Figure 3 .
Figure 3. Template similarity results on multi-view SAR image patches.
Table 1 lists the parameter information of the images.
Comparison of the relative accuracy of registrationBased on the experimental results in Table