PARTICLE SWARM IMAGE MATCHING ON EPIPOLAR LINE

This paper presents a method for determining the 3D position of an image point on a reference image using particle swarm optimization (PSO) to search the height (Z value) that gives the biggest Normalized Cross Correlation (NCC) coefficient. The searching area is in the surrounding of the height of the image point. The NCC coefficient evaluates the similarity with the image point and a corresponding point on an epipolar line in the search image. The position of corresponding image point on the epipolar line is determined by the height point on a sloping line locus. The PSO algorithm starts with a swarm of random particles. The position of each particle is a potential solution in the problem space. Each particle is given a randomized velocity and attracted toward the location of the best fitness. The position of each particle is iteratively modified by adding a newly computed velocity to its current position. The velocity is updated by three factors which are two attractions from local best position and global best position, two strengths of the attractions, and two uniform random numbers for each attraction. The iteration will stop when the current solution is convergent. The time of computation is highly related to the range of height and the interval of height enumeration when the approach to find a corresponding image point of an image point on a reference image is based on the height enumeration along sloping line locus. The precision of results can be improved by decreasing the interval of height enumeration. This shows the limitation of the enumeration method in the efficiency and accuracy. The issue is overcome by a method of using PSO algorithm. The proposed method using different parameters such as the size of image window, the number of particles, and the size of the height searching range has been applied to aerial stereo images. The accuracy of tested results is evaluated on the base of the comparison to the reference data from the results of least-square matching being manually given initial points. The evaluation result shows that tested results has given a solution to a level of less than 1 centimetre without using refined image matching method. The same level of accuracy can reach even when the searching range is bigger than 90 meters. But the difference of image window size may lead to the change of the matching result. And, without the procedures of both coarse-to-fine hierarchical solution and refined image matching method, the algorithm still can give the same accuracy level of least-square image matching resulting. This method also shows its ability to give reasonable matching results without manual assistance.


INTRODUCTION
Over the last decade, laser scanning techniques have shown its ability and advantage to produce dense point cloud in short time.It seems that traditional photogrammetric techniques have been replaced and less role to play.However, these laser scanning techniques have its limitation on determining the exact breakline position of object (Nex and Rinaudo, 2008).As a result, many researches have aimed to improve the performance of image matching techniques so that dense point clouds can be produced by using imagery.Image matching is one of the key technologies for producing automatically digital terrain models by digital photogrammetry.Generally image matching is only to find image coordinates of conjugate image points.The image coordinates are used to determine the spatial points of the conjugate image points.An orthoimage can be produced by finding images of grid points interpolated from the spatial points.However, there is one main problem of giving a reasonable initial point for starting image matching process in an area of two dimensions.The two dimensions can be reduced to 1 dimension when the condition of epipolar lines or Vertical Line Locus (VLL) is applied.VLL has been applied to finding the height of a spatial point.VLL is a vertical line of the spatial point.The height is determined on the base of height enumeration along the line.The determination is made when the height gives the most similar corresponding images of the spatial point.The searching area is then reduced to one dimension.VLL were applied to obtaining a digital elevation model (Kou, 2010), the building elevation (Liu and Wu, 2006), digital city in 3-dimension data (Jacobsen, 2006).According to the results of these applications using search method based on height listed one by one method, the search speed and the accuracy of the search resulting height are highly related to the search range and the interval of height enumeration.Ant Colony algorithm was used in the VLL methods to improve search speed and accuracy (Guo and Chen, 2006).Its experimental results show that better results can be given but search time is still too long.The process of determining a height by VLL starts from a point in spatial space.The method is generally applied to produce the height of grid points.Particle swarm image matching applied to VLL has been developed by the Hsia (2013).The quality of digital terrain models is able to be improved by adding the height information of feature points with that of grid data.This research method is used to collect the height of feature points under the assistant of epipolar line.The epipolar line in this research method is determined by height points on Sloping Line Locus (SLL).The SLL was extended from VLL (Zhao, Liu, and Liu, 2006).The process of height collecting by SLL starts from a reference image point in a target image.Its result is then the height of the spatial point corresponding to the reference image point.The height is determined on the basis of the degree of the similarity between the reference image point and a search image point on the epipolar line corresponding to height points on SLL.The approach to find a corresponding image point of a reference image point can be based on the height enumeration along SLL.Each height determined a candidate corresponding image point on an epipolar line on a search image.The corresponding image point is found when its height gives the biggest similarity.The time of computation is highly related to the range of height and the interval of height enumeration.The precision of results can be improved by decreasing the interval of height enumeration.Generally, image strategies, such as coarse-to-fine and triangulated irregular network (TIN), are combined with the approach in order to narrow the enumeration range.A refined image matching procedure using a least-square matching method is adopted to achieve better results and confirm matched results.This shows the limitation of the enumeration method in the efficiency and accuracy.The issue is overcome by a method of using PSO algorithm in the approach.The method of Particle Swarm Optimization (PSO) image matching on epipolar line can be applied to determine 3D positions of feature points, edge-end points and grid points on the imagery.This method shows its potential for producing high dense point clouds and determining the exact break-line position of object.PSO is a method that simulates bird swarm intelligent behaviour of searching foods.The experience-sharing mechanism of the swarm intelligence makes this method to have local and global search capabilities.The PSO method is therefore one of good searching methods for searching solution.The PSO algorithm starts with a swarm of random particles.The position of each particle is a potential solution in the problem space.Each particle is given a randomized velocity and attracted toward the location of the best fitness.The position of each particle is iteratively modified by adding a newly computed velocity to its current position.The velocity is updated by three factors which are two attractions from local best position and global best position, two strengths of the attractions, and two uniform random numbers in the range [0, 1] for each attraction.The iteration will stop when the current solution is convergent.This method was applied on medical image registration (Pei, Tian and Yang, 2007) and showed that the method is simplicity, reliability and features such as uneasily stopping at a local optimum value.Inertia weight PSO with fuzzy model was used for image matching method in order to increase noise immunity and improve the performance of convergence (Yang, Liu and Xin, 2008).Two ways to increase matching success rate was carried out in four areas such as cities, towns, valleys and farms (An, Chen and Wang, 2009).One way was to increase the number of particles and the other way was to adjust learning factor in linear function.An increase in the number of particles is able to improve the matching success rate.However it is not easy to find a reasonable number of particles for different matching cases.The aims of this study on image matching are to increase the search range and the matching success rate by integrating inertia weight PSO image matching with epipolar method and its particles distribute in a uniform form.

Particle Swarm Optimization method
Particle Swarm Optimization (PSO) algorithm (Kennedy andEberhart, 1995；Eberhart andKennedy, 1995) inspired by the food seeking behaviour of a flock of birds.A flock of birds searching for food spreads in the area with food and the birds share the information of their food seeking result and positions.The information assists birds in gradually moving to areas with more food and then birds can easily gain access to food.Food seeking area is considered to be the area with potential solutions when PSO is used in applications.Birds are considered to be particles.The food information of the position (called potential solution) where each particle is stayed is considered to be fitness evaluation.Each particle keeps its best fitness evaluation called local best solution.The best of local best solution is considered to be global best fitness evaluation called global best solution.Particles share these two best solutions leading them to move around in the area.The speed and direction of each particle's movement are adjusted according to the two best solutions.Consequently particles are able to move toward the best solution.The PSO algorithm can be described as two equations as follows: (1) The variables of V i d k and Z i d k denote separately the speed and the position of the particle "i" in d-dimension space at its "k" times.P i d means the position of the best solution of the particle "i" and P g d means the position of the best solution of all particles in d-dimension space.Two learning factors (c 1 and c 2 ) denote separately the weighting of attraction from P i d and P g d positions.Two random numbers (r 1 and r 2 ) are selected as uniform random numbers in the range [0, 1].These two random numbers let equation ( 1) have random feature and be useful for space exploration.The speed of each particle is updated at iteration k+1 by adding the attraction as shown in equation ( 1

Inertia Weight Particle Swarm Optimization Method
Inertia weight particle swarm optimization (IWPSO) algorithm modifies the equation ( 1) to be the equation ( 3) by weighting the previous speed in order to enhance searching capability of particle swarm (Shi and Eberhart, 1998a;1998b).The inertia weight w is used to balance the global and local searching.The searching behaviour of the particle swarm adjusted gradually from global search to local search according to the decreasing in weight (Eberhart and Shi, 2000).The maximum speed Vdmax is given as the same value as Zdmax in order to reduce the number of parameters (Eberhart and Shi, 2001).
w=w e +(k max -k)(w st -w e )/ k max (4) Where k max : the maximum number of iterations, w st : the initial value of inertia weight, w e : the final value of inertia weight This contribution has been peer-reviewed.doi:10.5194/isprsarchives-XL-4-87-2014

Particle Swarm Image Matching
The searching area is as the same range as the normalised crosscorrelation (NCC) image matching when IWPSO algorithm is applied to NCC image matching.Particle swarm searches the best matching points in 2-dimension image space.The search range, [Zdmin, Zdmax], in x or y dimension depends on the coordinate scope of the search area.Each particle's position computes a NCC value called fitness value.The local and global best fitness are respectively the biggest NCC value of each particle and all particles.Particle swarm moves in the search area to search for the location of the largest NCC value of the optimal matching position.Particle swarm moves until the difference between two successive global best fitness values is less than 1.0e-8 or the number of iterations equals to the maximum number of iterations (kmax).
The computing amount and matching success rate of the particle swarm image matching highly depend on the initial distribution and moving speed of particle swarm.The uniform distribution in grid pattern is generally used in order to assist the searching performance.In each grid cell there is a particle.The position and speed of each particle in grid cell are given in random manner so that the distribution of the initial position of the particle swarm can be simultaneously in the manner of both randomness and uniformity.

Epipolar Line
Vertical lines locus (VLL) is one of methods for collecting heights of ground points from stereo images.The VLL method determines the height by searching a correct point along the vertical line of the ground point.The sloping lines locus (SLL) is developed on the basis of VLL.The SLL determines the height of a reference image point on the target image by searching a correct height point along the sloping line passing through the reference image point and perspective center of the image.The correct height point is determined by finding the best similarity between the reference image point and a search image point on an epipolar line determined by the sloping line and collinear equations.The way to search the correct height point generally uses height enumeration.The accuracy and efficiency of the height point searching highly depend on the interval of height enumeration.It is not easy to give a suitable interval for helping the accuracy and efficiency at the same time.

Particle Swarm Image Matching on Epipolar Line
Particle swarm image matching on epipolar line is that IWPSO image matching is applied to SLL.The aims of this research method are to reduce the search dimension from two to one, and to improve the searching performance.Particles move along the sloping line of a reference image point to search the most similar search image point on an epipolar line on the search image in order to determine the height of the reference image point.The search range is as small as possible and must cover the true height in order to avoid searching wrong height.However, the search range should be large enough because the height information can't know precisely.The fitness value of each particle is NCC coefficient for measuring the similarity of the reference image point and a search image point corresponding to a height point in the search range.The modifications of the speed and position of the particle are described respectively as equations ( 3) and ( 2).The height of a ground point is fund when the global best fitness is obtained.The algorithm of this research method for determining the height of a reference image point is described as following: (1).Start and set k=0, threshold value (ε), maximum number of iterations (kmax), number of particles (m), initial value (wst=0.9)and final value (we=0.4) of inertia weight, learning factors (c1=c2=2.05),maximum speed (Vmax), the range of potential solutions [Zmin, Zmax], the values of interior and exterior orientation of camera (2).Produce initial position values (Zi(0),i=1,2,... ,m) for particles (3).Produce initial speed values (Vi(0),i=1,2,... ,m) for particles (4).Determine photo coordinates ((x"i(0), y"i(0)) of initial position (Zi(0)) by collinear equations (5).Calculate fitness value (Ti( 0)) (The objective function Ti(Zi) keeps the maximum of NCC coefficient of the particle (i).) of position (Zi( 0)) (6).Set Pi = Zi(0) (Pi is the local optimal solution of particle i) (7).Determine the globally optimal solution Pg as the position of T(0) (T(0) =max ((T1(0), T2 (0),... , Tm( 0)) ( 8).Update k by k=k+1 (9).Update inertia weight w by equation (4) (10).Update Vi(k) and Zi(k) by equation ( 3) and (2) (11).Determine photo coordinates ((x"i(k), y"i(k)) of position (Zi(k)) by collinear equations ( 12).Calculate fitness value (Ti(k)) of position (Zi(k)) ( 13).Update Pi and Pg (14 The fitness values in steps of ( 5) and ( 12) are the NCC coefficient calculated on the basis of two block images.The center of the two images locate at the image coordinates of photo coordinates ((x' i (k), y' i (k)), (x" i (k), y" i (k)).The (x' i (k),y' i (k)) are the photo coordinates of the reference image point and the ((x" i (k),y" i (k)) are the photo coordinates of a search image point.The image coordinates are determined by the affine transformation equations described as the equation (5).
Where x, y: the photo coordinates of a search image point, a, b, c, d, e, f : affine transformation parameters, L, S : image coordinates of the photo coordinates (x, y) The factors to affect the efficiency and accuracy of this research method are the initial position and speed, maximum speed of each particle, and the number of particles.A uniform distribution of the particle swarm is adopted to ensure that particles are able to move around the whole potential solution range.The whole range is therefore divided in small cells.There is a particle in each cell and its initial position is given randomly.The initial speed of each particle is also given randomly in the range of [-Vmax, Vmax].The smaller the maximum speed is, the greater opportunity the best solution is visited by particles.It is not easy to decide a suitable number of particles, since it highly depends on the size of search area.
According to the previous experience (Hsia, 2011), the number of particles can be determined by the density of the particles.The density is a function of the number of particles and the number of cells.The length of cells equals to the degree of accuracy level expected.

EXPERIMENTAL DESIGN AND RESULTS ANALYSIS
This research image matching method has been tested using aerial stereo images in order to know its efficiency and accuracy.
The experimental design and resulting analysis are presented in following sections.

Experimental Design
To reveal the effect of image texture on this research method, six ground points of different textures were selected from the area covered by aerial stereo images.The images of the six points are shown in Figure 1.

Results Analysis
The resulting heights of each image window size case with different density and search range were compared to the reference height.The minimum difference between reference height and the best resulting height of each image window size for each point is shown in 1.The tested results in Table 2 are shown in Figure 2 and 3. Table 1 shows that the maximum minimum difference is 0.09 meter, the maximum average of minimum differences is 0.04 meter and the minimum difference for each point is less than 1 centimetre.It tells that the best of resulting heights can reach the level of less than 1 centimetre and the effect of image texture, window size, particle density and search range leads to reach the level of less than 10 centimetres.At least one result of being less than 1 centimetre for each point shows that a good result for each point is able to obtain by using different window size and particle density.It also shows that a good result can be given when the window size is suitable and search range is bigger than 92 meters.Most of values bigger than 0.05 meter, in Figure 2, are those of window size bigger than 23×23.It shows that an increase in image window size can result in a decrease in accuracy of resulting heights.The minimum difference averages less than 2 centimetres, in Figure 3, are those of B, E and F points.The image points of artificial object texture can give better accuracy of tested result.The range of difference between reference height and all resulting heights of each image window size for each point and its average are shown in Table 2. Figure 4 also shows the tested results in Table 2.In Table 2 the smallest difference range for points, A, B, C and D, was 0.00 meter (i.e. less than 1 centimetre) and that for points, E and F, was 0.02 meter.It means that the effect of particle density and search range on the precision was less than 1 centimetre or 3 centimetres for tested points.Most difference ranges in Figure 4 are less than ten centimetres.Most of big difference ranges generally were given by the window size of 25×25.This shows that the precision of this research method can be within ten centimetres when image window size is suitable.Big window size may lead to a decrease in the precision.

CONCLUSIONS
It has been shown that height can be obtained by this research method even when search range is biger than 90 meters.The method does not need the procedures of both coarse-to-fine hierarchical solution and refined image matching method to increase its success rate and accuracy.The resulting accuracy can achieve the level of less than 1 centimetre when the window size is suitable.It shows this method has the ability to collect the 3D position of an image feature point.The image feature point can be part of break-line position of object.
).The speed is limited in the range[-Vdmax,  Vdmax].Vdmax represents the maximum speed in d-dimension space.The position of each particle is then changed by adding a speed V i d k to the current position Z i d k as shown is equation (2).The position is limited in the range[Zdmin, Zdmax].Zdmin and Vdmax represent separately the minimum and the maximum position in d-dimension space.

Figure 1 .
Figure 1.The images of points (L, R: left or right image)Eight kinds of image window size were used to determine the NCC coefficient.The window sizes were respectively 11×11, 13×13, …, 25×25 pixels.The minimum and maximum of search range were 10 and 92 meters.The minimum number and maximum number of particles were 4 and 20.Twenty six kinds of search ranges and seven kinds of number of particles were used in the tested.Therefore the minimum density and maximum density of particles were 1 particle within 7 meters and 1 particle within 1 meter.The accuracy of tests was evaluated on the base of the comparison between the tested results and the reference height measured by least-square matching being manually given initial points.

Figure 2 .
Figure 2. The minimum differences between matching results and reference data

Figure 4 .Figure 5 .
Figure 4.The difference ranges between matching results and reference data The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, VolumeXL-4, 2014   ISPRS Technical Commission IV Symposium, 14 -16 May 2014, Suzhou, China

Table 1 .
The minimum differences and its averages (Unit: meter)

Table 2 .
The difference ranges and averages (Unit: meter)