OBJECT-ORIENTED CHANGE DETECTION BASED ON SPATIOTEMPORAL RELATIONSHIP IN MULTITEMPORAL REMOTE-SENSING IMAGES

In this paper a novel object-oriented change detection approach in multitemporal remote-sensing images is proposed. In order to improve post classification comparison (PCC) performance, we propose to exploit spatiotemporal relationship between two images acquired at two different times. The probabilities of class transitions are used to describe the temporal dependence information, while the Markov Random Field (MRF) model is utilized to represent the spatial-contextual information. Training sets are required to get initial classification results by maximum likelihood method (ML).Then an estimation procedure: iterated conditional mode (ICM) is present to revise the classification results. Change detection (change/no change) and change type recognitions (from-to types of change) are achieved by compare classification maps acquired at two different times. Experimental results on two QuickBird images confirm that the proposed method can provide higher accuracy than the PCC method, which ignores spatiotemporal relationship between two images.


INTRODUCTION
Remote-Sensing images with large cover area and short revisit time are widely used in land use/land cover change detection since its birth.Land use/land cover change detection using remote sensing technology has always been an important research topic in remote sensing applications [1].According to the basic unit of the land use/land cover changes, change detection methods can be cataloged into two types: pixel-based method and object-oriented method.Many change detection algorithms belong to the former approach [2]- [10].The most popular methods of them are the image differencing algorithm [3] - [5],change vector analysis [6]- [10] and postclassification comparison [11].The image differencing algorithm is an unsupervised technique as well as change vector analysis, while PCC is a supervised technique for change detection.The image differencing algorithm is usually divided into three steps:1) pre-processing.The two images should be coregistered so that pixels with the same coordinates are corresponding to the same area on the ground.2) Difference  image generation.The two images are subtracted, pixel by pixel, to generate a further image difference image.3) Threshold decision.Under the assumption of few changes between two different images, changes are located in the tails of probability density function of the difference image.They can be detected through a certain threshold gained by ostu algorithm or bayesian theory.Change vector analysis can not only detect changes but also provide "from-to" types of change information using the gray  * Corresponding author: Liang Li (1987-), male, PhD candidate, majors in intelligent interpretation of remote sensing imagery.E-mail: liliang1987wuda@163.com value of all bands.The gray values corresponding with different bands of a pixel at each time compose a vector.Then for each couple of pixels, change vector is computed by subtracting vectors acquired at the two times.The changes in land-cover classes are determined based on both the magnitude and the direction of the change vector [6].The limitation of image differencing algorithm and change vector analysis is the false alarm caused by difference in light conditions, atmospheric conditions, and sensor calibration at the two acquisition images concerned.In addition, the acquisition of the optimal threshold value is also a bottleneck.However, PCC approach can overcome these problems.Two images are classified separately according to the training sets.It performs change detection and change type recognitions by comparing the classification maps between two images.The accuracy of this method depends on the performances of the classification maps.Therefore, training sets play an important part in this technique.Typical training sets are required.With the development of remote sensing sensors, the spatial resolution of remote sensing images has been greatly improved.Therefore, abundant spectral information and texture information are contained in high-resolution imagery.Pixel-based technique for change detection becomes ineffective in highresolution imagery.This is due to the fact that pixelbased technique doesn't take into account the spatialcontextual information in the multitemporal images considered.The object-oriented method is more suitable for high-resolution imagery [12]- [16].Objects are composed of a set of pixels which are adjacent in space and spectrally similar to each other.Thus object which contains various kind of information such as spatial, spectral and shape information, can be regarded as a homogeneous entirety.Image segmentation and GIS data are two ways to acquire the objects.The basic unit of object-oriented method is an object.In this way, it can suppress the salt and pepper noise effectively in high-resolution imagery.Some objectoriented methods have been developed for change detection, such as the multilevel model [12], Kolmogorov-Simirnov test technique [13], OCI method [14], post-classification comparison [15].Although the use of object-oriented postclassification comparison is widespread because it can detect changes and provide from-to types of change,it exhibits two major drawbacks: lack of temporal dependence information between the two images of the same area acquired at the two times and lack of spatial-contextual information in single image.In order to overcome these problems, object-oriented PCC technique considering spatiotemporal relationship is proposed in this paper.First, image segmentation with two images using marker-based watershed algorithm is applied to obtain image objects.According to acquired training sets, object-oriented classification using ML method for each image are performed to get initial classification maps.Next, spatiotemporal relationship which involves both temporal dependence information and spatialcontextual information, is presented to revise compound classification maps.Finally, change detection and change type recognitions are achieved by comparing the revised two classification maps.The processing flow of the proposed method is shown in Figure 1.This paper is organized into five sections.In Section  [22], [23].The higher priority of a pixel is, the lower is the gradient value.This algorithm can be summarized as follows: (a) labeling the markers and initialization of the priority queues, (b) region growing.A pixel p is extracted from the end of the highest priority queue.The neighbor pixels of p which are still outside the queue are put into the queue according to the gradient value, however, when the gradient value is lower than the priority of p, this neighbor is put into the queue with the same priority of pixel p.If pixel p has in its neighborhood one and only one labeled region, it is aggregated to this region and if pixel p is a neighbor of two regions with different labels, p is a boundary pixel and gets a special label.For a more detailed description of the marker-based watershed approach we refer the readers to [22].
In () Tk indicates the classification result of k-th object in q T which denotes the results of q-th random experiment at 2 t .Last, let us define the neighbor system of the k-th object as ( ) { , According to the Bayesian formulation, the optimal sets of classification results t C and h T that minimize the classification error can be derived by using the following decision rule: 12 , ( , ) arg max{ ( , ) / ( , )} The couple of classification results ( , ) th CT that provides the maximum in (1) also provides the following maxima: where ( , ) [( , ) / ( , )] is the conditional density function of the feature vectors in the images 1 X and 2 X ,given the classes p C and q T ,and 12 ( , ) P X X can be discarded since it is independent of p C and q T .Formula (2) is equivalent to the formula (3).T .Thus the following equation is expressed: 2)The following conditional independence are assumed: ( ( ) / ( )) q p X k T k are the conditional probabilities at the two times.
The assumption analysis to that defined by ( 4) was introduced in reference [11].It is worth noting that the effectiveness of assumption ( 2) is difficult to deduce in theory.The formulas ( 5)-( 7) are approximate formulas and more critical than the others.However, researchers have been applied the formula ( 6), (7) to change detection application [4] [11].It can be regarded as a good trade-off between computational efficiency and estimation accuracy.By substituting (4)-( 7) into (3), we can obtain following formula:

Markov Random Field
() p PC is a prior probabilities of the classification results for all objects in the image acquired at time 1 t .The objects involved in the image at 1 t are not spatially independent.The object is likely to be surrounded by objects with the same class, for example, the lakes prefer to adjacent to the lakes rather than the buildings.MRF technique is used to model the spatial-contextual information between objects [23].() p PC can be expressed as: where  is potential function and  is the weight of spatial-contextual information in the process.The higher is the adjacent probability of the two objects, the smaller is the  function value. is expressed as : ( ( ), ( )) ( ( ) / ( )) By applying ( 9)-( 11) into ( 8) and discarding the constant, the following formula can obtain: We can transform ( 12) into (13) using log function.
According to (13),we need to estimate the conditional probabilities

Conditional Probabilities
The conditional probabilities

Land-cover class transition and adjacency matrixes
The probabilities of land-cover class transitions which represent the temporal dependence information between two times establish the following mn  matrix.
where the element ( / ) ji P  with row i and column j in the matrix denotes the probability of an object belongs to land-cover class The conditional probabilities of the land-cover class adjacency which represent the spatial-contextual information at the time 1 t compose the following mm  matrix.
 where the element ( / ) ji P  with row i and column j in the matrix represents the probability of an object belongs to land-cover class j  at time 1 t ,on the condition that it is adjacent to the land-cover class i  at the same time.
In order to estimate all the probabilities in matrix T and A, training sets are required.The training sets should satisfy the following conditions:1)training sets should correspond to the same area on the ground at the two times;2)training sets should contain all the probabilities of the land-cover class transitions and adjacencies [11].In practical terms, it's impossible to select the training sets to satisfy above mentioned conditions.Different places at different times with different land policies formulated by local government leads to different matrix T and A, for example, in the period of the urbanization and city constructions, the probability of farmland transferred to the building is higher than it transferred to the lake.As a result, historical dataset can't be used to get current land-cover class transition and adjacency matrixes accurately.Consequently, we estimate the probabilities in matrix T and A from the whole image.

Iterated conditional mode for compound classification
The number of all possible classification results at the two times is () N mn .It's impossible to find optimal classification results which verify (13) using exhaustive method since () N mn is enormous.We propose a fast method by iterated conditional mode (ICM) technique [25].According to ICM, the land-cover classes of the current couple of objects at the two times is determined by the land-cover classes of the other couples of objects and the feature vectors of all couples of objects.ICM technique is realized according to the following steps.1) Initial classifications of the two images using ML approach independently.
2) Compute the land-cover class transitions and adjacencies matrixes depend on the initial classifications.
3) Update the classification results of the two images which verify (13), for all couples of objects at two times.4) Repeat 2)-3) until the convergence.

Change detection and change type recognition
The optimal compound classification results t C and h T at the two times can be obtained using

Data set description
The study area is located in the Wuhan city in the center of China.Two images with the same area of 758×623 pixels are acquired by QuickBird satellite in 2002(time t1) and 2005(time t2).The QuickBird imagery is made of four multispectral bands with a spatial resolution of 2.44m: the red band, the green band, the blue band, and the near infrared band and one panchromatic band with a spatial resolution of 0.61m.The bands selected for change detection are four multispectral bands with a spatial resolution of 2m after resampling.The dataset used in the experiments is shown in Figure 2 .The number of objects extracted from image segmentation by marker-based watershed method is 625. Figure 3 shows the image segmentation results.

Classification results
There are four classes in the image acquired in 2002 and five classes in the image acquired in 2005.Therefore we selected four classes: lake, building, Figure 3. Image segmentation results(black lines denotes the boundary of the objects).road, farmland at time t1 and five classes: lake, building, road, farmland, bare land at time t2.Because of the complexity of the remote sensing imagery, subclasses with different spectral information exist in one land-cover class, i.e., the lake contains many subclasses with different sediment concentration and chlorophyll content.This problem will reduce the accuracy of the image classification.In order to solve this problem, training sets containing all subclasses should be selected.Figure 4 shows the classification results at the two times.The object belongs to the lake on the ground truth involved in red rectangle A is classified to the building in Figure 4(a) using ML method, while it is classified to the lake in Figure 4(c) using the proposed approach.The reason is that this object has similar spectral features as the building, but it's more likely to belong to the lake, considering the spatial-contextual information that the object adjacent to it belongs to the lake.Many objects belong to bare-land on the ground truth involved in red rectangle B of Figure 4 The land-cover class transition and adjacency matrixes after four iterations are shown in Table 1 and 2. The land cover at time t1 and t2 are given by the columns and rows in Table 1.The land cover class and the adjacent class at time t1 are given by the columns and rows in Table 2.It is easy to observe in Table 1 that the lake tends to transform to other class while the building seldom converts to other class between two different times.Table 2 shows that objects have aggregation effect in space which indicates that objects are more likely adjacent to objects with the same land-cover class, for example, the probability that an object belonging to lake under the condition that the adjacent object belongs to the lake is as high as 0.72.However the probability that an object belonging to building under the condition that the adjacent object belongs to the road is 0.68 which is very high.The reason is that the buildings usually lie on both sides of the road in China.
The error matrixes of land-cover transitions for PCC method and the proposed method were computed in Table 3 and Table 4. Land-cover transitions from the ground truth area are given in the rows in two tables, while the land-cover transitions detected by pcc method are given in the columns in Table 3 and the land-cover transitions detected by the proposed method are given in the columns in Table 4 ( ij denotes the object transform from i  at 1 ).
Compared the two tables, we can see that the proposed approach can provide better accuracy than PCC method.In particular, the accuracy of 15(lake-bare land) and 22(building-building) was significantly improved.The reason for that can be described as follows: the building class, the road class, Nonetheless, the probability of the lake at t1 converting to bare land at t2 is high and the building is more likely adjacent to the building than the road in space.The proposed approach considering the temporal dependence information and the spatial contextual information can distinguish the above three classes and get an improved accuracy.Table 4.Error matrix of the land-cover transitions of the objects by the proposed algorithm Table 5 shows the accuracy of each classification map for the single time.At both times, the proposed algorithm can provide better accuracy.

Change detection results
In order to obtain the accuracy of PCC technique and proposed method, a standard change map according to the ground truth was made [Figure 5(c)].Compared with the change detection results by PCC technique [Figure 5(a)], the proposed method [ Figure 5(b)] can provide better accuracy.We also estimated the accuracy of the two methods by false alarms and missed alarms in Table 6.Compared with the PCC technique, the proposed method can reduce the false alarms dramatically.But the proposed method increased the missed alarms to 28.However, it can provide better overall accuracies which was reduced from 160 to 41 respectively.

CONCLUSIONS
An object-oriented change detection approach considering spatiotemporal relationship in multi-temporal remote-sensing images has been proposed in this information.Land-cover class transition matrix and adjacency matrix are computed through the compound classification results at the two times.ICM technique is adopted to get the optimal compound classification results.Change detection is achieved by comparing classification results between the two times.The method proposed in this paper is a supervised technique.A training set is acquired by each of the multitemporal images.Thus, this method is not sensitive to atmospheric conditions and sensor calibration.
Experimental results on the QuickBird imagery show the effectiveness of the proposed method.The accuracies of the classification maps are increased.Thus the overall error of change detection is reduced.
Compared with the PCC method, the proposed method can provide better accuracy for change detection.
As future developments of this paper, we plan to do as follows: 1) further analysis of the two assumptions and 2) introduce the texture information and shape information of objects into the proposed method.ACKNOWLEDGMENT Supported by the Geographic National Condition Monitoring Engineering Research Center of Sichuan Province (GC201506) and the science and technology plan of sichuan bureau of surveying, mapping and geo information(J2014ZC12, J2014ZC16) REFERENCE is the probability of q T , given the classification results p C , and () p PC is a priori probability for the classification results p C .In order to simplify the problem, we make two assumptions.1)The feature vectors 1 X only depends on the classification results p C ,and the feature vectors 2 X only depends on the classification results q is the conditional probability of feature vectors 2 X ,given the classification results q T .

2 (
from the training sets of two images.In the Gaussian case, the mean vector  and the covariance matrix  are used to estimate the conditional probabilities.The parameters  and  of each land-cover class can abstract from corresponding training sets. ... ( / ) ... ( / ) classification results of k-th object at the two times.Change detection and change type recognition are implemented by comparing ()

Figure 4 .
Figure 3. Image segmentation results(black linesdenotes the boundary of the objects).road, farmland at time t1 and five classes: lake, building, road, farmland, bare land at time t2.Because of the complexity of the remote sensing imagery, subclasses with different spectral information exist in one land-cover class, i.e., the lake contains many subclasses with different sediment concentration and chlorophyll content.This problem will reduce the accuracy of the image classification.In order to solve this problem, training sets containing all subclasses should be selected.Figure4shows the classification results at the two times.The object belongs to the lake on the ground truth involved in red rectangle A is classified to the building in Figure4(a) using ML method, while it is classified to the lake in Figure4(c) using the proposed approach.The reason is that this object has similar spectral features as the building, but it's more likely to belong to the lake, considering the spatial-contextual information that the object adjacent to it belongs to the lake.Many objects belong to bare-land on the ground truth involved in red rectangle B of Figure4(b) are mistakenly classified as the road since the road and bare-land both have strong reflectivity.However, these objects are classified correctly in Figure 4(d) because the objects belong to the lake at time t1 are more likely to the lake at time t1 are more likely to transform into buildings than the road considering temporal dependence.(a) (b)

Figure 5 .
Figure 5. Change detection results(black areas denote the changed areas): (a) PCC technique with 440 changed objects and 185 unchanged objects;(b) proposed method with 301 changed objects and 324 unchanged objects; and (c) ground truth with 316 changed objects and 309 unchanged objects.
paper.The approach adopts the marker-based watershed algorithm to obtain objects.Change detection technique based on object can effectively suppress the salt and pepper noise in remote sensing imagery.The object-oriented compound classification which takes into account the spatiotemporal relationship between the two times is presented.The probabilities of land-cover class transitions are employed to describe the temporal dependence information and the Markov Random Field (MRF) model is utilized to represent the spatial-contextual of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-7/W3, 2015 36th International Symposium on Remote Sensing of Environment, 11-15 May 2015, Berlin, Germany This contribution has been peer-reviewed.doi:10.5194/isprsarchives-XL-7-W3-1241-20151247

MARKER-BASED WATERSHED IMAGE SEGMENTATION
1, Marker-based watershed image segmentation is described.Object-oriented PCC technique considering spatiotemporal relationship is proposed in Section 2. Experimental results and discussions are reported in Section 3. The conclusion is drawn in Section 4.
order to get a segmentation map shared by two considered images, we generate a new image by combining two multispectral images.The bands of new image are composed of bands from two images.
i Xkindicates the feature vector of k-th object at the time i t , 1, 2. i  p Ck  .() p Ckindicates the classification result of k-th object in p C which denotes the results of p-th random experiment at 1 t .Let the set { ,1 } N Sn  ,be composed of all the possible sets of classification results in the image at 2 t ,where { ( ), 1, 2,..., } qq T T k k N  ,with () q Tk .

Table 1 .
Land-cover class transition matrix

Table 2 .
Land-cover class adjacency matrix and the bare land class all have strong reflectivity with high gray value, thus, a classifier only considering spectral information can't distinguish them.

Table 5 .
Accuracies of the classification maps at each time by pcc and proposed algorithm 4.

Table 6 .
Change detection results by the two methods