APPLICATION OF ECOC SVMS IN REMOTE SENSING IMAGE CLASSIFICATION

: Image processing has been one of the efficient technologies for GIS data requisition. Support Vector Machines (SVMs) have peculiar advantages in handling problems with small sample sizes, nonlinearity, and high dimensionality. However, SVMs can only solve two-class problems while multi-class decision is impossible. Error correcting output coding (ECOC) SVMs enhance the ability of fault tolerance when solving multi-class classification problems, which makes ECOC SVMs suitable for remote sensing image classification. In this paper, the generalization ability of ECOC SVMs is discussed. ECOC SVMs with optimum coding matrices are selected by experiment, and applied to remote sensing image classification. Experimental results show that, compared with Conventional multi-class classification methods, less SVM sub-classifiers are needed for ECOC SVMs in remote sensing image classification, and the classification accuracy is also improved.


INTRODUCTION
Image processing has been one of the efficient technologies for GIS data requisition.Support Vector Machine (SVM) method has been widely and extensively studied recently, andhas been successfully applied in remote sensing image classification (Custavo et al., 2004) and (ZHU et al., 2002).However, it was originally proposed for two-class classification problems, which is not suitable for remote sensing image classification with multiple geographical object classes.Dietterich proposed a framework in which the error correcting output coding (ECOC) was used to convert multi-class problem into several two-class problems (Dietterich et al., 1995).The combination of it with SVMs, ECOC SVMs, enhances the fault tolerance of classification models when solving the multi-class classification problems, which makes ECOC SVMs effective in remote sensing image classification.In this study, the generalization ability of ECOC SVMs is analyzed, and new ideas on how to construct ECOC SVMs are suggested and applied in remote sensing image classification.

Principles of error correcting output coding
ECOC is a kind of method which converts a multi-class classification problem into two-class classification problems.In  (1)The rows of a coding matrix should not be correlated, while the columns of a coding matrix should be neither correlated, nor complementary; (2)A column should be neither all "0", nor all "1"; In order to have available ECOCs satisfy the above conditions, exhaustive enumeration coding method, BCH coding method, column selection method, random coding method (Dietterich et al., 1995), and searching coding method are commonly used (Jiang et al., 2005).
The above-mentioned methods have their own characteristics: the ECOCs generated by enumeration coding method have strong error correcting capability, however the code length increases as the class number increases and the learning process is complicated.The numbers of code words in the coding matrix obtained by BCH coding method are always power of 2. Therefore if the class number m is not the power of 2, some heuristic methods are needed for shortening the code length and reducing the code words, which makes this method inconvenience to use.Column selection method and random coding method are non-deterministic methods.Searching coding method is suitable for any number of classes, moreover it can automatically generates alternative codes according to different indices such as class number, minimum Hamming distance, etc. However it cannot overcome the problem of identical columns.At present, there is no universal deterministic coding method which can obtain available ECOCs for any class number K.

Decoding:
Decoding is the decision fusion stage of ECOC SVMs.The algorithm framework of ECOC is shown in Figure 1 The two-class classification results are fused in order to output the multi-class classification result.Classical decoding methods include: Hamming distance decoding and the inverse Hamming distance decoding which is improved version of the former, as well as Euclidean distance decoding, center distance decoding, loss function based decoding, etc. (Escalera et al., 2010).
Among the above methods, the principles of Hamming distance decoding can be described as follows: for a K-class classification problem, combine the binary results of L basic classifiers in order, obtaining the binary classification code C for each classification object.The new sample's class is represented by the code word which has the minimum Hamming distance with C among the K standard codes.

Experiments on generalization ability of ECOC SVMs
It is generally believed that, for coding performance evaluation, the performance of ECOCs is related to the mathematical characteristics, such as code length, minimum inter-code Hamming distance, and the distribution order of code words (Xia et al., 2003).To verify the relationship between the generalization ability of ECOC SVMs and the mathematical characteristics of ECOC coding as well as the generalization ability of each sub-SVM classifier, experiments are carried out in this study for assessment.

2.3.1
Tests on UCI data base: The Page Blocks, Landsat Satellite, Zoo, Segment and Optdigits data sets in UCI data base are chosen for testing.The details of each data set are tabulated in Table 2

A experimental result for the generalization ability of ECOC SVMs:
Analyzing the generalization ability results of ECOC SVMs for UCI database and remote sensing image data, it can be concluded that after sorting the generalization ability of SVM sub-classifiers, the prediction accuracy of the forward sequence of ECOCs is higher than that of the backward sequence, whereas the prediction accuracy of the original sequence (which can be viewed as random ECOCs) generally lies between the other two.The more overlapped code words between the forward, backward sequences, the closer the prediction accuracies of them.When the code length increases to a certain extent, the prediction accuracy of forward sequence decreases and finally becomes stable.The random sequence increases with fluctuations and finally becomes smooth, whereas the backward sequence keeps an increasing trend.
The above analysis shows that when the code length increases, if the generalization ability of the ECOC SVMs corresponding to the newly added columns is good, the coding performance is improved, such as forward sequences; if the generalization ability is bad, the coding performance worsens, such as backward sequences; for random sequences (original sequences), the generalization ability of each newly added SVM is at random, therefore the final prediction accuracy fluctuates.
It can be seen that the performance of ECOC SVMs is neither improved with the increasing of the minimum Hamming distance, nor does it deteriorates with the increasing of code length.Instead, it has a complicated relationship with code length and inter-code minimum Hamming distance.ECOC SVMs' performance depends on the performance of the corresponding sub-SVMs.The mathematical characteristics expressed in coding are minor.When the performance of all sub-SVMs is not evaluable or considered to be identical, the code length, inter-code minimum Hamming distance, and the sorting method of code words can be used as evaluation indices of ECOCs.
Therefore, for sub-SVMs with good generalization ability, ECOC coding can be short; however if the performance of sub-SVMs cannot be determined, the code length should be longer and the inter-code Hamming distance should also be longer, but the code length should be appropriate.The longer is by no means the better.

Data for experiments and tools
The remote sensing image data chosen in this study is the Landsat-TM image of someplace.The bands selected are TM1~TM7 with image size being 640×400 pixels.Figure 4 shows the combination of bands 7, 4, 2. The software tools used in the experiments are ENVI 4.8 and LibSVM toolbox.Feature extraction and selection are the most significant factors for classification accuracy and reliability, as well as the classifier design and performance.In the experiments, the PCA transform information (Francesco et al., 2004), NDVI (McFeeters, 1996), and texture information (Gong et al., 1992), as well as the original spectral information are extracted and combined for classification.

Generating ECOC coding
In the experiments, according to visual interpretation and survey, the remote sensing data is classified into 5 classes, namely field, sand, bare land, forest, and shadow.After obtaining the class number K, the exhaustive enumeration coding method can be used to generate available ECOCs satisfying valid conditions, whose code length CL satisfies .Then the ECOC coding selection operation is performed subsequently.
Using the idea in this study to test the data, we obtain the classification accuracies corresponding to the forward sequences ECOC codes, shown in Figure 5 Finally, the ECOC codes with code length being 7 are selected to construct ECOC SVMs.
Figure 5. the prediction accuracies corresponding to the forward sequences ECOC codes Figure 6. the classification results of the selected remote sensing image

Result analysis
The experimental result of ECOC SVMs for the TM remote sensing image is shown in Figure 6 Evaluating the classification accuracy of ECOC SVMs for the remote sensing image, we obtain the error matrix which is tabulated in Table 2 Using the same training data and test data for traditional 1-V-R and 1-V-1 SVMs for classification, and evaluating the classification accuracy, we obtain the error matrix tabulated in Table 3 and 4 The overall classification accuracies and the kappa coefficients of the three multi-class classification methods are shown in Table 5.

Figure
Figure 1.ECOC algorithm framework

Figure 4 .
Figure 4. the remote sensing image for test Feature extraction and selection are performed for the remote sensing image, and the image is classified by using ECOC SMVs combined with original spectral information.The main steps include relevant data preprocessing, extracting attributed feature information, generating ECOC coding matrix, selecting training samples, data normalization, SVM kernel parameter settings, ECOC coding selection, classification by ECOC SVMs and decoding, as well as the accuracy assessment of the final classification results.

Table 1 .
An error correcting coding matrix for class number k=4 and code length n=7 2.

2 Coding and decoding of ECOC 2.2.1 Coding:
An available ECOC should satisfy the following conditions:

Table 2
. The prediction accuracies of the SVM sub-classifiers are used as the index for generalization ability to sort the SVM sub-classifiers.The sub-classifiers are sorted in forward, original (random), and backward sequences.Take the first m=CL SVM sub-classifiers of all the sorted SVM subclassifiers according to code length.The predicted results after decoding are compared with the real results, and then the prediction accuracies using different sorting methods are calculated and shown in Figure2.
. The explanation on UCI data base used in the tests The testing process is described here.For given samples in n classes, construct an n by k ECOC coding matrix (k is the minimum code length) using exhaustive enumeration coding method, then separately train SVM sub-classifiers according to each column of the ECOC coding matrix.ECOC SVMs composed by codes from column 1 to k in each data set are tested in the experiments.In the experiments, all SVM sub-classifiers unitedly use RBF kernel function with identical parameters.

Table 5 .
the overall classification accuracies and the kappa coefficients among 3 methods It can be seen through comparison that for same experimental remote sensing data, the overall classification accuracy and kappa coefficient of ECOC SVMs are both higher than the other two4.CONCLUSIONSThe influencing factors of ECOC SVMs performance are discussed in this study.It is pointed out that for a given training data set, reasonable ECOCs should be constructed based on SVM sub-classifiers with good generalization ability to improve the generalization performance of ECOC SVMs.ECOC SVMs improve the fault tolerance of classification models when solving the multi-class classification problem for remote sensing images.It has the advantage of fewer subclassifiers and high classification accuracy.Experiments show that ECOC SVMs outperforms 1-V-R SVMs and 1-V-1 SVMs.Priority Academic Program Development of Jiangsu Higher Education Institutions.