HYPERSPECTRAL IMAGE CLASSIFICATION USING MULTI-LAYER PERCEPTRON MIXER (MLP-MIXER)

The classifying of hyperspectral images (HSI) is a difficult task given the high dimensionality of the space, the huge number of spectral bands, and the small number of labeled data. As such, we offer a unique hyperspectral image classification methodology to address these issues based on sophisticated Multi-Layer Perceptron (MLP) algorithms. In this paper, we propose using MLP-Mixer to classify HSI data in three data benchmarks of Pavia, Salinas, and Indian Pines. Based on the results, the proposed MLP-Mixer achieved a high level of classification accuracy and produced noise-free and homogenous classification maps in all study areas. For the classification of HSI data in Salinas, Indian Pines, and Pavia, the proposed MLP-Mixer achieved an average accuracy of 99.82%, 99.81%, and 99.23%, respectively.


INTRODUCTION
Advances in image acquisition techniques have resulted in higher spatial-spectral image resolutions, more efficient image processing models, and the continuous creation of large amounts of high-quality data.As a result, low-cost, highquality data from sensors, combined with the availability of advanced computing resources, including graphics processing units (GPUs), has resulted in superior computer models that allow researchers to better understand the morphological changes, ground surface, and human processes with greater precision and detail (Fu et al., 2017).Image classification and semantic segmentation are both well-studied sub-domains of computer vision that have been used in RS image analysis (A. Jamali et al., 2022;Jamali et al., 2021;Jamali and Mahdianpari, 2022), image segmentation and face recognition, among other applications.Image classification groups and categorizes all objects in an image into a single class, whereas semantic segmentation assigns each pixel in an image to a set of predetermined labels/classes, where the same labels share certain properties (Kemker et al., 2018).
Land cover mapping and change detection (Akar and Tunc Gormus, 2021;Q. Li et al., 2021), soil carbon prediction (Meng et al., 2022), vegetation classification (Gao et al., 2022), forest biomass understanding, and tree species identification/mapping (Vangi et al., 2021), urban monitoring and understanding (Wambugu et al., 2021), and other applications benefit from the rich spatial-spectral data gained from Hyperspectral (HIS) data.However, labeling HSI data is expensive and labor-intensive due to the various sensors utilized to gather the data and the domain expertise required.This explains why there are so few labeled HSI benchmark datasets, causing HSI classification to lag behind other vision-based and image-processing areas due to the scarcity of annotated labels and the intricate nature of HSI data (X.Cao et al., 2018).As such, the use of deep learning models such as deep and very deep Convolutional Neural Networks (CNNs) is a challenge in HSI data classification.
On the other hand, conventional classifiers, such as random forest and support vector machines, have had remarkable achievements in HSI classification (Belgiu and L. Drăguţ, 2016), but they only use spectral data, limiting their maximum efficiency.Furthermore, the high dimensionality of spatialspectral data, combined with a small number of training examples, makes it difficult to increase classification performance (Z.Zhong et al., 2018).As such, to improve the classification accuracy of HIS data, in this paper, we will investigate the use of Multi-Layer Perceptron (MLP)-Mixer as one of the current advanced MLP algorithms for the classification of HSI data.It should be noted that the MLP-Mixer considers both spectral and spatial information in the classification process.

MLP-Mixer
The architecture of the MLP-Mixer (Tolstikhin et al., 2021)  channel-mixing and token-mixing.MLPs that blend channels enable communication between them (Tolstikhin et al., 2021).They function independently on each token and take individual rows of the table as input.Token-mixing MLPs enable communication between distinct spatial locations (tokens); they work independently on each channel and accept specific table columns as inputs.Interleaving these two types of layers allows interaction of both input dimensions (Tolstikhin et al., 2021).

Study area and remote sensing data
We conduct experiments on three different HSI data benchmarks to assess the efficacy and performance of the proposed classifier: • The Airborne Visible/ Infrared Imaging Spectrometer (AVIRIS) sensor gathered the Indian Pines HSI, which northwestern Indiana.It has 145 by145 pixels with a spatial resolution of 20 m per pixel and 220 spectral bands.The ground truth data is divided into 16 classes.

•
The AVIRIS sensor captured the Salinas image over Salinas, California, which consists of 512 by 217 pixels with a spatial resolution of 3.7 m per pixel and 224 spectral bands.The ground truth data is divided into 16 classes.

•
The final HSI is Pavia University data taken using the Reflective Optics System Imaging Spectrometer (ROSIS-03) sensor, which has 610 by 610 pixels and 115 spectral bands, with a spatial resolution of 1.3 m.There are nine classes in the ground truth data.

Accuracy assessment
The classification results of the MLP-Mixer are evaluated in terms of overall accuracy, kappa index, and average accuracy (see Equations 1-3).

RESULTS AND DISCUSSION
To better understand the performance of the proposed MLPmixer for HSI classification, its predicted classification maps are presented in Figure .2. Moreover, statistical results of the MLP-Mixer in terms of average accuracy, kappa index, and overall accuracy are shown in Table 1.As seen in Figure 2 and Table 1, the proposed classifier illustrated great capability in the classification of HSI data in all three data benchmarks of Indian Pines, Salinas, and Pavia.For the Indian Pines dataset, the MLP-Mixer achieved kappa index, overall accuracy, and average accuracy of 99.78%, 99.8%, and 99.81%, respectively.Moreover, average accuracy, kappa index, and overall accuracy of 99.23%, 99.49%, and 99.62% were obtained by the MLP-Mixer, respectively, for the Pavia dataset.In addition, the MLP-Mixer obtained kappa index, overall accuracy, and average accuracy of 99.7%, 99.73%, and 99.82%, respectively, for the Salinas dataset.

Figure. 1 .
Figure. 1.The architecture of the MLP-Mixer.Per-patch linear embeddings, Mixer layers, and a classifier head comprise MLP-Mixer.One token-mixing MLP and one channel-mixing MLP, each with two fully connected layers and a GELU nonlinearity, are present in the mixer layers.Other features are skip-connections, dropout, and channel layer norm(Tolstikhin et al., 2021).

Figure. 2 .
Figure. 2. Ground truth data and results of the MLP-Mixer; a) Indian Pines reference data, b) Classified Indian Pines HIS data c) Pavia reference data d) Classified Pavia HIS data e) Salinas reference data f) Classified Salinas HIS data.

Table 1 .
Classification results of HIS data by the MLP-Mixer (KI= Kappa Index, OA= Overall Accuracy, AA= Average Accuracy)This paper proposes a novel approach for spatial-spectral classification of HSI data based on the current state-of-the-art MLP algorithms.The proposed classifier of MLP-Mixer demonstrated excellent capability in the classification of HSI data in three data benchmarks of Pavia, Indian Pines, and Salinas.The presented MLP-Mixer achieved an average accuracy of 99.82%, 99.81%, and 99.23% for the classification of HSI data in Salinas, Indian Pines, and Pavia, respectively.Our method can be compared to other MLP classifiers in future works.