Multiscale Fusion Transformer Network for Hyperspectral Image Classification

Abstract

Convolutional neural network (CNN) has excellent ability to model locally contextual information. However, CNNs face challenges for descripting long-range semantic features, which will lead to relatively low classification accuracy of hyperspectral images. To address this problem, this article proposes an algorithm based on multiscale fusion and transformer network for hyperspectral image classification. Firstly, the low-level spatial-spectral features are extracted by multi-scale residual structure. Secondly, an attention module is introduced to focus on the more important spatial-spectral information. Finally, high-level semantic features are represented and learned by a token learner and an improved transformer encoder. The proposed algorithm is compared with six classical hyperspectral classification algorithms on real hyperspectral images. The experimental results show that the proposed algorithm effectively improves the land cover classification accuracy of hyperspectral images.

Keywords

hyperspectral image / land cover classification / multi-scale / transformer