A Bridge Transformer Network With Deep Graph Convolution for Hyperspectral Image Classification

Abstract

Transformers have been widely applied to hyperspectral image classification, leveraging their self-attention mechanism for powerful global modelling. However, two key challenges remain as follows: excessive memory and computational costs from calculating correlations between all tokens (especially as image size or spectral bands increase) and limited ability to model local boundary information due to lacking explicit enhancement mechanisms. This paper proposes a novel method, bridge transformer network fused with deep graph convolution (BTDGC), to address these issues. The framework includes three components as follows: a double random masking mechanism (DRMM) that forces the model to infer masked features from context during training, a bridge transformer (BT) module with bridge tokens for cross-region feature interaction and a Deep Graph Convolutional Pooling (DGCP) module that preserves spatial topology while aggregating hierarchical information. Experiments on standard hyperspectral datasets show BTDGC outperforms mainstream methods in classification accuracy and robustness, effectively balancing global modelling and local boundary representation. The code is available at https://github.com/jenny3489/BTDGC.

Keywords

convolution / graph convolutional network / masking mechanism / transforms

Funding

The authors have nothing to report.

Conflicts of Interest

The authors declare no conflicts of interest.

Data Availability Statement

The data that support the findings of this study are openly available in https://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Se nsing_Scenes#Pavia_Centre_and_University, https://www.ehu.eus/cc wintco/index.php/Hyperspectral_Remote_Sensing_Scenes#Salinas and https://rsidea.whu.edu.cn/resource_WHUHi_sharing.htm.

References

[1] M. Dalponte, H. O. Ørka, T. Gobakken, D. Gianelle, and E. Næsset, “Tree Species Classification in Boreal Forests With Hyperspectral Data,” IEEE Transactions on Geoscience and Remote Sensing 51, no. 5 (2012): 2632-2645, https://doi.org/10.1109/TGRS.2012.2216272.

[2] Y. Zhang, Y. Xu, W. Xiong, et al., Inversion Study of Heavy Metals in Soils of Potentially Polluted Sites Based on UAV Hyperspectral Data and Machine Learning Algorithms,” in 2021 11th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS) (IEEE, 2021), 1-5.

[3] C. Weber, R. Aguejdad, X. Briottet, et al., Hyperspectral Imagery for Environmental Urban Planning,” in IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium (IEEE, 2018), 1628-1631.

[4] L. Luft, C. Neumann, M. Freude, N. Blaum, and F. Jeltsch, “Hyperspectral Modeling of Ecological Indicators - A New Approach for Monitoring Former Military Training Areas,” Ecological Indicators 46 (2014): 264-285, https://doi.org/10.1016/j.ecolind.2014.06.025.

[5] F. Melgani and L. Bruzzone, “Classification of Hyperspectral Remote Sensing Images With Support Vector Machines,” IEEE Transactions on Geoscience and Remote Sensing 42, no. 8 (2004): 1778-1790, https://doi.org/10.1109/tgrs.2004.831865.

[6] Y. Guo, S. Han, Y. Li, C. Zhang, and Y. Bai, “K-Nearest Neighbor Combined With Guided Filter for Hyperspectral Image Classification,” Procedia Computer Science 129 (2018): 159-165, https://doi.org/10.1016/j.procs.2018.03.066.

[7] M. S. Aydemir and G. Bilgin, “2D 2 PCA-Based Hyperspectral Image Classification With Utilization of Spatial Information,” in 2013 5th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS) (IEEE, 2013), 1-4.

[8] S. Yu, S. Jia, and C. Xu, “Convolutional Neural Networks for Hyperspectral Image Classification,” Neurocomputing 219 (2017): 88-98, https://doi.org/10.1016/j.neucom.2016.09.010.

[9] Y. Chen, H. Jiang, C. Li, X. Jia, and P. Ghamisi, “Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks,” IEEE Transactions on Geoscience and Remote Sensing 54, no. 10 (2016): 6232-6251, https://doi.org/10.1109/tgrs.2016.2584107.

[10] S. Yu, S. Jia, and C. Xu, “Convolutional Neural Networks for Hyperspectral Image Classification,” Neurocomputing 219 (2017): 88-98, https://doi.org/10.1016/j.neucom.2016.09.010.

[11] M. Kanthi, T. H. Sarma, and C. S. Bindu, “A 3D-Deep CNN Based Feature Extraction and Hyperspectral Image Classification,” in 2020 IEEE India Geoscience and Remote Sensing Symposium (InGARSS) (IEEE, 2020), 229-232.

[12] K. Shi, Q. Liu, Z. Zheng, and L. Xiao, “Efficient Implementation for Composite CNN-Based HSI Classification Algorithm With Huawei Ascend Framework,” in 2023 13th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS) (IEEE, 2023), 1-5.

[13] C. Yu, M. Zhao, M. Song, et al., Hyperspectral Image Classification Method Based on CNN Architecture Embedding With Hashing Semantic Feature,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12, no. 6 (2019): 1866-1881, https://doi.org/10.1109/jstars.2019.2911987.

[14] W. Yao, C. Lian, and L. Bruzzone, “ClusterCNN: Clustering-Based Feature Learning for Hyperspectral Image Classification,” IEEE Geoscience and Remote Sensing Letters 18, no. 11 (2020): 1991-1995, https://doi.org/10.1109/lgrs.2020.3010837.

[15] H. Zhang, Y. Chen, X. He, and X. Shen, “Boosting CNN for Hyperspectral Image Classification,” in 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS (IEEE, 2021), 3673-3676.

[16] W. Kong, L. Gu, Z. Wang, and L. Chen, “Multi-Branch Graph Neural Network Model for Hyperspectral Image Classification,” in 2023 China Automation Congress (CAC) (IEEE, 2023), 440-445.

[17] Y. Ding, Y. Chong, S. Pan, and C. Zheng, “Diversity-Connected Graph Convolutional Network for Hyperspectral Image Classification,” IEEE Transactions on Geoscience and Remote Sensing 61 (2023): 1-18, https://doi.org/10.1109/tgrs.2023.3298848.

[18] Y. Xiao, R. Ma, S. Chang, et al., Dynamic Evolution Graph Attention Network for Semi-Supervised Hyperspectral Image Classification,” IEEE Geoscience and Remote Sensing Letters 21 (2024): 1-5, https://doi.org/10.1109/lgrs.2024.3401133.

[19] Z. Zhao, H. Wang, and X. Yu, “Spectral-Spatial Graph Attention Network for Semisupervised Hyperspectral Image Classification,” IEEE Geoscience and Remote Sensing Letters 19 (2021): 1-5, https://doi.org/10.1109/lgrs.2021.3059509.

[20] L. Ma, Q. Wang, J. Zhang, and Y. Wang, “Parallel Graph Attention Network Model Based on Pixel and Superpixel Feature Fusion for Hyperspectral Image Classification,” in IGARSS 2023-2023 IEEE International Geoscience and Remote Sensing Symposium (IEEE, 2023), 7226-7229.

[21] Q. Song, F. Mo, K. Ding, et al., MCFNet: Multiscale Cross-Domain Fusion Network for HSI and Lidar Data Joint Classification,” IEEE Transactions on Geoscience and Remote Sensing 63 (2025): 1-12, https://doi.org/10.1109/tgrs.2025.3567297.

[22] H. Yang, H. Yu, D. Hong, Z. Xu, Y. Wang, and M. Song, “Hyperspectral Image Classification Based on Multi-Level Spectral-Spatial Transformer Network,” in 2022 12th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS) (IEEE, 2022), 1-4.

[23] Q. Song, B. Sun, and S. Li, “Multimodal Sparse Transformer Network for Audio-Visual Speech Recognition,” IEEE Transactions on Neural Networks and Learning Systems 34, no. 12 (2023): 10028-10038, https://doi.org/10.1109/tnnls.2022.3163771.

[24] M. Ahmad, U. Ghous, M. Usama, and M. Mazzara, “WaveFormer: Spectral-Spatial Wavelet Transformer for Hyperspectral Image Classification,” IEEE Geoscience and Remote Sensing Letters 21 (2024): 1-5, https://doi.org/10.1109/lgrs.2024.3353909.

[25] H. Zhou, X. Zhang, C. Zhang, and Q. Ma, “Vision Transformer With Contrastive Learning for Hyperspectral Image Classification,” IEEE Geoscience and Remote Sensing Letters 20 (2023): 1-5, https://doi.org/10.1109/lgrs.2023.3255867.

[26] Z. Wang, Y. Li, Z. Cheng, and Y. Zhang, “CNN and Transformer Hybrid Network for Hyperspectral Image Classification,” in IGARSS 2024-2024 IEEE International Geoscience and Remote Sensing Symposium (IEEE, 2024), 9091-9095.

[27] A. B. Hamida, A. Benoit, P. Lambert, and C. B. Amar, “3-D Deep Learning Approach for Remote Sensing Image Classification,” IEEE Transactions on Geoscience and Remote Sensing 56, no. 8 (2018): 4420-4434, https://doi.org/10.1109/tgrs.2018.2818945.

[28] M. He, B. Li, and H. Chen, “Multi-Scale 3D Deep Convolutional Neural Network for Hyperspectral Image Classification,” in 2017 IEEE International Conference on Image Processing (ICIP) (IEEE, 2017), 3904-3908.

[29] Y. Sun, W. Hu, F. Liu, M. Jiang, F. Huang, and D. Xu, “SpeFormer: An Efficient Hardware-Software Cooperative Solution for Sparse Spectral Transformer,” in 2022 IEEE 9th International Conference on Cyber Security and Cloud Computing (CSCloud)/2022 IEEE 8th International Conference on Edge Computing and Scalable Cloud (EdgeCom) (IEEE, 2022), 180-185.

[30] S. Liu, C. Fu, Y. Duan, X. Wang, and F. Luo, “Spatial-Spectral Enhancement and Fusion Network for Hyperspectral Image Classification With Few Labeled Samples,” IEEE Transactions on Geoscience and Remote Sensing 63 (2024): 1-14, https://doi.org/10.1109/tgrs.2024.3523578.

[31] W. Song, S. Li, L. Fang, and T. Lu, “Hyperspectral Image Classification With Deep Feature Fusion Network,” IEEE Transactions on Geoscience and Remote Sensing 56, no. 6 (2018): 3173-3184, https://doi.org/10.1109/tgrs.2018.2794326.

[32] N. Chen, Z. Huang, X. Yue, et al., SVAFormer: Integrating Random and Hierarchical Spectral View Attention for Hyperspectral Image Classification,” IEEE Transactions on Geoscience and Remote Sensing 62 (2024): 1-13, https://doi.org/10.1109/tgrs.2024.3509478.

[33] Z. Li, M. Qi, Y. Wang, X. Gong, C. Liu, and J. Wang, “An Open-Set Domain Adaptation Framework for Hyperspectral Image Classification With Pixel-Aware Weighting and Decoupled Alignment,” IEEE Geoscience and Remote Sensing Letters 22 (2025): 1-5, https://doi.org/10.1109/lgrs.2025.3565605.

[34] J. Dong, M. Liang, Z. He, and C. Zhou, “Hierarchical and Bidirectional Contrastive Learning for Hyperspectral Image Classification,” IEEE Transactions on Geoscience and Remote Sensing 63 (2025): 1-17, https://doi.org/10.1109/tgrs.2025.3582963.