Spatial-Spectral Joint Network for Cholangiocarcinoma Microscopic Hyperspectral Image Classification

Abstract

Accurate histopathology classification is a crucial factor in the diagnosis and treatment of Cholangiocarcinoma (CCA). Hyperspectral images (HSI) provide rich spectral information than ordinary RGB images, making them more useful for medical diagnosis. The Convolutional Neural Network (CNN) is commonly employed in hyperspectral image classification due to its remarkable capacity for feature extraction and image classification. However, many existing CNN-based HSI classification methods tend to ignore the importance of image spatial context information and the interdependence between spectral channels, leading to unsatisfied classification performance. Thus, to address these issues, this paper proposes a Spatial-Spectral Joint Network (SSJN) model for hyperspectral image classification that utilizes spatial self-attention and spectral feature extraction. The SSJN model is derived from the ResNet18 network and implemented with the non-local and Coordinate Attention (CA) modules, which extract long-range dependencies on image space and enhance spatial features through the Branch Attention (BA) module to emphasize the region of interest. Furthermore, the SSJN model employs Conv-LSTM modules to extract long-range dependencies in the image spectral domain. This addresses the gradient disappearance/explosion phenomena and enhances the model classification accuracy. The experimental results show that the proposed SSJN model is more efficient in leveraging the spatial and spectral information of hyperspectral images on multidimensional microspectral datasets of CCA, leading to higher classification accuracy, and may have useful references for medical diagnosis of CCA.

Keywords

self-attention / microscopic hyperspectral images / image classification / Conv-LSTM