Hyperspectral Image Super-Resolution Network Based on Reinforcing Inter-Spectral Incremental Information

Abstract

Hyperspectral images typically have high spectral resolution but low spatial resolution, which impacts the reliability and accuracy of subsequent applications, for example, remote sensing classification and mineral identification. But in traditional methods via deep convolution neural networks, indiscriminately extracting and fusing spectral and spatial features makes it challenging to utilize the differentiated information across adjacent spectral channels. Thus, we proposed a multi-branch interleaved iterative upsampling hyperspectral image super-resolution reconstruction network (MIIUSR) to address the above problems. We reinforce spatial feature extraction by integrating detailed features from different receptive fields across adjacent channels. Furthermore, we propose an interleaved iterative upsampling process during the reconstruction stage, which progressively fuses incremental information among adjacent frequency bands. Additionally, we add two parallel three dimensional (3D) feature extraction branches to the backbone network to extract spectral and spatial features of varying granularity. We further enhance the backbone network’s construction results by leveraging the difference between two dimensional (2D) channel-grouping spatial features and 3D multi-granularity features. The results obtained by applying the proposed network model to the CAVE test set show that, at a scaling factor of ×4, the peak signal to noise ratio, spectral angle mapping, and structural similarity are 37.310 dB, 3.525 and 0.9438, respectively. Besides, extensive experiments conducted on the Harvard and Foster datasets demonstrate the superior potential of the proposed model in hyperspectral super-resolution reconstruction.

Keywords

image processing / hyperspectral image / super-solution / incremental information