Image compressed sensing reconstruction network based on self-attention mechanism

Abstract

For image compression sensing reconstruction, most algorithms use the method of reconstructing image blocks one by one and stacking many convolutional layers, which usually have defects of obvious block effects, high computational complexity, and long reconstruction time. An image compressed sensing reconstruction network based on self-attention mechanism (SAMNet) was proposed. For the compressed sampling, self-attention convolution was designed, which was conducive to capturing richer features, so that the compressed sensing measurement value retained more image structure information. For the reconstruction, a self-attention mechanism was introduced in the convolutional neural network. A reconstruction network including residual blocks, bottleneck transformer (BoTNet), and dense blocks was proposed, which strengthened the transfer of image features and reduced the amount of parameters dramatically. Under the Set5 dataset, when the measurement rates are 0.01, 0.04, 0.10, and 0.25, the average peak signal-to-noise ratio (PSNR) of SAMNet is improved by 1.27, 1.23, 0.50, and 0.15 dB, respectively, compared to the CSNet+. The running time of reconstructing a 256×256 image is reduced by 0.147 3, 0.178 9, 0.231 0, and 0.252 4 s compared to ReconNet. Experimental results showed that SAMNet improved the quality of reconstructed images and reduced the reconstruction time.

Keywords

convolutional neural network / compressed sensing / self-attention mechanism / dense block / image reconstruction

References

[1] SHANNON C E. Communication in the presence of noise. Proceedings of the IRE, 1949, 37(1): 10-21.

[2] DONOHO D L. Compressed sensing. IEEE Transactions on Information Theory, 2006, 52(4): 1289-1306.

[3] YAO K. Research on OFDM channel estimation based on compressed sensing. Beijing: Beijing University of Posts and Telecommunications, 2021.

[4] WANG Q. Information processing method for wireless communication based on compressed sensing. China New Communications, 2021, 23(13): 9-10.

[5] NARAYANAN S, SAHOO S K, MAKUR A. Greedy pursuits assisted basis pursuit for reconstruction of joint-sparse signals. Signal Processing, 2018, 142: 485-491.

[6] LEE J, CHOI J W, SHIM B. Sparse signal recovery via tree search matching pursuit. Journal of Communications and Networks, 2016, 18(5): 699-712.

[7] JI S H, XUE Y, CARIN L. Bayesian compressive sensing. IEEE Transactions on Signal Processing, 2008, 56(6): 2346-2356.

[8] WANG B, MA S X. Improvement of Gaussian random measurement matrices in compressed sensing. Advanced Materials Research, 2011, 301/302/303: 245-250.

[9] MOUSAVI A, DASARATHY G, BARANIUK R G. DeepCodec: adaptive sensing and recovery via deep convolutional neural networks. 2017: arXiv: 1707.03386.

[10] YAO H T, DAI F, ZHANG S L, et al. DR²-Net: deep residual reconstruction network for image compressive sensing. Neurocomputing, 2019, 359: 483-493.

[11] LIAN Q S, FU L P, CHEN S Z, et al. A compressed sensing algorithm based on multi-scale residual reconstruction network. Acta Automatica Sinica, 2019, 45(11): 2082-2091.

[12] XIE X M, WANG Y X, SHI G M, et al. Adaptive measurement network for CS image reconstruction//Computer Vision. Singapore: Springer Singapore, 2017: 407-417.

[13] ZHANG J, GHANEM B. ISTA-net: interpretable optimization-inspired deep network for image compressive sensing//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, UT, USA. New York: IEEE, 2018: 1828-1837.

[14] MOUSAVI A, PATEL A B, BARANIUK R G. A deep learning approach to structured signal recovery//2015 53rd Annual Allerton Conference on Communication, Control, and Computing, September 29-October 2, 2015, Monticello, IL, USA. New York: IEEE, 2016: 1336-1343.

[15] KULKARNI K, LOHIT S, TURAGA P, et al. ReconNet: non-iterative reconstruction of images from compressively sensed measurements//2016 IEEE Conference on Computer Vision and Pattern Recognition, June 27-30, 2016, Las Vegas, NV, USA. New York: IEEE, 2016: 449-458.

[16] LU X T, DONG W S, WANG P Y, et al. ConvCSNet: a convolutional compressive sensing framework based on deep learning. 2018: arXiv: 1801.10342.

[17] DU J, XIE X M, WANG C Y, et al. Fully convolutional measurement network for compressive sensing image reconstruction. Neurocomputing, 2019, 328: 105-112.

[18] SHI W Z, JIANG F, LIU S H, et al. Image compressed sensing using convolutional neural network. IEEE Transactions on Image Processing, 2020, 29: 375-388.

[19] SUN Y B, CHEN J W, LIU Q S, et al. Dual-path attention network for compressed sensing image reconstruction. IEEE Transactions on Image Processing, 2020, 29: 9482-9495.

[20] HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks//2017 IEEE Conference on Computer Vision and Pattern Recognition, July 21-26, 2017, Honolulu, HI, USA. New York: IEEE, 2017: 2261-2269.

[21] ZHANG Z Q, GAO D H, XIE X M, et al. Dual-channel reconstruction network for image compressive sensing. Sensors, 2019, 19(11): 2549.

[22] LI X, WANG W H, HU X L, et al. Selective kernel networks//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 15-20, 2019, Long Beach, CA, USA. New York: IEEE, 2020: 510-519.

[23] HU J, SHEN L, SUN G. Squeeze-and-excitation networks//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, UT, USA. New York: IEEE, 2018: 7132-7141.

[24] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module. Computer Vision-ECCV 2018. Cham: Springer, 2018: 3-19.

[25] BELLO I, ZOPH B, LE Q, et al. Attention augmented convolutional networks//2019 IEEE/CVF International Conference on Computer Vision, October 27-November 2, 2019. Seoul, Korea. New York: IEEE, 2019: 3285-3294.

[26] SRINIVAS A, LIN T Y, PARMAR N, et al. Bottleneck transformers for visual recognition//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 20-25, 2021. Nashville, TN, USA. New York: IEEE, 2021: 16514-16524.

[27] PARK J, WOO S, LEE J Y, et al. BAM: bottleneck attention module. 2018: arXiv: 1807.06514.

[28] SHAW P, USZKOREIT J, VASWANI A. Self-attention with relative position representations. 2018: arXiv: 1803.02155.

[29] XIE X M, WANG C Y, DU J, et al. Full image recover for block-based compressive sensing//2018 IEEE International Conference on Multimedia and Expo, July 23-27, 2018, San Diego, CA, USA. New York: IEEE, 2018: 1-6.

[30] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition//2016 IEEE Conference on Computer Vision and Pattern Recognition, June 27-30, 2016, Las Vegas, NV, USA. New York: IEEE, 2016: 770-778.

[31] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, July 21-26, 2017, Honolulu, HI, USA. New York: IEEE, 2017: 1132-1140.

[32] SHI W Z, JIANG F, ZHANG S P, et al. Deep networks for compressed image sensing//2017 IEEE International Conference on Multimedia and Expo, July 10-14, 2017, Hong Kong, China. New York: IEEE, 2017: 877-882.

[33] ZHOU S W, HE Y, LIU Y H, et al. Multi-channel deep networks for block-based image compressive sensing. IEEE Transactions on Multimedia, 2021, 23: 2627-2640.