Magnetic Resonance Image Super-Resolution Based on GAN and Multi-Scale Residual Dense Attention Network

Abstract

The application of image super-resolution(SR)has brought significant assistance in the medical field,aiding doctors to make more precise diagnoses. However,solely relying on a convolutional neural network(CNN)for image SR may lead to issues such as blurry details and excessive smoothness. To address the limitations,we proposed an algorithm based on the generative adversarial network(GAN)framework. In the generator network,three different sizes of convolutions connected by a residual dense structure were used to extract detailed features,and an attention mechanism combined with dual channel and spatial information was applied to concentrate the computing power on crucial areas. In the discriminator network,using InstanceNorm to normalize tensors sped up the training process while retaining feature information. The experimental results demonstrate that our algorithm achieves higher peak signal-to-noise ratio(PSNR)and structural similarity index measure(SSIM)compared to other methods,resulting in an improved visual quality.

Keywords

magnetic resonance(MR) / image super-resolution(SR) / attention mechanism / generative adversarial network(GAN) / multi-scale convolution

References

[1] TANG Y Q, PAN H, ZHU Y P. A survey of super resolution image reconstruction[J]. Journal of Electronics, 2020, 48 (7):1407-1420. (in Chinese)

[2] DONG C, LOY C C, HE K M, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2):295-307.

[3] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE,2017:136-144.

[4] GOODFELLOW I J, POUGHT-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]// Advances in Neural Information Processing Systems. Cambridge: MIT Press,2014:2672-2680.

[5] LEDIG C, THEIS L, HUSZÁR F, et al.Photo-realistic single image super-resolution using a generative adversarial network[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE,2017:105-114.

[6] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE,2016:770-778.

[7] XI Z H, HOU C Y, YUAN K P. Medical image super resolution reconstruction based on residual network[J]. Computer Engineering and Applications, 2019, 55(19):191-197. (in Chinese)

[8] CHEN Y, SHI F, CHRISTODOULOU A G, et al. Efficient and accurate MRI super-resolution using a generative adversarial network and 3D multi-level densely connected network[C]// Proceedings of the Medical Image Computing and Computer Assisted Intervention. Cham:Springer,2018:91-99.

[9] ANKITHA B, SRIKANTH C, VENKATESH D, et al. Enhancing the resolution of brain MRI images using generative adversarial networks (GANs)[C]// International Conference on Computational Intelligence and Sustainable Engineering Solutions. New York: IEEE,2023:19-25.

[10] IWAMOTO Y, TAKEDA K, LI Y H, et al. Unsupervised MRI super resolution using deep external learning and guided residual dense network with multimodal image priors[J]. IEEE Transactions on Emerging Topics in Computational Intelligence, 2023, 7(2):426-435.

[11] LIN J H, MIAO Q, SURAWECH C, et al. High-resolution 3D MRI with deep generative networks via novel slice-profile transformation super-resolution[J]. IEEE Access, 2023,11:95022-95036.

[12] ZHANG Y L, TIAN Y P, KONG Y, et al. Residual dense network for image super-resolution[C]// IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE,2018:2472-2481.

[13] LI J C, FANG F M, MEI K F, et al. Multi-scale residual network for image super-resolution[C]// Proceedings of the European Conference on Computer Vision. New York: IEEE,2018:517-532.

[14] CHEN N, WANG S Y, LU R, et al. Clothing parsing based on multi-scale fusion and improved self-attention mechanism[J]. Journal of Donghua University (English Edition), 2023, 40(6):661-666.

[15] REN H, WANG X G. Review of attention mechanism[J]. Journal of Computer Applications, 2021, 41(S1):1-6. (in Chinese)

[16] ZHA J H, YAN C R, ZHANG Y T, et al. Image retrieval with text manipulation by local feature modification[J]. Journal of Donghua University (English Edition), 2023, 40(4):404-409.