Face image super-resolution reconstruction algorithm based on residual attention mechanism

Abstract

Aiming at the problems such as low reconstruction efficiency, fuzzy texture details, and difficult convergence of reconstruction network face image super-resolution reconstruction algorithms, a new super-resolution reconstruction algorithm with residual concern was proposed. Firstly, to solve the influence of redundant and invalid information about the face image super-resolution reconstruction network, an attention mechanism was introduced into the feature extraction module of the network, which improved the feature utilization rate of the overall network. Secondly, to alleviate the problem of gradient disappearance, the adaptive residual was introduced into the network to make the network model easier to converge during training, and features were supplemented according to the needs during training. The experimental results showed that the proposed algorithm had better reconstruction performance, more facial details, and clearer texture in the reconstructed face image than the comparison algorithm. In objective evaluation, the proposed algorithm's peak signal-to-noise ratio and structural similarity were also better than other algorithms.

Keywords

face image / super-resolution reconstruction / residual network / attention mechanism

References

[1] JOURABLOO A, LIU X. Pose-invariant 3D face alignment//International Conference On Computer Vision (ICCV), December 7-13, 2015, Santiago, Chile. New York: IEEE, 2015: 3694-3702.

[2] LI Y J, LIU S F, YANG J M, et al. Generative face completion//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI, USA. New York: IEEE, 2017: 5892-5900.

[3] JIN Y G, BOUGANIS C S. Robust multi-image based blind face hallucination//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 7-12, 2015, Boston, MA, USA. New York: IEEE, 2015: 5252-5260.

[4] YANG J, LUO L, QIAN J J, et al. Nuclear norm based matrix regression with applications to face recognition with occlusion and illumination changes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(1): 156-171.

[5] ROTH J, TONG Y Y, LIU X M. Adaptive 3D face reconstruction from unconstrained photo collections//IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 27-30, 2016, Las Vegas, NV, USA. New York: IEEE, 2016: 4197-4206.

[6] BAKER S, KANADE T. Hallucinating faces//Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580), March 28-30, 2000, Grenoble, France. New York: IEEE, 2002: 83-88.

[7] WANG X G, TANG X O. Hallucinating face by eigentransformation. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 2005, 35(3): 425-434.

[8] DONG C, LOY C C, HE K M, et al. Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295-307.

[9] DONG C, LOY C C, TANG X O. Accelerating the super-resolution convolutional neural network//European Conference on Computer Vision, October 11-14, 2016, Cham: Springer, 2016: 391-407.

[10] KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks//Conference on Computer Vision and Pattern Recognition (CVPR), June 27-30, 2016, Las Vegas, NV, USA. New York: IEEE, 2016: 1646-1654.

[11] ZHANG Y L, TIAN Y P, KONG Y, et al. Residual dense network for image super-resolution//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, UT, USA. New York: IEEE, 2018: 2472-2481.

[12] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition//IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 27-30, 2016, Las Vegas, NV, USA. New York: IEEE, 2016: 770-778.

[13] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module//European Conference on Computer Vision, September 8-14, 2018, Munich, Germany. Cham: Springer, 2018: 3-19.

[14] LIU Y, DONG Z L, LIM K P, et al. A densely connected face super-resolution network based on attention mechanism//2020 15th IEEE Conference on Industrial Electronics and Applications (ICIEA), November 9-13, 2020, Kristiansand, Norway. New York: IEEE, 2020: 148-152.

[15] HU J, SHEN L, SUN G. Squeeze-and-excitation networks//IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, UT, USA. New York: IEEE, 2018: 7132-7141.

[16] ZHU X Z, CHENG D Z, ZHANG Z, et al. An empirical study of spatial attention mechanisms in deep networks//2019 IEEE/CVF International Conference on Computer Vision (ICCV), October 27-November 2, 2019, Seoul, Korea (South). New York: IEEE, 2019: 6687-6696.

[17] CHEN C F, GONG D H, WANG H, et al. Learning spatial attention for face super-resolution. IEEE Transactions on Image Processing, 2020, 30: 1219-1231.

[18] LIU S, GANG R P, LI C H, et al. Adaptive deep residual network for single image super-resolution. Computational Visual Media, 2019, 5(4): 391-401.

[19] WANG H F, XU Y, WEI Y M, et al. Image super-resolution reconstruction based on parallel convolution and residual network. Journal of Computer Applications, 2022, 42(5): 1570-1576.

[20] KEYS R. Cubic convolution interpolation for digital image processing. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1981, 29(6): 1153-1160.

[21] LIU Z W, LUO P, WANG X G, et al. Deep learning face attributes in the wild//2015 IEEE International Conference on Computer Vision (ICCV), December 7-13, 2015, Santiago, Chile. New York: IEEE, 2015: 3730-3738.

[22] LE V, BRANDT J, LIN Z, et al. Interactive facial feature localization//European Conference on Computer Vision, October 7-13, 2012, Florence, Italy, Heidelberg: Springer, 2012: 679-692.

[23] PEREYRA M, SCHNITER P, CHOUZENOUX É, et al. A survey of stochastic simulation and optimization methods in signal processing. IEEE Journal of Selected Topics in Signal Processing, 2016, 10(2): 224-241.

[24] LEDIG C, THEIS L, HUSZÁR F, et al. Photo-realistic single image super-resolution using a generative adversarial network//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI, USA. New York: IEEE, 2017: 105-114.

[25] WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2004, 13(4): 600-612.