Adaptive-basis decomposition-based low-rank network for efficient non-uniform motion deblurring

Abstract

In this study, we present a unified sparsity-driven framework that significantly enhances motion deblurring performance by integrating two key components: a custom-designed dataset and a low-rank module (LRM). This framework leverages the inherent sparsity of per-pixel blur kernels to bolster both deblurring accuracy and model interpretability. Firstly, we propose an adaptive-basis decomposition-based deblurring (ADD) approach, which constructs a tailored training dataset to enhance the generalization capacity of the deblurring network. The ADD framework adaptively decomposes motion blur into sparse basis elements, effectively addressing the intricacies associated with non-uniform blurs. Secondly, an LRM is proposed to improve the interpretability of deblurring models as a plug-and-play module, primarily designed to identify and harness the intrinsic sparse features in sharp images. A series of ablation studies have been conducted to substantiate the synergistic advantages of combining the proposed ADD with the LRM for overall improvement in deblurring efficacy. Subsequently, we empirically demonstrate through rigorous experimentation that incorporating the LRM into an existing Uformer network leads to substantial enhancement in reconstruction performance. This integration yields a sparsity-guided low-rank network (SGLRN). Operating under the overarching principle of sparsity, SGLRN consistently outperforms state-of-the-art methods across multiple standard deblurring benchmarks. Comprehensive experimental results, assessed through quantitative metrics and qualitative visual evaluations, provide compelling evidence of its effectiveness. The overall deblurring results are available at Google Drive.

References

[1] Wang M, Zhu F, Bai Y. An improved image blind deblurring based on dark channel prior. Optoelectronics letters, 2021, 17(1): 40-46 J]

[2] Lu Y C, Liu T P, Lin C H. Two-stage single image deblurring network based on deblur kernel estimation. Multimedia tools and applications, 2023, 82(11): 17055-17074 J]

[3] Sun Y, Zhi X, Jiang S, et al.. Image fusion for the novelty rotating synthetic aperture system based on vision transformer. Information fusion, 2024, 104: 102163 J]

[4] Zhang S, Tang G, Liu X, et al.. Retinex based low-light image enhancement using guided filtering and variational framework. Optoelectronics letters, 2018, 14(2): 156-160 J]

[5] Delbracio M, Garcia-Dorado I, Choi S, et al.. Polyblur: removing mild blur by polynomial reblurring. IEEE transactions on computational imaging, 2021, 7: 837-848 J]

[6] Gong D, Yang J, Liu L, et al.. From motion blur to motion flow: a deep learning solution for removing heterogeneous motion blur. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, July 21–26, 2017, Honolulu, Hawaii, USA, 2017 New York IEEE 2319-2328 [C]

[7] Zhang J, Pan J, Ren J, et al.. Dynamic scene deblurring using spatially variant recurrent neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 18–22, 2018, Salt Lake City, Utah, USA, 2018 New York IEEE 2521-2529 [C]

[8] Purohit K, Rajagopalan A N. Region-adaptive dense network for efficient motion deblurring. Proceedings of the AAAI Conference on Artificial Intelligence, February 7–12, 2020, New York, USA, 2020 Washington AAAI Press 11882-11889 [C]

[9] Wen Y, Chen J, Sheng B, et al.. Structure-aware motion deblurring using multi-adversarial optimized cyclegan. IEEE transactions on image processing, 2021, 30: 6142-6155 J]

[10] Li Y, Ren D, Shu X, et al.. Learning single image defocus deblurring with misaligned training pairs. Proceedings of the AAAI Conference on Artificial Intelligence, February 7–10, 2023, Washington DC, USA, 2023 Washington AAAI Press 1495-1503 [C]

[11] Nah S, Hyun Kim T, Mu Lee K. Deep multi-scale convolutional neural network for dynamic scene deblurring. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, July 21–26, 2017, Honolulu, Hawaii, USA, 2017 New York IEEE 3883-3891 [C]

[12] Deng J, Dong W, Socher R, et al.. Imagenet: a large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, June, 2009, Miami, Florida, USA, 2009 New York IEEE 248-255 [C]

[13] Hirsch M, Sra S, Schölkopf B, et al.. Efficient filter flow for space-variant multiframe blind deconvolution. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June, 2010, San Francisco, California, USA, 2010 New York IEEE 607-614 [C]

[14] Liu Y L, Lai W S, Chen Y S, et al.. Single-image HDR reconstruction by learning to reverse the camera pipeline. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June, 2020, Seattle, Washington, USA, 2020 New York IEEE 1651-1660 [C]

[15] Rim J, Lee H, Won J, et al.. Real-world blur dataset for learning and benchmarking deblurring algorithms. 16th European Conference on Computer Vision, August 23–28, 2020, Glasgow, UK, 2020 Heidelberg Springer International Publishing 184-201 [C]

[16] Ljubenović M, Figueiredo M A T. Blind image deblurring using class-adapted image priors. Proceedings of the 2017 IEEE International Conference on Image Processing, September, 2017, Beijing, China, 2017 New York IEEE 490-494 [C]

[17] Xie J, Hou G, Wang G, et al.. A variational framework for underwater image dehazing and deblurring. IEEE transactions on circuits and systems for video technology, 2021, 32(6): 3514-3526 J]

[18] Mohammad-Djafari A, Dumitru M. Bayesian sparse solutions to linear inverse problems with non-stationary noise with Student-t priors. Digital signal processing, 2015, 47: 128-156 J]

[19] Hu Z, Huang J B, Yang M H. Single image deblurring with adaptive dictionary learning. Proceedings of the 2010 IEEE International Conference on Image Processing, September, 2010, Hong Kong, China, 2010 New York IEEE 1169-1172 [C]

[20] Zhang H, Yang J, Zhang Y, et al.. Close the loop: joint blind image restoration and recognition with sparse representation prior. Proceedings of the 2011 International Conference on Computer Vision, November, 2011, Barcelona, Spain, 2011 New York IEEE 770-777 [C]

[21] Tofighi M, Li Y, Monga V. Blind image deblurring using row-column sparse representations. IEEE signal processing letters, 2017, 25(2): 273-277 J]

[22] Pan J, Sun D, Pfister H, et al.. Deblurring images via dark channel prior. IEEE transactions on pattern analysis and machine intelligence, 2017, 40(10): 2315-2328 J]

[23] Liu J, Sun Y, Xu X, et al.. Image restoration using total variation regularized deep image prior. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, May, 2019, Brighton, UK, 2019 New York IEEE 7715-7719 [C]

[24] Cai J, Zuo W, Zhang L. Dark and bright channel prior embedded network for dynamic scene deblurring. IEEE transactions on image processing, 2020, 29: 6885-6897 J]

[25] Wu F, Dong W, Huang T, et al.. Hybrid sparsity learning for image restoration: an iterative and trainable approach. Signal processing, 2021, 178: 107751 J]

[26] Li M, Gao S, Zhang C, et al.. Blind motion deblurring via L0 sparse representation. Computers & graphics, 2021, 97: 248-257 J]

[27] ZHA Z, WEN B, YUAN X, et al. Low-rankness guided group sparse representation for image restoration[J]. IEEE transactions on neural networks and learning systems, 2022.

[28] Mandracchia B, Liu W, Hua X, et al.. Optimal sparsity allows reliable system-aware restoration of fluorescence microscopy images. Science advances, 2023, 9(35): 9245 J]

[29] Wang Z, Cun X, Bao J, et al.. Uformer: a general U-shaped transformer for image restoration. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June, 2022, New Orleans, Louisiana, USA, 2022 New York IEEE 17683-17693 [C]

[30] Kupyn O, Budzan V, Mykhailych M, et al.. Deblurgan: blind motion deblurring using conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June, 2018, Salt Lake City, Utah, USA, 2018 New York IEEE 8183-8192 [C]

[31] Tao X, Gao H, Shen X, et al.. Scale-recurrent network for deep image deblurring. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June, 2018, Salt Lake City, Utah, USA, 2018 New York IEEE 8174-8182 [C]

[32] Gao H, Tao X, Shen X, et al.. Dynamic scene deblurring with parameter selective sharing and nested skip connections. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June, 2019, Long Beach, California, USA, 2019 New York IEEE 3848-3856 [C]

[33] Chen L, Chu X, Zhang X, et al.. Simple baselines for image restoration. 18th European Conference on Computer Vision, August, 2022, Copenhagen, Denmark, 2022 Heidelberg Springer International Publishing 17-33 [C]

RIGHTS & PERMISSIONS

Tianjin University of Technology