Graph-Based Transform and Dual Graph Laplacian Regularization for Depth Map Denoising

Abstract

Owing to the constraints of depth sensing technology, images acquired by depth cameras are inevitably mixed with various noises. For depth maps presented in gray values, this research proposes a novel denoising model, termed graph-based transform(GBT) and dual graph Laplacian regularization(DGLR)(DGLRGBT). This model specifically aims to remove Gaussian white noise by capitalizing on the nonlocal self-similarity(NSS) and the piecewise smoothness properties intrinsic to depth maps. Within the group sparse coding(GSC) framework, a combination of GBT and DGLR is implemented. Firstly, within each group, the graph is constructed by using estimates of the true values of the averaged blocks instead of the observations. Secondly, the graph Laplacian regular terms are constructed based on rows and columns of similar block groups, respectively. Lastly, the solution is obtained effectively by combining the alternating direction multiplication method(ADMM) with the weighted thresholding method within the domain of GBT.

Keywords

depth map / graph signal processing / dual graph Laplacian regularization(DGLR) / graph-based transform(GBT) / group sparse coding(GSC)

References

[1] IBRAHIM M M, LIU Q, KHAN R, et al. Depth map artefacts reduction:a review[J]. IET Image Processing, 2020, 14(12):2630-2644.

[2] ZHONG Z W, LIU X M, JIANG J J, et al. Guided depth map super-resolution:a survey[J]. ACM Computing Surveys, 2023,55 (14Sup.):1-36.

[3] ZHANG Y B, FENG Y H, LIU X M, et al. Color-guided depth image recovery with adaptive data fidelity and transferred graph Laplacian regularization[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(2):320-333.

[4] WANG Z X, YAN Z Q, YANG J. SGNet:structure guided network via gradient-frequency awareness for depth map super-resolution[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2024, 38(6):5823-5831.

[5] ZHONG Z W, LIU X M, JIANG J J, et al. Deep attentional guided image filtering[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023, 35(9):12236-12250.

[6] ZHAO Z X, ZHANG J S, XU S, et al. Discrete cosine transform network for guided depth map super-resolution[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE,2022:5687-5697.

[7] WANG K, ZHAO L J, ZHANG J J, et al. Joint depth map super-resolution method via deep hybrid-cross guidance filter[J]. Pattern Recognition, 2023,136:109260.

[8] ZHANG Y Y, HE X H, CHEN H G, et al. Depth map super-resolution via learned nonlocal model and enhanced local regularization[J]. Signal Processing, 2024,218:109368.

[9] BUADES A, COLL B, MOREL J M. A non-local algorithm for image denoising[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). New York: IEEE,2005:60-65.

[10] GASTAL E S L, OLIVEIRA M M. Adaptive manifolds for real-time high-dimensional filtering[J]. ACM Transactions on Graphics, 2012, 31(4):1-13.

[11] DABOV K, FOI A, KATKOVNIK V, et al. Image denoising by sparse 3-D transform-domain collaborative filtering[J]. IEEE Transactions on Image Processing, 2007, 16(8):2080-2095.

[12] GU S H, ZHANG L, ZUO W M, et al. Weighted nuclear norm minimization with application to image denoising[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE,2014:2862-2869.

[13] ZHA Z Y, YUAN X, WEN B H, et al. From rank estimation to rank approximation:rank residual constraint for image restoration[J]. IEEE Transactions on Image Processing, 2019,29:3254-3269.

[14] ZHA Z Y, WEN B H, YUAN X, et al. Image restoration via reconciliation of group sparsity and low-rank models[J]. IEEE Transactions on Image Processing, 2021,30:5223-5238.

[15] ZHA Z Y, WEN B H, YUAN X, et al. Low-rankness guided group sparse representation for image restoration[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023, 34(10):7593-7607.

[16] HU W, LI X, CHEUNG G, et al. Depth map denoising using graph-based transform and group sparsity[C]//2013 IEEE 15th International Workshop on Multimedia Signal Processing (MMSP). New York: IEEE,2013:1-6.

[17] YAN C G, LI Z S, ZHANG Y B, et al. Depth image denoising using nuclear norm and learning graph model[J]. ACM Transactions on Multimedia Computing,Communications,and Applications, 2020, 16(4):1-17.

[18] OU Y, SWAMY M N S, LUO J Q, et al. Single image denoising via multi-scale weighted group sparse coding[J]. Signal Processing, 2022,200:108650.

[19] LI F, RU Y M, LV X G. Patch-based weighted SCAD prior for rician noise removal[J]. Journal of Scientific Computing, 2021, 90(1):26.

[20] LIU S Q, HU Q, LI P F, et al. Speckle suppression based on weighted nuclear norm minimization and grey theory[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(5):2700-2708.

[21] LIU S Q, GAO L L, LEI Y, et al. SAR speckle removal using hybrid frequency modulations[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(5):3956-3966.

[22] MARTINS W A, LIMA J B, RICHARD C, et al. A primer on graph signal processing[M]//Signal Processing and Machine Learning Theory. Cambridge,Massachusetts: Academic Press, 2024:961-1008.

[23] SCHARSTEIN D, SZELISKI R.High-accuracy stereo depth maps using structured light[C]//2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2003,1:195-202.

[24] SCHARSTEIN D, PAL C. Learning conditional random fields for stereo[C]//2007 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2007,1-8:1688.

[25] HIRSCHMULLER H, SCHARSTEIN D. Evaluation of cost functions for stereo matching[C]//2007 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2007,1-8:2134.

[26] ZHANG K, ZUO W M, CHEN Y J, et al. Beyond a Gaussian denoiser:residual learning of deep CNN for image denoising[J]. IEEE Transactions on Image Processing, 2017, 26(7):3142-3155.

[27] ZHANG K, ZUO W M, ZHANG L. FFDNet:toward a fast and flexible solution for CNN based image denoising[J]. IEEE Transactions on Image Processing, 2018, 27(9):4608-4622.

Funding

National Natural Science Foundation of China(62372100)