Wavelet based deep learning for depth estimation from single fringe pattern of fringe projection profilometry

Abstract

Depth estimation from single fringe pattern is a fundamental task in the field of fringe projection three-dimensional (3D) measurement. Deep learning based on a convolutional neural network (CNN) has attracted more and more attention in fringe projection profilometry (FPP). However, most of the studies focus on complex network architecture to improve the accuracy of depth estimation with deeper and wider network architecture, which takes greater computational and lower speed. In this letter, we propose a simple method to combine wavelet transform and deep learning method for depth estimation from the single fringe pattern. Specially, the fringe pattern is decomposed into low-frequency and high-frequency details by the two-dimensional (2D) wavelet transform, which are used in the CNN network. Experiment results demonstrate that the wavelet-based deep learning method can reduce the computational complexity of the model by 4 times and improve the accuracy of depth estimation. The proposed wavelet-based deep learning models (UNet-Wavelet and hNet-Wavelet) are efficient for depth estimation of single fringe pattern, achieving better performance than the original UNet and hNet models in both qualitative and quantitative evaluation.

References

[1] GorthiS S, RastogiP. Fringe projection techniques: whether we are?[J]. Optics and lasers in engineering, 2009, 48(2):133-140

[2] SongL M, LiX Y, YangY G, et al.. Structured-light based 3D reconstruction system for cultural relic packaging[J]. Sensors (Basel), 2018, 18(9):2981

[3] LiB, AnY, CappelleriD, et al.. High-accuracy, high-speed 3D structured light imaging techniques and potential applications to intelligent robotics[J]. International journal of intelligent robotics and applications, 2017, 1(1): 86-103

[4] ZhangS. Absolute phase retrieval methods for digital fringe projection profilometry: a review[J]. Optics and lasers in engineering, 2018, 107: 28-37

[5] ZuoC, FengS J, HuangL, et al.. Phase shifting algorithms for fringe projection profilometry: a review[J]. Optics and lasers in engineering, 2018, 109: 23-59

[6] ZhangS. High-speed 3D shape measurement with structured light methods: a review [J]. Optics and lasers in engineering, 2019, 106: 119-131

[7] SongL M, GaoY, ZhuX J, et al.. A 3D measurement method based on multi-view fringe projection by using a turntable[J]. Optoelectronics letters, 2016, 12(6):389-394

[8] JiY, ChenY, SongL M, et al.. 3D defect detection of connectors based on structured light[J]. Optoelectronics letters, 2021, 17(2):107-111

[9] YuH T, HanB W, BaiL F, et al.. Untrained deep learning-based fringe projection profilometry[J]. APL photonics, 2022, 7: 016102

[10] SpoorthiG E, GorthiR K S S, GorthiS. PhaseNet 2.0: phase unwrapping of noisy data based on deep learning approach[J]. IEEE transactions on image processing, 2020, 29: 4862-4872

[11] SamV D J, JorisJ J D. Deep neural networks for single shot structured light profilometry[J]. Optics express, 2019, 27: 17091-17101

[12] NguyenH, WangY Z, WangZ Y. Single-shot 3D shape reconstruction using structured light and deep convolutional neural networks[J]. Sensors, 2020, 20(13):3718

[13] NguyenH, LyK L, TranT, et al.. hNet: single-shot 3D shape reconstruction using structured light and h-shaped global guidance network[J]. Results in optics, 2021, 4: 100104

[14] YUAN M, ZHU X, HOU L. Depth estimation from single frame fringe projection pattern based on R2U-Net[J]. Laser and optoelectronics progress, 2021: 1–19. (in Chinese)

[15] JiaT, et al.. Depth measurement based on a convolutional neural network and structured light[J]. Measurement science and technology, 2022, 33: 025202

[16] WangL, LuD Q, QiuR W, et al.. 3D reconstruction from structured-light profilometry with dual-path hybrid network[J]. Eurasip journal on advances in signal processing, 2022, 2022: 14

[17] HuangH B, HeR, SunZ N, et al.. Wave-let-SRNet: a wavelet-based CNN for multi-scale face super resolution[C], 2017, IEEE: 1698-1706

[18] XueS K, QiuW Y, LiuF, et al.. Wavelet-based residual attention network for image super-resolution[J]. Neurocomputing, 2020, 382: 116-126

[19] LiuW, YanQ, ZhaoY Z. Densely self-guided wavelet network for image denoising[C], 2020, New York, IEEE: 1742-1750

[20] LiuL, LiuJ Z, YuanS X, et al.. Wavelet-based dual-branch network for image demoiréing[C], 2020, Cham, Springer: 86-102

[21] ZuoC, QianJ M, FengS J, et al.. Deep learning in optical metrology: a review[J]. Light: science and applications, 2022, 11(1): 39