Phase unwrapping based on deep learning in light field fringe projection 3D measurement

Abstract

Phase unwrapping is one of the key roles in fringe projection three-dimensional (3D) measurement technology. We propose a new method to achieve phase unwrapping in camera array light filed fringe projection 3D measurement based on deep learning. A multi-stream convolutional neural network (CNN) is proposed to learn the mapping relationship between camera array light filed wrapped phases and fringe orders of the expected central view, and is used to predict the fringe order to achieve the phase unwrapping. Experiments are performed on the light field fringe projection data generated by the simulated camera array fringe projection measurement system in Blender and by the experimental 3×3 camera array light field fringe projection system. The performance of the proposed network with light field wrapped phases using multiple directions as network input data is studied, and the advantages of phase unwrapping based on deep learning in light filed fringe projection are demonstrated.

References

[1] ChenR, XuJ, ZhangS. Comparative study on 3D optical sensors for short range applications[J]. Optics and lasers in engineering, 2022, 149: 106763

[2] GengJ. Structured-light 3D surface imaging: a tutorial[J]. Advances in optics and photonics, 2011, 3(2):128-160

[3] MarrugoA G, GaoF, ZhangS. State-of-the-art active optical techniques for three-dimensional surface metrology: a review[J]. Journal of the Optical Society of America A, 2020, 37(9):B60-B77

[4] ZuoC, FengS J, HuangL, et al.. Phase shifting algorithms for fringe projection profilometry: a review[J]. Optics and lasers in engineering, 2018, 109: 23-59

[5] ZhangS. High-speed 3D shape measurement with structured light methods: a review[J]. Optics and lasers in engineering, 2018, 106: 119-131

[6] YinW, ZuoC, FengS J, et al.. High-speed three-dimensional shape measurement using geometry-constraint-based number-theoretical phase unwrapping[J]. Optics and lasers in engineering, 2019, 115: 21-31

[7] PistellatoM, BergamascoF, AlbarelliA, et al.. Robust phase unwrapping by probabilistic consensus[J]. Optics and lasers in engineering, 2019, 121: 428-440

[8] ZhangS. Absolute phase retrieval methods for digital fringe projection profilometry: a review[J]. Optics and lasers in engineering, 2018, 107: 28-37

[9] AnH, CaoY, WuH, et al.. Spatial-temporal phase unwrapping algorithm for fringe projection profilometry[J]. Optics express, 2021, 29(13):20657-20672

[10] FengS J, ChenQ, GuG, et al.. Fringe pattern analysis using deep learning[J]. Advanced photonics, 2019, 1(2):025001-025001

[11] ZhengY, WangS D, LiQ, et al.. Fringe projection profilometry by conducting deep learning from its digital twin[J]. Optics express, 2020, 28(24):36568-36583

[12] ShiJ S, ZhuX J, WangH Y, et al.. Label enhanced and patch based deep learning for phase retrieval from single frame fringe pattern in fringe projection 3D measurement[J]. Optics express, 2019, 27(20):28929-28943

[13] NguyenH, WangY Z, WangZ Y. Single-shot 3D shape reconstruction using structured light and deep convolutional neural networks[J]. Sensors, 2020, 20(13):3718

[14] MachineniR C, SpoorthiG E, VengalaK S, et al.. End-to-end deep learning-based fringe projection framework for 3D profiling of objects[J]. Computer vision and image understanding, 2020, 199: 103023

[15] SpoorthiG E, GorthiR K S S, GorthiS. PhaseNet 2.0: phase unwrapping of noisy data based on deep learning approach[J]. IEEE transactions on image processing, 2020, 29: 4862-4872

[16] YinW, ChenQ, FengS J, et al.. Temporal phase unwrapping using deep learning[J]. Scientific reports, 2019, 9(1):1-12

[17] QianJ M, FengS J, TaoT Y, et al.. Deep-learning-enabled geometric constraints and phase unwrapping for single-shot absolute 3D shape measurement[J]. APL photonics, 2020, 5(4):046105

[18] OrthA, CrozierK B. Light field moment imaging[J]. Optics letters, 2013, 38(15): 2666-2668

[19] TaoT Y, ChenQ, ZhangY Z, et al.. Multi-view phase unwrapping with composite fringe patterns[C]//International Conference on Optical and Photonics Engineering (icOPEN 2016), September 26–30, 2016, Chengdu, China. Washington: SPIE, 2017, 10250: 240-245

[20] CaiZ W, LiuX L, ChenZ Z, et al.. Light-field-based absolute phase unwrapping[J]. Optics letters, 2018, 43(23):5717-5720

[21] WangZ W, YangY, LiuX L, et al.. Light-field-assisted phase unwrapping of fringe projection profilometry[J]. IEEE access, 2021, 9: 49890-49900

[22] SongL M, DongX X, XiJ T, et al.. A new phase unwrapping algorithm based on three wavelength phase shift profilometry method[J]. Optics & laser technology, 2013, 45: 319-329