A method for obtaining complete point cloud of tongue surface based on fringe projection trinocular system

Abstract

The traditional tongue diagnosis process has the problem of poor objectivity. Applying computer vision technology to tongue diagnosis can effectively promote its objectivity. Binocular stereo vision combined with structured light fringe projection technology is a common method for 3D measurement. However, in the measurement scenario of tongue diagnosis, due to the presence of saliva and fluids on the tongue surface, there are high-reflectance areas with significant random distribution in the fringe images, leading to errors in phase calculation and point cloud loss. A trinocular measurement system was proposed based on fringe projection, where a trinocular system and three binocular subsystems were composed of three cameras. Dual-epipolar constraint based on phase and order constraints was introduced to enhance the accuracy of trinocular stereo matching. Supplementary matching points were utilized to optimize the trinocular matching point sets, reconstructing point clouds in high-reflectance areas. The results indicated that, compared to traditional binocular systems, this system achieved improved matching and reconstruction accuracy. Particularly in real tongue surface measurements, it could generate point clouds with clear textures and complete features. It could effectively measure the highly reflective area of the tongue surface and facilitate objective tongue diagnosis.

Keywords

3D measurement / trinocular system / fringe projection / dual-epipolar constraint

References

[1] WANG Y H, ZHANG Q, HU Y, et al. 3D small-field surface imaging based on microscopic fringe projection profilometry:a review. Chinese Optics, 2021, 14(3): 447-457.

[2] RAO L, DA F P. High dynamic range 3D shape determination based on automatic exposure selection. Journal of Visual Communication and Image Representation, 2018, 50: 217-226.

[3] JIANG M N, LI F, YAN T T. 3D measurement of highly-reflective surface based on phase detection of projected grating. Electronics Optics and Control, 2021, 28(1): 94-97.

[4] JIANG H Z, ZHAO H J, LI X D. High dynamic range fringe acquisition: a novel 3-D scanning technique for high-reflective surfaces. Optics and Lasers in Engineering, 2012, 50(10): 1484-1493.

[5] PENG G Z, CHEN W J. Fringe pattern inpainting based on convolutional neural network denoising regularization. Acta Optica Sinica, 2020, 40(18): 1810002.

[6] FENG W, XU S N, WANG H H, et al. Three-dimensional measurement method of highly reflective surface based on per-pixel modulation. Chinese Optics, 2022, 15(3): 488-497.

[7] LIU F, HE C Q, SHEN A M, et al. Optimized compensation method of divisional projection for saturated region of structured light. Acta Optica Sinica, 2018, 38(6): 0612001.

[8] LIN H, GAO J, MEI Q, et al. Adaptive digital fringe projection technique for high dynamic range three-dimensional shape measurement. Optics Express, 2016, 24(7): 7703-7718.

[9] ZHAO S B, LIU L Y, MA M Y. Adaptive high-dynamic range three-dimensional shape measurement using DMD camera. IEEE Access, 2019, 7: 67934-67943.

[10] SALAHIEH B, CHEN Z Y, RODRIGUEZ J J, et al. Multi-polarization fringe projection imaging for high dynamic range objects. Optics Express, 2014, 22(8): 10064-10071.

[11] RIVIERE J, RESHETOUSKI I, FILIPI L, et al. Polarization imaging reflectometry in the wild. ACM Transactions on Graphics, 2017, 36(6): 206.

[12] FENG S J, CHEN Q, ZUO C, et al. Fast three-dimensional measurements for dynamic scenes with shiny surfaces. Optics Communications, 2017, 382: 18-27.

[13] LIU G H, LIU X Y, FENG Q Y. 3D shape measurement of objects with high dynamic range of surface reflectivity. Applied Optics, 2011, 50(23): 4557-4565.

[14] NAYAR S K, IKEUCHI K, KANADE T. Surface reflection: Physical and geometrical perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1991, 13(7): 611-634.

[15] LI M H, CAO Y P, WU H T. Three-dimensional reconstruction for highly reflective diffuse object based on online measurement. Optics Communications, 2023, 533: 129276.

[16] MA Y P, LI Q W, XING J, et al. An intelligent object detection and measurement system based on trinocular vision. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(3): 711-724.

[17] CONEN N, LUHMANN T, MAAS H G. Development and evaluation of a miniature trinocular camera system for surgical measurement applications. Journal of Photogrammetry Remote Sensing & Geoinformation Science, 2017, 85(2): 127-138.

[18] XIAO Y, BIN L K. A prism-based single-lens stereovision system: From trinocular to multi-ocular. Image and Vision Computing, 2007, 25(11): 1725-1736.

[19] FU L H, WANG C Y, HE J J, et al. Camera pose measurement method based on feature matching. Journal of Measurement Science and Instrumentation, 2023, 14(1): 1-8.

[20] TOWERS C E, TOWERS D P, JONES J D C. Absolute fringe order calculation using optimised multi-frequency selection in full-field profilometry. Optics and Lasers in Engineering, 2005, 43(7): 788-800.

[21] ZHEN T X, HUANG S, LI Y F, et al. Key techniques for vision based 3D reconstruction: a review. Acta Automatica Sinica, 2020, 46(4): 631-652.

[22] STENTOUMIS C, GRAMMATIKOPOULOS L, KALISPERAKIS I, et al. A local adaptive approach for dense stereo matching in architectural scene reconstruction. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2013, XL-5/W1: 219-226.

[23] KAZHDAN M, HOPPE H. Screened poisson surface reconstruction. ACM Transactions on Graphics, 2013, 32(3): 29.