Underwater computational imaging: a survey

Abstract

With the increasingly urgent demand for marine research and development, optical imaging technology remains crucial for underwater close-range information detection. However, the inherent obstacles of light transmission in strongly attenuating environments constitute a bottleneck that restricts the development of traditional optical imaging technology. Underwater computational imaging has emerged gradually, leveraging its cross-disciplinary advantages. It deeply couples optical system design with signal calculation and processing and has a high utilization rate of focusing information. It can achieve qualitative breakthroughs in imaging resolution, scale, dimension, and hardware convenience. However, existing work is mostly limited to the extension of free-space computational imaging techniques to underwater environments, lacking systematic research on common needs and key technologies. Therefore, it is essential to refine the connotation and advantages of underwater computational imaging technology, especially in combination with highly complex and nonlinear application scenarios, and to identify potential development space and breakthroughs.

Keywords

Underwater optical sensing / Underwater computational imaging connotation / Imaging chain / Underwater optical imaging applications

References

[1] Barbastathis G, Ozcan A, Situ G. On the use of deep learning for computational imaging. Optica. 2019, 6(8): 921-943

[2] Beall C, Lawrence BJ, Ila V, Dellaert F (2010) 3D reconstruction of underwater structures. 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, p 4418–4423. https://doi.org/10.1109/IROS.2010.5649213

[3] Bekerman Y, Avidan S, Treibitz T (2020) Unveiling optical properties in underwater images. 2020 IEEE International Conference on Computational Photography, Saint Louis, p 1–12. https://doi.org/10.1109/ICCP48838.2020.9105267

[4] Berman D, Levy D, Avidan S, Treibitz T. Underwater single image color restoration using haze-lines and a new quantitative dataset. IEEE Trans Pattern Anal Mach Intell. 2020, 43(8): 2822-2837

[5] Bianco G, Gallo A, Bruno F, Muzzupappa M (2013) A comparative analysis between active and passive techniques for underwater 3D reconstruction of close-range objects. Sensors 13(8):11007–11031. https://doi.org/10.3390/s130811007

[6] Bochdansky AB, Jericho MH, Herndl GJ. Development and deployment of a point-source digital inline holographic microscope for the study of plankton and particles to a depth of 6000 m. Limnol Oceanogr Methods. 2013, 11(1): 28-40

[7] Bodenmann A, Thornton B, Ura T. Generation of high-resolution three-dimensional reconstructions of the seafloor in color using a single camera and structured light. J Field Robot. 2017, 34(5): 833-851

[8] Brandou V, Allais AG, Perrier M, Malis E, Rives P, Sarrazin J et al (2007) 3D reconstruction of natural underwater scenes using the stereovision system IRIS. OCEANS 2007-Europe, Aberdeen, p 1–6. https://doi.org/10.1109/OCEANSE.2007.4302315

[9] Bruno F, Bianco G, Muzzupappa M, Barone S, Razionale AV (2011) Experimentation of structured light and stereo vision for underwater 3D reconstruction. ISPRS J Photogramm Remote Sens 66(4):508–518. https://doi.org/10.1016/j.isprsjprs.2011.02.009

[10] Chadebecq F, Vasconcelos F, Lacher R, Maneas E, Desjardins A, Ourselin S et al (2020) Refractive two-view reconstruction for underwater 3D vision. Int J Comput Vis 128:1101–1117. https://doi.org/10.1007/s11263-019-01218-9

[11] Chiang JY, Chen Y. Underwater image enhancement by wavelength compensation and dehazing. IEEE Trans Image Process. 2011, 21(4): 1756-1769

[12] Davis CS, Thwaites FT, Gallager SM, Hu Q (2005) A three-axis fast-tow digital Video Plankton Recorder for rapid surveys of plankton taxa and hydrography. Limnol Oceanogr Methods 3(2):59–74. https://doi.org/10.4319/lom.2005.3.59

[13] Guo JC, Li CY, Guo CL, Chen SJ. Research progress of underwater image enhancement and restoration methods. J Image Graph. 2017, 22(3): 0273-0287

[14] Han M, Lyu Z, Qiu T, Xu M. A review on intelligence dehazing and color restoration for underwater images. IEEE Trans Syst Man Cybern Syst. 2018, 50(5): 1820-1832

[15] Horisaki R, Takagi R, Tanida J. Learning-based imaging through scattering media. Opt Express. 2016, 24(13): 13738-13743

[16] Hu HF, Li JQ, Li XB, Liu TG. Underwater polarization difference imaging with three degrees of freedom. Acta Opt Sin. 2021, 41(3): 0329001

[17] Jaffe JS. Computer modeling and the design of optimal underwater imaging systems. IEEE J Ocean Eng. 1990, 15(2): 101-111

[18] Knox C. Holographic microscopy as a technique for recording dynamic microscopic subjects. Science. 1966, 153(3739): 989-990

[19] Lai W, Ding J, Lin Y, Chuang Y (2015) Blur kernel estimation using normalized color-line prior. 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, p 64–72

[20] Li C, Guo C, Ren W, Cong R, Hou J, Kwong S et al (2019) An underwater image enhancement benchmark dataset and beyond. IEEE Trans Image Process 29:4376–4389. https://doi.org/10.1109/TIP.2019.2955241

[21] Li S, Deng M, Lee J, Sinha A, Barbastathis G. Imaging through glass diffusers using densely connected convolutional networks. Optica. 2018, 5(7): 803-813

[22] Li W, Loomis NC, Hu Q, Davis CS (2007) Focus detection from digital in-line holograms based on spectral l₁ norms. J Opt Soc Am A 24(10):3054–3062. https://doi.org/10.1364/JOSAA.24.003054

[23] Li Y, Xue Y, Tian L. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media. Optica. 2018, 5(10): 1181-1190

[24] Liu F, Han P, Wei Y, Yang K, Huang S, Li X et al (2018) Deeply seeing through highly turbid water by active polarization imaging. Opt Lett 43(20):4903–4906. https://doi.org/10.1364/OL.43.004903

[25] Liu R, Fan X, Zhu M, Hou M, Luo Z. Real-world underwater enhancement: challenges, benchmarks, and solutions under natural light. IEEE Trans Circuits Syst Video Technol. 2020, 30(12): 4861-4875

[26] Lyu M, Wang H, Li G, Zheng S, Situ G. Learning-based lensless imaging through optically thick scattering media. Adv Photonics. 2019, 1(3): 1-10

[27] Mait JN, Euliss GW, Athale RA. Computational imaging. Adv Opt Photonics. 2018, 10(2): 409-483

[28] Malkiel E, Alquaddoomi O, Katz J. Measurements of plankton distribution in the ocean using submersible holography. Meas Sci Technol. 1999, 10(12): 1142

[29] McGlamery B (1980) A computer model for underwater camera systems. Proceeding Society of Photo-Optical Instrumentation Engineers, Ocean Optics VI, Monterey, 1979, p 221–231. https://doi.org/10.1117/12.958279

[30] Narasimhan SG, Nayar SK, Sun B, Koppal SJ (2005) Structured light in scattering media. Tenth IEEE International Conference on Computer Vision, Beijing, p 420–427. https://doi.org/10.1109/ICCV.2005.232

[31] Negahdaripour S, Zhang H, Han X (2002) Investigation of photometric stereo method for 3-D shape recovery from underwater imagery. OCEANS '02 MTS/IEEE, Biloxi, p 1010–1017. https://doi.org/10.1109/OCEANS.2002.1192106

[32] Nie Y, He ZY. Underwater imaging and real-time optical image processing under illumination by light sources with different wavelengths. Acta Opt Sin. 2014, 34(7): 59-65

[33] Peng Y, Cosman PC. Underwater image restoration based on image blurriness and light absorption. IEEE Trans Image Process. 2017, 26(4): 1579-1594

[34] Pinto AM, Matos AC. MARESye: a hybrid imaging system for underwater robotic applications. Inf Fusion. 2020, 55: 16-29

[35] Samson S, Hopkins T, Remsen A, Langebrake L, Sutton T, Patten J (2001) A system for high-resolution zooplankton imaging. IEEE J Ocean Eng 26(4):671–676. https://doi.org/10.1109/48.972110

[36] Sarafraz A, Haus BK. A structured light method for underwater surface reconstruction. ISPRS J Photogramm Remote Sens. 2016, 114: 40-52

[37] Schechner YY, Karpel N (2004) Clear underwater vision. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, DC, p 536–543. https://doi.org/10.1109/CVPR.2004.1315078

[38] Spier O, Treibitz T, Gilboa G (2017) In situ target-less calibration of turbid media. 2017 IEEE International Conference on Computational Photography, Stanford, p 1–9. https://doi.org/10.1109/ICCPHOT.2017.7951491

[39] Sun H, Hendry DC, Player MA, et al. . In situ underwater electronic holographic camera for studies of plankton. IEEE J Ocean Eng. 2007, 32(2): 373-382

[40] Suo JL, Liu YB, Ji XY, Dai QH. Computational photography: keys, methods and applications. Acta Autom Sin. 2015, 41(4): 669-685

[41] Treibitz T, Schechner YY. Active polarization descattering. IEEE Trans Pattern Anal Mach Intell. 2008, 31(3): 385-399

[42] Wang Y, Liu H, Chau L. Single underwater image restoration using adaptive attenuation-curve prior. IEEE Trans Circuits Syst I Regul Pap. 2017, 65(3): 992-1002

[43] Watson J, Zielinski O. Subsea optics and imaging. 2013, Amsterdam, Elsevier

[44] Zhao YQ, Dai HM, Shen LH, Zhang JC (2020) Review of underwater polarization clear imaging methods. Infrared and Laser Engineering 49(6):43–53. https://doi.org/10.3788/IRLA20190574

[45] Zhong S, Li Z, Wang R (2012) Influence of back-scattering on underwater laser imaging. 42(7):735–738. https://doi.org/10.3969/j.issn.1001-5078.2012.07.003

[46] Zhou L, Xiao Y, Chen W. Imaging through turbid media with vague concentrations based on cosine similarity and convolutional neural network. IEEE Photonics J. 2019, 11(4): 1-15

[47] Zuo C, Feng SJ, Zhang XY, Han J, Qian C. Deep learning based computational imaging: status, challenges, and future. Acta Opt Sin. 2020, 40(1): 45-70

Funding

National Natural Science Foundation of China(62171421)