A two-stage restoration of distorted underwater images using compressive sensing and image registration

Abstract

Imaging through a time-varying water surface exhibits severe non-rigid geometric distortions and motion blur. Theoretically, although the water surface possesses smoothness and temporal periodicity, random fluctuations are inevitable in an actual video sequence. Meanwhile, considering the distribution of information, the image structure contributes more to the restoration. In this paper, a new two-stage restoration method for distorted underwater video sequences is presented. During the first stage, salient feature points, which are selected through multiple methods, are tracked across the frames, and the motion fields at all pixels are estimated using a compressive sensing solver to remove the periodic distortions. During the second stage, the combination of a guided filter algorithm and an image registration method is applied to remove the structural-information-oriented residual distortions. Finally, the experiment results show that the method outperforms other state-of-the-art approaches in terms of the recovery effect and time.

Keywords

Periodic wave / Random wave / Compressive sensing (CS) / Image registration (IR) / Structural information

References

[1] Gilles J, Osher S. Wavelet burst accumulation for turbulence mitigation. J Electron Imaging, 2016, 25(3): 033003.

[2] Panetta K, Gao C, Agaian S. Human-visual-system-inspired underwater image quality measures. IEEE J Ocean Eng, 2015, 41(3): 541-551.

[3] Paul N, De Chillaz A, Collette J. On-line restoration for turbulence degraded video in nuclear power plant reactors. Signal Image Video Process, 2015, 9(3): 601-610.

[4] Turlaev DG, Dolin LS. On observing underwater objects through a wavy water surface: a new algorithm for image correction and laboratory experiment. Izvestiya Atmos Ocean Phys, 2013, 49(3): 339-345.

[5] Zhang Z, Yang X. Bio-inspired motion planning for reaching movement of a manipulator based on intrinsic tau jerk guidance. Adv Manuf, 2019, 7: 315-325.

[6] Holohan ML, Dainty JC. Low-order adaptive optics: a possible use in underwater imaging?. Opt Laser Technol, 1997, 29(1): 51-55.

[7] Murase H. Surface shape reconstruction of a nonrigid transparent object using refraction and motion. IEEE Trans Pattern Anal Mach Intell, 1992, 14: 1045-1052.

[8] Seemakurthy K, Rajagopalan AN. Deskewing of underwater images. IEEE Trans Image Process, 2015, 24(3): 1046-1059.

[9] Tian Y, Narasimhan SG (2009) Seeing through water: image restoration using model-based tracking. In: Proceedings of the IEEE 12th international conference on computer vision. IEEE, Kyoto, pp 2303–2310

[10] Cox C, Munk W. Slopes of the sea surface deduced from photographs of sun glitter. Bull Scripps Inst Oceanogr, 1956, 6: 401-479.

[11] Efros A, Isler V, Shi J et al (2004) Seeing through water. In: Proceedings of conference and workshop on neural information processing systems. Neural Information Processing Systems Foundation, Vancouver, pp 393–400

[12] Donate A, Dahme G, Ribeiro E (2006) Classification of textures distorted by water waves. In: Proceedings of international conference on pattern recognition. IEEE, Hong Kong, pp 421–424

[13] Donate A, Ribeiro E (2006) Improved reconstruction of images distorted by water waves. In: Advances in computer graphics and computer vision. Springer, Berlin, pp 264–277

[14] Wen Z, Lambert A, Fraser D, et al. Bispectral analysis and recovery of images distorted by a moving water surface. Appl Opt, 2010, 49(33): 6376-6384.

[15] Kanaev AV, Ackerman J, Fleet E et al (2009) Imaging through the air-water interface. In: Proceedings of computational optical sensing and imaging. OSA, San Jose, pp 13–15

[16] Kanaev AV, Hou W, Restaino SR, et al. Restoration of images degraded by underwater turbulence using structure tensor oriented image quality (STOIQ) metric. Opt Express, 2015, 23(13): 17077-17090.

[17] Kanaev AV, Hou W, Restaino SR et al (2014) Correction methods for underwater turbulence degraded imaging. In: Proceedings of remote sensing of clouds and the atmosphere XIX; and optics in atmosphereic propagation and adaptive systems XVII. Proc. SPIE, 92421P

[18] Zhang R, He D, Li Y, et al. Synthetic imaging through wavy water surface with centroid evolution. Opt Express, 2018, 26(20): 26009-26019.

[19] Lau CP, Lai YH, Lui LM. Variational models for joint subsampling and reconstruction of turbulence-degraded images. J Sci Comput, 2019, 78(3): 1488-1525.

[20] Lau CP, Lai YH, Lui LM. Reconstruction of atmospheric turbulence-distorted images via RPCA and quasiconformal maps. Inverse Probl, 2019, 35(7): 074002.

[21] Oreifej O, Guang S, Pace T et al (2011) A two-stage reconstruction approach for seeing through water. In: Proceedings of IEEE conference on computer vision and pattern recognition. IEEE, Colorado, pp 1153–1160

[22] Halder KK, Paul M, Tahtali M, et al. Correction of geometrically distorted underwater images using shift map analysis. J Opt Soc Am A, 2017, 34(4): 666-673.

[23] Zhang Z, Yang X. Reconstruction of distorted underwater images using robust registration. Opt Express, 2019, 27(7): 9996-10008.

[24] Li Z, Murez Z, Kriegman D et al (2018) Learning to see through turbulent water. In: Proceedings of 2018 IEEE winter conference on applications of computer vision. IEEE, Lake Tahoe, pp 512–520

[25] Lau CP, Yung CP, Lui LM. Image retargeting via beltrami representation. IEEE Trans Image Process, 2018, 27(12): 5787-5801.

[26] James JG, Agrawal P, Rajwade A (2019) Restoration of non-rigidly distorted underwater image using a combination of compressive sensing and local polynomial image representations. In: Proceedings of the IEEE international conference on computer vision. IEEE, Seoul, pp 7839–7848

[27] He K, Sun J, Tang X. Guided image filtering. IEEE Trans Pattern Anal Mach Intell, 2012, 35(6): 1397-1409.

[28] Bay H, Tuytelaars T, Van GL (2006) Surf: speeded up robust features. In: Proceedings of European conference on computer vision. Graz, pp 404–417

[29] Rosten E, Porter R, Drummond T. Faster and better: a machine learning approach to corner detection. IEEE Trans Pattern Anal Mach Intell, 2008, 32(1): 105-119.

[30] Leutenegger S, Chli M, Siegwart RY (2011) BRISK: Binary robust invariant scalable keypoints. In: Proceedings of international conference on computer vision. IEEE, Barcelona, pp 2548–2555

[31] Nam T, Kim S, Jung D. Hardware implementation of KLT tracker for real-time intruder dectection and tracking using on-board camera. Int J Aeronaut Space Sci, 2019, 20(1): 300-314.

[32] Nakanishi-Ohno Y, Hukushima K. Data-driven diagnosis for compressed sensing with cross validation. Phys Rev E, 2018, 98(5): 052120.

[33] Sun T, Tang YG, Zhang Z. Structural information reconstruction of distorted underwater images using image registration. Appl Sci, 2019, 10(16): 5670.

[34] Rueckert D, Sonoda LI, Hayes C, et al. Nonrigid registration using freeform deformations: application to breast MR images. IEEE Trans Med Imaging, 1999, 18(8): 712-721.

[35] Berdinsky D, Kim TW, Cho D, et al. Based of T-meshes and the refinement of hierarchical B-splines. Comput Methods Appl Mech Eng, 2015, 283: 841-855.

[36] Xue W, Zhang L, Mou XQ, et al. Gradient magnitude similarity deviation: a highly efficient perceptual image quality index. IEEE Trans Image Process, 2013, 23(2): 684-695.

[37] Zhang L, Zhang L, Mou XQ, et al. FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process, 2011, 20(8): 2378-2386.

Funding

National Key R&D Program of China(2018YFB1309200)