Underwater image enhancement via histogram similarity-oriented color compensation complemented by multiple attribute adjustment

Abstract

Underwater images are often influenced by color casts, low contrast, and blurred details. We observe that images taken in natural settings typically have similar histograms across color channels, while underwater images do not. To improve the natural appearance of an underwater image, it is critical to improve the histogram similarity across its color channels. To address this problem, we develop a histogram similarity-oriented color compensation method that corrects color casts by improving the histogram similarity across color channels in the underwater image. In addition, we apply the multiple attribute adjustment method, including max-min intensity stretching, luminance map-guided weighting, and high-frequency edge mask fusion, to enhance contrast, saturation, and sharpness, effectively addressing problems of low contrast and blurred details and eventually enhancing the overall appearance of underwater images. Particularly, the method proposed in this work is not based on deep learning, but it effectively enhances a single underwater image. Comprehensive empirical assessments demonstrated that this method exceeds state-of-the-art underwater image enhancement techniques. To facilitate public assessment, we made our reproducible code available at https://gitee.com/wanghaoupc/UIE_HS2CM2A.

Keywords

Histogram similarity / Color compensation / Multiple attribute adjustment / Underwater image enhancement

References

[1] Ancuti CO, Ancuti C, De Vleeschouwer C, et al. . Color balance and fusion for underwater image enhancement. IEEE Trans Image Process. 2018, 27(1): 379-393

[2] Canny J. A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell. 1986, 8(6): 679-698

[3] Chen L, Jiang Z, Tong L, et al. . Perceptual underwater image enhancement with deep learning and physical priors. IEEE Trans Circuits Syst Video Technol. 2021, 31(8): 3078-3092

[4] Drews PL, Nascimento ER, Botelho SS, et al. . Underwater depth estimation and image restoration based on single images. IEEE Comput Graphics Appl. 2016, 36(2): 24-35

[5] Fan H, Qi L, Chen C, et al. . Underwater optical 3-d reconstruction of photometric stereo considering light refraction and attenuation. IEEE J Oceanic Eng. 2022, 47(1): 46-58

[6] Guo ZH, Guo DS, Zheng HY, Gu ZR, Zheng B, Dong JY (2021) Image harmonization with transformer. 2021 IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, pp 14850–14859

[7] Hou G, Pan Z, Huang B, et al. . Hue preserving-based approach for underwater colour image enhancement. IET Image Proc. 2018, 12(2): 292-298

[8] Jiang ZY, Li ZX, Yang SZ, Fan X, Liu RS (2022) Target oriented perceptual adversarial fusion network for underwater image enhancement. IEEE Trans Circuits Syst Video Technol 32(10):6584–6598

[9] Krzysztof N (2016) Aces filmic tone mapping curve. https://knarkowicz.wordpress.com/2016/01/06/aces-filmic-tone-mapping-curve/. Accessed 6 Jan 2016

[10] Li C, Anwar S, Hou J, et al. . Underwater image enhancement via medium transmission-guided multi-color space embedding. IEEE Trans Image Process. 2021, 30: 4985-5000

[11] Li C, Anwar S, Porikli F. Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recogn. 2020, 98: 107038

[12] Li C, Guo C, Ren W, et al. . An underwater image enhancement benchmark dataset and beyond. IEEE Trans Image Process. 2020, 29: 4376-4389

[13] Li CY, Guo JC, Pang YW, Chen SJ, Wang J (2016) Single underwater image restoration by blue-green channels dehazing and red channel correction. 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China, pp 1731–1735

[14] Li L, Dong B, Rigall E, et al. . Marine animal segmentation. IEEE Trans Circuits Syst Video Technol. 2022, 32(4): 2303-2314

[15] Li T, Rong S, Zhao W, et al. . Underwater image enhancement using adaptive color restoration and dehazing. Opt Express. 2022, 30(4): 6216-6235

[16] Li X, Hou G, Li K, et al. . Enhancing underwater image via adaptive color and contrast enhancement, and denoising. Eng Appl Artif Intell. 2022, 111: 104759

[17] Liang Z, Ding XY, Jin J, Wang YF, Wang YL, Fu XP (2022) A color cast image enhancement method based on affine transform in poor visible conditions. IEEE Geosci Remote Sens Lett 19:1503905. https://doi.org/10.1109/LGRS.2022.3156264

[18] Lin YF, Shen LQ, Wang ZY, Wang K, Zhang X (2021b) Attenuation coefficient guided two-stage network for underwater image restoration. IEEE Signal Proc Lett 28:199–203

[19] Lin Y, Zhou J, Ren W, et al. . Autonomous underwater robot for underwater image enhancement via multi-scale deformable convolution network with attention mechanism. Comput Electron Agr. 2021, 191: 106497

[20] Liu R, Fan X, Zhu M, et al. . Real-world underwater enhancement: Challenges, benchmarks, and solutions under natural light. IEEE Trans Circuits Syst Video Technol. 2020, 30(12): 4861-4875

[21] Liu YT, Gu K, Cao JC, Wang SQ, Zhai GT, Dong JY et al (2023) UIQI: a comprehensive quality evaluation index for underwater images. IEEE Trans Multimedia. https://doi.org/10.1109/TMM.2023.3301226

[22] Lowe DG. Distinctive image features from scale-invariant keypoints. Int J Comput Vision. 2004, 60(2): 91-110

[23] Ma L, Liu R, Wang Y, et al. . Low-light image enhancement via self-reinforced retinex projection model. IEEE Trans Multimedia. 2023, 25: 3573-3586

[24] Meng Y, Rigall E, Chen X, et al. . Physics-guided generative adversarial networks for sea subsurface temperature prediction. IEEE Trans Neur Net Lear. 2023, 34(7): 3357-3370

[25] Panetta K, Gao C, Agaian S. Human-visual-system-inspired underwater image quality measures. IEEE J Oceanic Eng. 2016, 41(3): 541-551

[26] Peng YT, Cosman PC. Underwater image restoration based on image blurriness and light absorption. IEEE Trans Image Process. 2017, 26(4): 1579-1594

[27] Qi Q, Li K, Zheng H, et al. . Sguie-net: Semantic attention guided underwater image enhancement with multi-scale perception. IEEE Trans Image Process. 2022, 31: 6816-6830

[28] Qi Q, Zhang Y, Tian F, et al. . Underwater image co-enhancement with correlation feature matching and joint learning. IEEE Trans Circuits Syst Video Technol. 2022, 32(3): 1133-1147

[29] Rao Y, Liu WJ, Li KQ, Fan H, Wang S, Dong JY (2023) Deep color compensation for generalized underwater image enhancement. IEEE Trans Circuits Syst Video Technol. https://doi.org/10.1109/TCSVT.2023.3305777

[30] She M, Nakath D, Song Y, et al. . Refractive geometry for underwater domes. ISPRS J Photogramm Remote Sens. 2022, 183: 525-540

[31] Shi ZS, Liang J, Li QQ, Zheng HY, Gu ZR, Dong JY et al (2021) Multi-modal multi-action video recognition. 2021 IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, pp 13658–13667

[32] Song HJ, Chang LB, Chen ZW, Ren P (2021) Enhancement-Registration-Homogenization (ERH): a comprehensive underwater visual reconstruction paradigm. IEEE Trans Pattern Anal Mach Intell 44(10):6953–6967

[33] Song H, Chang L, Wang H, et al. . Dual-model: Revised imaging network and visual perception correction for underwater image enhancement. Eng Appl Artif Intell. 2023, 125: 106731

[34] Song W, Wang Y, Huang D, et al. . Enhancement of underwater images with statistical model of background light and optimization of transmission map. IEEE Trans Broadcast. 2020, 66(1): 153-169

[35] Sun SX, Wang H, Zhang H, Li MJ, Xiang M, Luo C et al (2022) Underwater image enhancement with reinforcement learning. IEEE J Ocean Eng. https://doi.org/10.1109/JOE.2022.3152519

[36] Wang H, Sun SX, Bai X, Wang J, Ren P (2023a) A reinforcement learning paradigm of configuring visual enhancement for object detection in underwater scenes. IEEE J Ocean Eng 48(2):443–461

[37] Wang H, Sun S, Ren P. Meta underwater camera: A smart protocol for underwater image enhancement. ISPRS J Photogramm Remote Sens. 2023, 195: 462-481

[38] Wang H, Sun SX, Ren P (2023c) Underwater color disparities: cues for enhancing underwater images toward natural color consistencies. IEEE Trans Circuits Syst Video Technol. https://doi.org/10.1109/TCSVT.2023.3289566

[39] Wang H, Sun SX, Wu XH, Li L, Zhang H, Li MJ et al (2021) A YOLOv5 baseline for underwater object detection. OCEANS 2021: San Diego–Porto, San Diego, CA, USA, pp 1–4

[40] Wang Y, Li N, Li Z, et al. . An imaging-inspired no-reference underwater color image quality assessment metric. Comput Electr Eng. 2018, 70: 904-913

[41] Xie C, Guo H, Dong JY (2022) LSENet: location and seasonality enhanced network for multiclass ocean front detection. IEEE Trans Geosci Remote Sens 60:4207609. https://doi.org/10.1109/TGRS.2022.3176635

[42] Yang M, Sowmya A. An underwater color image quality evaluation metric. IEEE Trans Image Process. 2015, 24(12): 6062-6071

[43] Yu F, He B, Liu J, et al. . Dual-branch framework: Auv-based target recognition method for marine survey. Eng Appl Artif Intell. 2022, 115: 105291

[44] Yuan JY, Cai ZC, Cao W (2022) TEBCF: real-world underwater image texture enhancement model based on blurriness and color fusion. IEEE Trans Geosci Remote Sens 60:4204315. https://doi.org/10.1109/TGRS.2021.3110575

[45] Yuan J, Cao W, Cai Z, et al. . An underwater image vision enhancement algorithm based on contour bougie morphology. IEEE Trans Geosci Remote Sens. 2021, 59(10): 8117-8128

[46] Zhang W, Dong L, Xu W. Retinex-inspired color correction and detail preserved fusion for underwater image enhancement. Comput Electron Agric. 2022, 192: 106585

[47] Zhang W, Jin S, Zhuang P, et al. . Underwater image enhancement via piecewise color correction and dual prior optimized contrast enhancement. IEEE Signal Process Lett. 2023, 30: 229-233

[48] Zhang WD, Wang YD, Li CY (2022b) Underwater image enhancement by attenuated color channel correction and detail preserved contrast enhancement. IEEE J Ocean Eng 47(3):718–735

[49] Zhang W, Zhuang P, Sun HH, et al. . Underwater image enhancement via minimal color loss and locally adaptive contrast enhancement. IEEE Trans Image Process. 2022, 31: 3997-4010

[50] Zhao W, Rong S, Li T, et al. . Enhancing underwater imagery via latent low-rank decomposition and image fusion. IEEE J Oceanic Eng. 2023, 48(1): 147-159

[51] Zhou JC, Li BS, Zhang DH, Yuan JY, Zhang WS, Cai ZC et al (2023a) UGIF-Net: an efficient fully guided information flow network for underwater image enhancement. IEEE Trans Geosci Remote Sens 61:4206117. https://doi.org/10.1109/TGRS.2023.3293912

[52] Zhou JC, Liu Q, Jiang QP, Ren WQ, Lam KM, Zhang WS (2023b) Underwater camera: improving visual perception via adaptive dark pixel prior and color correction. Int J Comput Vis. https://doi.org/10.1007/s11263-023-01853-3

[53] Zhou JC, Wang YY, Li CY, Zhang WS (2023c) Multicolor light attenuation modeling for underwater image restoration. IEEE J Ocean Eng 48(4):1–16

[54] Zhou J, Wang Y, Zhang W, et al. . Underwater image restoration via feature priors to estimate background light and optimized transmission map. Opt Express. 2021, 29(18): 28228-28245

[55] Zhou JC, Zhang DH, Ren WQ, Zhang WS (2022) Auto color correction of underwater images utilizing depth information. IEEE Geosci Remote Sens Lett 19:1504805. https://doi.org/10.1109/LGRS.2022.3170702

[56] Zhuang P, Li C, Wu J. Bayesian retinex underwater image enhancement. Eng Appl Artif Intell. 2021, 101: 104171

[57] Zhuang PX, Wu JM, Porikli F, Li CY (2022) Underwater image enhancement with hyper-laplacian reflectance priors. IEEE Trans Image Proc 31:5442–5455

Funding

Taishan Scholar Project