MCAL-Net: multispace color compensation and adaptive luminance transfer network for underwater images

Abstract

Vision serves as a crucial information source for underwater observation and operations; however, the quality of underwater imaging is often compromised by remarkable color distortion and detail loss, which are further exacerbated under nonuniform lighting conditions. The existing traditional nonlearning solutions often struggle to adapt to diverse underwater degradation, while purely data-driven learning strategies are often limited by scarce and low-quality samples, making it difficult to achieve satisfactory results. In contrast to existing joint learning frameworks, we propose a unified yet decoupled framework for effectively addressing the challenges of color correction and illumination enhancement in underwater images. Our proposed method employs distinct prediction and learning strategies to tackle these two key issues individually, thereby overcoming the limitations associated with the reference of learning samples that neglect lighting conditions. Consequently, the proposed approach yields enhanced overall visual effects for underwater image enhancement. Comparative experiments and ablation experiments on publicly available datasets have validated the effectiveness of the proposed self-attention-driven adaptive luminance transfer and multiple color space feature encoding. The source code and pretrained models are available on the project home page: https://github.com/OUCVisionGroup/MCAL-Net.

Keywords

Underwater image enhancement / Deep learning / Color compensation / Luminance transfer / Uneven illumination enhancement

References

[1] AdityaS, YangY, BaralC, AloimonosY, FermüllerC. Image understanding using vision and reasoning through scene description graph. Comput Vis Image Underst, 2018, 173: 33-45

[2] Akkaynak D, Treibitz T (2019) Sea-Thru: a method for removing water from underwater images. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 1682–1691

[3] AncutiCO, AncutiC, De VleeschouwerC, BekaertP. Color balance and fusion for underwater image enhancement. IEEE Trans Image Proc, 2017, 27: 379-393

[4] AnwarS, LiCY. Diving deeper into underwater image enhancement: a survey. Signal Proc-Image Commun, 2020, 89115978

[5] BermanD, LevyD, AvidanS, TreibitzT. Underwater single image color restoration using haze-lines and a new quantitative dataset. IEEE Trans Pattern Anal Mach Intell, 2021, 43: 2822-2837

[6] BorjiA. Pros and cons of GAN evaluation measures. Comput Vis Image Underst, 2019, 179: 41-65

[7] Cao K, Peng YT, Cosman PC (2018) Underwater image restoration using deep networks to estimate background light and scene depth. In: 2018 IEEE Southwest Symposium on Image Analysis and Interpretation. IEEE, pp 1–4

[8] CongR, YangW, ZhangW, LiC, GuoCL, HuangQ, et al.. PUGAN: physical model-guided underwater image enhancement using GAN with dual-discriminators. IEEE Trans Image Proc, 2023, 32: 4472-4485

[9] Cong X, Zhao Y, Gui J, Hou J, Tao D (2024) A comprehensive survey on underwater image enhancement based on deep learning. Preprint at arXiv:2405.19684

[10] Cui Z, Qi GJ, Gu L, You S, Zhang Z, Harada T (2021) Multitask AET with orthogonal tangent regularity for dark object detection. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, pp 2553–2562

[11] Drews P, do Nascimento E, Moraes F, Botelho S, Campos M (2013) Transmission estimation in underwater single images. In: 2013 IEEE International Conference on Computer Vision Workshops (ICCVW). IEEE, pp 825–830

[12] Fu XY, Fan ZW, Ling M, Huang Y, Ding XH (2017) Two-step approach for single underwater image enhancement. In: 2017 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS). IEEE, pp 789–794

[13] Fu XY, Zhuang PX, Huang Y, Liao YH, Zhang XP, Ding X (2014) A retinex-based enhancing approach for single underwater image. In: 2014 IEEE International Conference on Image Processing (ICIP). IEEE, pp 4572–4576

[14] GaldranA, PardoD, PicónA, Alvarez-GilaA. Automatic red-channel underwater image restoration. J Vis Commun Image Represent, 2015, 26: 132-145

[15] GaoSB, ZhangM, ZhaoQ, ZhangXS, LiYJ. Underwater image enhancement using adaptive retinal mechanisms. IEEE Trans Image Proc, 2019, 28: 5580-5595

[16] GraciasN, Santos-VictorJ. Underwater video mosaics as visual navigation maps. Comput Vis Image Underst, 2000, 79: 66-91

[17] GuZ, LiuX, HuZ, WangG, ZhengB, WatsonJ, et al.. Underwater computational imaging: a survey. Intell Mar Technol Syst, 2023, 12

[18] Guo CL, Li CY, Guo JC, Loy CC, Hou JH, Kwong S et al (2020) Zero-reference deep curve estimation for low-light image enhancement. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 1780–1789

[19] HouGJ, LiJM, WangGD, YangH, HuangBX, PanZK. A novel dark channel prior guided variational framework for underwater image restoration. J Vis Commun Image Represent, 2020, 66102732

[20] HouGJ, LiN, ZhuangPX, LiKQ, SunHH, LiCY. Non-uniform illumination underwater image restoration via illumination channel sparsity prior. IEEE Trans Circuits Syst Video Technol, 2024, 34: 799-814

[21] Huang DM, Wang Y, Song W, Sequeira J, Mavromatis S (2018) Shallow-water image enhancement using relative global histogram stretching based on adaptive parameter acquisition. In: 24th International Conference MultiMedia Modeling. Springer, pp 453–465

[22] HuangZX, LiJJ, HuaZ, FanLW. Underwater image enhancement via adaptive group attention-based multiscale cascade transformer. IEEE Trans Instrum Meas, 2022, 715015618

[23] IslamMJ, XiaYY, SattarJ. Fast underwater image enhancement for improved visual perception. IEEE Robot Autom Lett, 2020, 5: 3227-3234

[24] LeiXZ, FeiZX, ZhouWJ, ZhouHY, FeiMR. Low-light image enhancement based on cell vibration energy model and lightness difference. Comput Vis Image Underst, 2024, 247104079

[25] LiCY, AnwarS, HouJH, CongRM, GuoCL, RenWQ. Underwater image enhancement via medium transmission-guided multi-color space embedding. IEEE Trans Image Proc, 2021, 30: 4985-5000

[26] LiCY, AnwarS, PorikliF. Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recognit, 2020, 98107038

[27] LiCY, GuoCL, RenWQ, CongRM, HouJH, KwongS, et al.. An underwater image enhancement benchmark dataset and beyond. IEEE Trans Image Proc, 2019, 29: 4376-4389

[28] LiCY, GuoJC, CongRM, PangYW, WangB. Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior. IEEE Trans Image Proc, 2016, 25: 5664-5677

[29] LiJ, SkinnerKA, EusticeRM, Johnson-RobersonM. WaterGAN: unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robot Autom Lett, 2017, 3: 387-394

[30] LiK, FanH, QiQ, YanC, SunK, WuQMJ. TCTL-Net: template-free color transfer learning for self-attention driven underwater image enhancement. IEEE Trans Circuits Syst Video Technol, 2024, 34: 4682-4697

[31] LiK, WuL, QiQ, LiuW, GaoX, ZhouL, et al.. Beyond single reference for training: underwater image enhancement via comparative learning. IEEE Trans Circuits Syst Video Technol, 2023, 33: 2561-2576

[32] LiY, MiZ, WangY, JiangS, FuX. TAFormer: a transmission-aware transformer for underwater image enhancement. IEEE Trans Circuits Syst Video Technol, 2025, 35: 601-616

[33] LiuR, FanX, ZhuM, HouM, LuoZ. Real-world underwater enhancement: challenges, benchmarks, and solutions under natural light. IEEE Trans Circuits Syst Video Technol, 2020, 30: 4861-4875

[34] Liu Z, Lin YT, Cao Y, Hu H, Wei YX, Zhang Z et al (2021) Swin transformer: hierarchical vision transformer using shifted windows. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, pp 9992–10002

[35] NarvekarND, KaramLJ. A no-reference image blur metric based on the cumulative probability of blur detection (CPBD). IEEE Trans Image Proc, 2011, 20: 2678-2683

[36] PanettaK, GaoC, AgaianS. Human-visual-system-inspired underwater image quality measures. IEEE J Ocean Eng, 2016, 41: 541-551

[37] PengLT, ZhuCL, BianLH. U-shape transformer for underwater image enhancement. IEEE Trans Image Proc, 2023, 32: 3066-3079

[38] PengYT, CaoK, CosmanPC. Generalization of the dark channel prior for single image restoration. IEEE Trans Image Proc, 2018, 27: 2856-2868

[39] QiQ, LiK, ZhengH, GaoX, HouG, SunK. SGUIE-Net: semantic attention guided underwater image enhancement with multi-scale perception. IEEE Trans Image Proc, 2022, 31: 6816-6830

[40] RaoY, LiuW, LiK, FanH, WangS, DongJ. Deep color compensation for generalized underwater image enhancement. IEEE Trans Circuits Syst Video Technol, 2024, 34: 2577-2590

[41] ReinhardE, AdhikhminM, GoochB, ShirleyP. Color transfer between images. IEEE Comput Graph Appl, 2001, 21: 34-41

[42] Schonberger JL, Frahm JM (2016) Structure-from-motion revisited. In: IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 4104–4113

[43] Simonyan K (2014) Very deep convolutional networks for large-scale image recognition. Preprint at arXiv:1409.1556

[44] Song W, Wang Y, Huang DM, Tjondronegoro D (2018) A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration. In: 19th Paciffc-Rim Conference on Multimedia. Springer, pp 678–688

[45] SunX, LiuLP, LiQ, DongJY, LimaE, YinRY. Deep pixel-to-pixel network for underwater image enhancement and restoration. IET Image Proc, 2019, 13: 469-474

[46] Wang H, Frery AC, Li MJ, Ren P (2023a) Underwater image enhancement via histogram similarity-oriented color compensation complemented by multiple attribute adjustment. Intell Mar Technol Syst 1:12

[47] Wang ML, Li JY, Zhang CS (2023b) Low-light image enhancement by deep learning network for improved illumination map. Comput Vis Image Underst 232:103681

[48] YangH, TianF, QiQ, WuQMJ, LiKQ. Underwater image enhancement with latent consistency learning-based color transfer. IET Image Proc, 2022, 16: 1594-1612

[49] YangM, SowmyaA. An underwater color image quality evaluation metric. IEEE Trans Image Proc, 2015, 24: 6062-6071

[50] YangN, ZhongQH, LiK, CongRM, ZhaoY, KwongS. A reference-free underwater image quality assessment metric in frequency domain. Signal Proc-Image Commun, 2021, 94116218

[51] ZhangWD, ZhouL, ZhuangPX, LiGH, PanXP, ZhaoWY, et al.. Underwater image enhancement via weighted wavelet visual perception fusion. IEEE Trans Circuits Syst Video Technol, 2024, 34: 2469-2483

[52] Zhao C, Cai WL, Dong CY, Hu CW (2024) Wavelet-based fourier information interaction with frequency diffusion adjustment for underwater image restoration. In: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 8281–8291

Funding

National Natural Science Foundation of China(62371431; 61906177)

Key Laboratory of Dynamic Cognitive System of Electromagnetic Spectrum Space(23-1-3-hygg-20-hy)

Fundamental Research Funds for the Central Universities(202262004)

RIGHTS & PERMISSIONS

The Author(s)