Turbidity-adaptive underwater image enhancement method using image fusion

Abstract

Clear, correct imaging is a prerequisite for underwater operations. In real freshwater environment including rivers and lakes, the water bodies are usually turbid and dynamic, which brings extra troubles to quality of imaging due to color deviation and suspended particulate. Most of the existing underwater imaging methods focus on relatively clear underwater environment, it is uncertain that if those methods can work well in turbid and dynamic underwater environments. In this paper, we propose a turbidity-adaptive underwater image enhancement method. To deal with attenuation and scattering of varying degree, the turbidity is detected by the histogram of images. Based on the detection result, different image enhancement strategies are designed to deal with the problem of color deviation and blurring. The proposed method is verified by an underwater image dataset captured in real underwater environment. The result is evaluated by image metrics including structure similarity index measure, underwater color image quality evaluation metric, and speeded-up robust features. Test results exhibit that the method can correct the color deviation and improve the quality of underwater images.

Graphical abstract

Keywords

turbidity / underwater image enhancement / image fusion / underwater robots / visibility

References

[1] Boudhane M, Nsiri B. Underwater image processing method for fish localization and detection in submarine environment. Journal of Visual Communication and Image Representation, 2016, 39: 226–238

[2] Lin Y H, Yu C M, Wu C Y. Towards the design and implementation of an image-based navigation system of an autonomous underwater vehicle combining a color recognition technique and a fuzzy logic controller. Sensors, 2021, 21(12): 4053

[3] Liu J G, Wang Y C, Li B, Ma S G. Current research, key performances and future development of search and rescue robots. Frontiers of Mechanical Engineering, 2007, 2(4): 404–416

[4] Li T C, Wang J L, Yao K N. Visibility enhancement of underwater images based on active polarized illumination and average filtering technology. Alexandria Engineering Journal, 2022, 61(1): 701–708

[5] Zhuang P X, Li C Y, Wu J M. Bayesian retinex underwater image enhancement. Engineering Applications of Artificial Intelligence, 2021, 101: 104171

[6] Liang Z, Wang Y F, Ding X Y, Mi Z T, Fu X P. Single underwater image enhancement by attenuation map guided color correction and detail preserved dehazing. Neurocomputing, 2021, 425: 160–172

[7] Li C Y, Anwar S, Hou J H, Cong R M, Guo C L, Ren W Q. Underwater image enhancement via medium transmission-guided multi-color space embedding. IEEE Transactions on Image Processing, 2021, 30: 4985–5000

[8] AncutiC, Ancuti C O, HaberT, BekaertP. Enhancing underwater images and videos by fusion. In: Proceedings of 2012 IEEE CVPR Conference. 2012, 81–88

[9] Ancuti C O, Ancuti C, De Vleeschouwer C, Bekaert P. Color balance and fusion for underwater image enhancement. IEEE Transactions on Image Processing, 2018, 27(1): 379–393

[10] Wang Y, Zhang J, Cao Y, Wang Z F. A deep CNN method for underwater image enhancement. In: Proceedings of 2017 IEEE International Conference on Image Processing (ICIP). IEEE, 2017, 1382–1386

[11] Guo Y C, Li H Y, Zhuang P X. Underwater image enhancement using a multiscale dense generative adversarial network. IEEE Journal of Oceanic Engineering, 2020, 45(3): 862–870

[12] Liu R S, Fan X, Zhu M, Hou M J, Luo Z X. Real-world underwater enhancement: challenges, benchmarks, and solutions under natural light. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(12): 4861–4875

[13] Fu X Y, Cao X Y. Underwater image enhancement with global–local networks and compressed-histogram equalization. Signal Processing: Image Communication, 2020, 86: 115892

[14] Yang M, Hu K, Du Y X, Wei Z Q, Sheng Z B, Hu J T. Underwater image enhancement based on conditional generative adversarial network. Signal Processing: Image Communication, 2020, 81: 115723

[15] Jiang Q, Zhang Y F, Bao F X, Zhao X Y, Zhang C M, Liu P D. Two-step domain adaptation for underwater image enhancement. Pattern Recognition, 2022, 122: 108324

[16] Li C Y, Guo C L, Ren W Q, Cong R M, Hou J H, Kwong S, Tao D C. An underwater image enhancement benchmark dataset and beyond. IEEE Transactions on Image Processing, 2020, 29: 4376–4389

[17] Liu Y Q, Chen Y Y, Fang X M. A review of turbidity detection based on computer vision. IEEE Access: Practical Innovations, Open Solutions, 2018, 6: 60586–60604

[18] Chen S S, Han L S, Chen X Z, Li D, Sun L, Li Y. Estimating wide range total suspended solids concentrations from MODIS 250-m imageries: an improved method. ISPRS Journal of Photogrammetry and Remote Sensing, 2015, 99: 58–69

[19] ZhangX H. Water quality turbidity detection based on image recognition system design and implementation. In: Proceedings of 2016 the First International Conference on Real Time Intelligent Systems. 2018, 613: 63–70

[20] O’Byrne M, Schoefs F, Pakrashi V, Ghosh B. An underwater lighting and turbidity image repository for analysing the performance of image-based non-destructive techniques. Structure and Infrastructure Engineering, 2018, 14(1): 104–123

[21] Xie K, Pan W, Xu S X. An underwater image enhancement algorithm for environment recognition and robot navigation. Robotics, 2018, 7(1): 14

[22] Hu H F, Zhang Y B, Li X B, Lin Y, Cheng Z Z, Liu T G. Polarimetric underwater image recovery via deep learning. Optics and Lasers in Engineering, 2020, 133: 106152

[23] Wang Y B, Cao J, Rizvi S, Hao Q, Fang Y M. Underwater image restoration based on adaptive color compensation and dual transmission estimation. IEEE Access: Practical Innovations, Open Solutions, 2020, 8: 207834–207843

[24] Zhang W H, Li G, Ying Z Q. A new underwater image enhancing method via color correction and illumination adjustment. 2017 IEEE Visual Communications and Image Processing, 2017, 1–4

[25] Jaffe J S. Computer modeling and the design of optimal underwater imaging systems. IEEE Journal of Oceanic Engineering, 1990, 15(2): 101–111

[26] Amer K O, Elbouz M, Alfalou A, Brosseau C, Hajjami J. Enhancing underwater optical imaging by using a low-pass polarization filter. Optics Express, 2019, 27(2): 621–643

[27] Li Y J, Lu H M, Li K C, Kim H, Serikawa S. Non-uniform de-scattering and de-blurring of underwater images. Mobile Networks and Applications, 2018, 23(2): 352–362

[28] Peng Y T, Cosman P C. Underwater image restoration based on image blurriness and light absorption. IEEE Transactions on Image Processing, 2017, 26(4): 1579–1594

[29] Huang J, Liu G X. Multi-color space threshold segmentation and self-learning k-NN algorithm for surge test EUT status identification. Frontiers of Mechanical Engineering, 2016, 11(3): 311–315

[30] Zhu D Q, Liu Z Q, Zhang Y M. Underwater image enhancement based on colour correction and fusion. IET Image Processing, 2021, 15(11): 2591–2603

[31] Yang X, Li H, Chen R. Underwater image enhancement with image colorfulness measure. Signal Processing: Image Communication, 2021, 95: 116225

[32] Wang Y Q, Yu X N, An D, Wei Y G. Underwater image enhancement and marine snow removal for fishery based on integrated dual-channel neural network. Computers and Electronics in Agriculture, 2021, 186: 106182

[33] Galdran A, Pardo D, Picón A, Alvarez-Gila A. Automatic red-channel underwater image restoration. Journal of Visual Communication and Image Representation, 2015, 26: 132–145

[34] He K M, Sun J, Tang X O. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341–2353

[35] YuH F, Li X B, LouQ, LeiC B, LiuZ X. Underwater image enhancement based on DCP and depth transmission map. Multimedia Tools and Applications, 2020, 79(27–28): 20373–20390

[36] Zhang M H, Peng J H. Underwater image restoration based on a new underwater image formation model. IEEE Access: Practical Innovations, Open Solutions, 2018, 6: 58634–58644

[37] He K M, Sun J, Tang X O. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(6): 1397–1409

[38] Wei L S, Jiao Z X. Research and application of visual location technology for solder paste printing based on machine vision. Frontiers of Mechanical Engineering, 2009, 4(2): 184–191

[39] Lin S, Chi K C, Li W T, Tang Y D. Underwater optical image enhancement based on dominant feature image fusion. Acta Photonica Sinica, 2020, 49(3): 310003

[40] KimY, KohY J, LeeC. KimS, KimC S. Dark image enhancement based onpairwise target contrast and multi-scale detail boosting. In: Proceedings of 2015 IEEE International Conference on Image Processing. 2015, 1404–1408

[41] Shen D H, Zareapoor M, Yang J. Multimodal image fusion based on point-wise mutual information. Image and Vision Computing, 2021, 105: 104047

[42] Bay H, Tuytelaars T, Goo L V. Surf: speeded up robust features. In: Proceedings of 2006 the 9th European Conference on Computer Vision. 2006, 3951, 404–417

[43] Huang Y F, Liu M Y, Yuan F. Color correction and restoration based on multi-scale recursive network for underwater optical image. Signal Processing: Image Communication, 2021, 93: 116174

[44] Mi Z T, Li Y Y, Wang Y F, Fu X P. Multi-purpose oriented real-world underwater image enhancement. IEEE Access: Practical Innovations, Open Solutions, 2020, 8: 112957–112968

[45] Chang Y K, Jung C, Ke P, Song H, Hwang J. Automatic contrast-limited adaptive histogram equalization with dual gamma correction. IEEE Access: Practical Innovations, Open Solutions, 2018, 6: 11782–11792

[46] Buchsbaum G. A spatial processor model for object colour perception. Journal of the Franklin Institute, 1980, 310(1): 1–26

[47] Yang M, Sowmya A. An underwater color image quality evaluation metric. IEEE Transactions on Image Processing, 2015, 24(12): 6062–6071

Nomenclature

A	Ambient light
Ac	Ambient light of color channel c
B	Blurred image
B_i (i = 0,1,2,3)	B_0 is the image without filtering. B_1, B_2 and B_3 are blurred images filtered by G_1, G_2, and G_3, respectively
B_{\mathrm{\infty }}(x{)}^c	Value of infinite pixel x of color channels c of ambient light image
c	Color channel (R, G, B) of image
c_{\text{1}}, c_{\text{2}}, c_{\text{3}}	c_{\text{1}}, c_{\text{2}}, and c_{\text{3}} are the weight coefficients set as c_{\text{1}}=0.4680, c_{\text{2}}=0.2745, and c_{\text{3}}=0.2576
C_{i1}, C_{i2}	Two other channels in addition to attenuation channel
\overline{C_{i1}}, \overline{C_{i2}}	Average of C_{i1} and C_{i2}, respectively
\overline{C_j}	Mean of attenuation channel
C_{jc}	Compensation channel
C_{\mathrm{v}\mathrm{a}\mathrm{r}}	Standard deviation of chroma
d(x)	Object distance of pixel x
Deviatio{n}_c	Deviation of color channel c
Deviatio{n}_{\text{level}}	Color deviation level
Deviatio{n}_{max}	Maximum deviation of image
E_{\text{UCIQE}}	Value of UCIQE of image
F(x)	Fusion image
G	Gaussian differential filter
Gi (i = 1,2,3)	Gaussian differential filter with different filter ratio
I	Image to be sharpened
I(x)	Pixel value at x of image I
I^c(x)	Pixel value at x of color channel c of degraded image
I_k\left(x\right)	Number k input image
I_{\text{sharp}}	Sharpened image
I_{\mathrm{v}\mathrm{a}\mathrm{r}}	Variance of channels
J	Undegraded image
J^c(x)	Pixel value at xof color channel c of original image
J_{\text{dark}}	Dark channel images
K	Amount of input images
L, A, B	Channels of Lab color space
L1, L2	Thresholds for color deviation level detecting, and they are set as 40 and 60, respectively
\overline{L}, \overline{A}, \overline{B}	Means of channels of Lab color space
L_{\text{con}}	Contrast of luminance
m	Constant parameter for compensation, it is set as 0.18
Mea{n}_c	Mean of color channel c
Mea{n}_{\text{sum}}	Mean of all color channels of image
n	Constant parameter for compensation, it is set as 0.15
N	Linear normalization operator, also named histogram stretching in the literature
S	Sharpened image in normalized unsharp masking method
S_{\text{aver}}	Average of saturation
T\left(x\right)	Pixel value at x of transmission map
T1, T2	Variance thresholds for turbidity level detecting, and they are set as 9 and 28, respectively
T^c(x)	Pixel value at x of color channel c of transmission map
Turbidit{y}_{\text{level}}	Turbidity level of image
w	Compensation level
W_k	Normalized weight map
WL	Laplacian weight map
WS	Saliency weight map
WT	Saturation weight map
x	Localization of pixel
y	Pixel localization of region \mathrm{\Omega }(x)
\alpha	Attenuation coefficient of water
\beta	Special parament in normalized unsharp masking method
\eta	Parameter to adjust the dehaze level
δ	A small regularization term that ensures that each input contributes to the output, and it is set as 0.1
\mathrm{\Omega }(x)	Local region centered at pixel x
\sigma	Filter ratio

Appendix

Table A1 Underwater image evaluation based on UCIQE

Image	Underwater image evaluation
	Origin	Gray world	CLAHE	UDCP	Our method
1	0.29482	0.39397	0.34491	0.30368	0.49016
2	0.34675	0.39382	0.38193	0.36540	0.50846
3	0.33249	0.34176	0.35667	0.39435	0.52372
4	0.36472	0.37469	0.40091	0.41527	0.53806
5	0.33711	0.43044	0.38948	0.35630	0.51864
6	0.39073	0.41309	0.41042	0.41415	0.51215
7	0.30806	0.30848	0.31754	0.33841	0.51689
8	0.29159	0.29569	0.29350	0.34770	0.48641
9	0.30773	0.32783	0.31055	0.33060	0.48963
10	0.30766	0.30950	0.31975	0.37578	0.51486
11	0.33487	0.33830	0.36050	0.39308	0.46614
12	0.29352	0.29745	0.29799	0.35100	0.50293
13	0.28471	0.28601	0.29181	0.34124	0.47248
14	0.34364	0.36475	0.36003	0.37045	0.45168
15	0.31619	0.33560	0.3320	0.34455	0.46578
16	0.53360	0.51917	0.53510	0.54501	0.55259
17	0.50480	0.48477	0.50146	0.51459	0.52190
18	0.47984	0.43658	0.47940	0.47915	0.49416
19	0.49808	0.47624	0.49343	0.50811	0.50991

Table A2 Average of UCIQE of Table A1

Method	Image average evaluation	Image amplification/%
Origin	0.361627	100.0
Gray world	0.375165	103.7
CLAHE	0.377757	104.4
UDCP	0.394148	108.9
Our method	0.501924	138.7

Fig.A1 Transmission maps of guide filter based on different ratios.

Fig.A2 Intermediate results of images 1–10.

Fig.A3 Intermediate results of images 11–19.

Acknowledgements

This work was supported by the Guangdong Innovative and Entrepreneurial Research Team Program, China (Grant No. 2019ZT08Z780), in part by the Dongguan Introduction Program of Leading Innovative and Entrepreneurial Talents, China, in part by the National Key R&D Program of China (Grant No. 2017YFC0821200), in part by the Guangdong Basic and Applied Basic Research Foundation, China (Grant No. 2021A1515011717), and in part by the Space Trusted Computing and Electronic Information Technology Laboratory of BICE, China (Grant No. OBCandETL-2020-06).

RIGHTS & PERMISSIONS

2022 Higher Education Press