Nighttime image dehazing using color cast removal and dual path multi-scale fusion strategy

Abstract

Nighttime image dehazing aims to remove the effect of haze on the images captured in nighttime, which however, raises new challenges such as severe color distortion, more complex lighting conditions, and lower contrast. Instead of estimating the transmission map and atmospheric light that are difficult to be accurately acquired in nighttime, we propose a nighttime image dehazing method composed of a color cast removal and a dual path multi-scale fusion algorithm. We first propose a human visual system (HVS) inspired color correction model, which is effective for removing the color deviation on nighttime hazy images. Then, we propose to use dual path strategy that includes an underexposure and a contrast enhancement path for multi-scale fusion, where the weight maps are achieved by selecting appropriate exposed areas under Gaussian pyramids. Extensive experiments demonstrate that the visual effect of the hazy nighttime images in real-world datasets can be significantly improved by our method regarding contrast, color fidelity, and visibility. In addition, our method outperforms the state-of-the-art methods qualitatively and quantitatively.

Graphical abstract

Keywords

nighttime image dehazing / color cast removal / dual path / multi-scale fusion

References

[1] Gao Y , Su Y , Li Q M , Li H Y , Li J . Single image dehazing via self-constructing image fusion. Signal Processing, 2020, 167 : 107284–

[2] Li Y, You S D, Brown M S, Tan R T. Haze visibility enhancement: a survey and quantitative benchmarking. Computer Vision and Image Understanding, 2017, 165: 1−16

[3] Dai C G , Lin M X , Wu X J , Zhang D . Single hazy image restoration using robust atmospheric scattering model. Signal Processing, 2020, 166 : 107257–

[4] Lin Y T , Wu Y , Yan C G , Xu M L , Yang Y . Unsupervised person re-identification via cross-camera similarity exploration. IEEE Transactions on Image Processing, 2020, 29 : 5481– 5490

[5] Liu Q , Gao X B , He L H , Lu W . Haze removal for a single visible remote sensing image. Signal Processing, 2017, 137 : 33– 43

[6] Zhao D , Xu L , Yan Y H , Chen J , Duan L Y . Multi-scale optimal fusion model for single image dehazing. Signal Processing: Image Communication, 2019, 74 : 253– 265

[7] Li C Y , Guo J C , Cong R M , Pang Y W , Wang B . Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior. IEEE Transactions on Image Processing, 2016, 25( 12): 5664– 5677

[8] Li C Y , Guo C L , Guo J C , Han P , Fu H Z , Cong R M . PDR-Net: perception-inspired single image dehazing network with refinement. IEEE Transactions on Multimedia, 2020, 22( 3): 704– 716

[9] Yuan F , Huang H . Image haze removal via reference retrieval and scene prior. IEEE Transactions on Image Processing, 2018, 27( 9): 4395– 4409

[10] Liu P J , Horng S J , Lin J S , Li T R . Contrast in haze removal: configurable contrast enhancement model based on dark channel prior. IEEE Transactions on Image Processing, 2019, 28( 5): 2212– 2227

[11] He K M , Sun J , Tang X . Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33( 12): 2341– 2353

[12] Zhu Q S , Mai J M , Shao L . A fast single image haze removal algorithm using color attenuation prior. IEEE Transactions on Image Processing, 2015, 24( 11): 3522– 3533

[13] Berman D, Treibitz T, Avidan S. Non-Local image dehazing. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2016, 1674−1682

[14] Santra S, Chanda B. Day/night unconstrained image dehazing. In: Proceedings of the 23rd International Conference on Pattern Recognition. 2016, 1406−1411

[15] Kim G, Kwon J. Robust pixel-wise dehazing algorithm based on advanced haze-relevant features. In: Proceedings of British Machine Vision Conference. 2017, 1−12

[16] Yang A P , Liu J , Ji Z , Pan Y W . Detail-preserving single nighttime image dehazing. Journal of Electronic Imaging, 2020, 29( 4): 043010–

[17] Finlayson G D, Trezzi E. Shades of gray and colour constancy. In: Proceedings of the 12th Color Imaging Conference: Color Science and Engineering Systems, Technologies, Applications. 2004, 37−41

[18] Van De Weijer J , Gevers T , Gijsenij A . Edge-based color constancy. IEEE Transactions on Image Processing, 2007, 16( 9): 2207– 2214

[19] Gao S , Yang K , Li C . Color constancy using double-opponency. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37( 10): 1973– 1985

[20] Ancuti C O , Ancuti C , Vleeschouwer C D , Bekaert P . Color balance and fusion for underwater image enhancement. IEEE Transactions on Image Processing, 2018, 27( 1): 379– 393

[21] Galdran A . Image dehazing by artificial multiple-exposure image fusion. Signal Processing, 2018, 149 : 135– 147

[22] Zhang X S , Gao S B , Li C Y , Li Y J . A retina inspired model for enhancing visibility of hazy images. Frontiers in Computational Neuroscience, 2015, 9 : 1– 13

[23] Zhang X , Gao S , Li R , Du X , Li C , Li Y . A retinal mechanism inspired color constancy model. IEEE Transactions on Image Processing, 2016, 25( 3): 1219– 1232

[24] Pei S C, Lee T Y. Nighttime haze removal using color transfer pre-processing and dark channel prior. In: Proceedings of the 19th IEEE International Conference on Image Processing. 2012, 957−960

[25] Zhang J, Cao Y, Wang Z. Nighttime haze removal based on a new imaging model. In: Proceedings of IEEE International Conference on Image Processing. 2014, 4557−4561

[26] Li Y, Tan R T, Brown M S. Nighttime haze removal with glow and multiple light colors. In: Proceedings of IEEE International Conference on Computer Vision. 2015, 226−234

[27] Zhang J, Cao Y, Fang S, Kang Y, Chen C W. Fast haze removal for nighttime image using maximum reflectance prior. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2017, 7016−7024

[28] Yang M M , Liu J C , Li Z G . Superpixel-based single nighttime image haze removal. IEEE Transactions on Multimedia, 2018, 20( 11): 3008– 3018

[29] Ancuti C, Ancuti C O, Vleeschouwer C D, Bovik A C. Night-time dehazing by fusion. In: Proceedings of IEEE International Conference on Image Processing. 2016, 2256−2260

[30] Ancuti C , Ancuti C O , Vleeschouwer C D , Bovik A C . Day and night-time dehazing by local airlight estimation. IEEE Transactions on Image Processing, 2020, 29 : 6264– 6275

[31] Liao Y, Su Z, Liang X, Qiu B. HDP-Net: haze density prediction network for nighttime dehazing. In: Hong R, Cheng W H, Yamasaki T, Wang M, Ngo C W, eds. Advances in Multimedia Information Processing. Springer, Cham, 2018, 469−480

[32] Kuanar S, Rao K R, Mahapatra D, Bilas M. Night time haze and glow removal using deep dilated convolutional network. 2019, arXiv preprint arXiv: 1902.00855

[33] Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z H, Shi W Z. Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2017, 4681−4690

[34] Engin D, Genc A, Ekenel H K. Cycle-Dehaze: enhanced CycleGAN for single image dehazing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2018, 938―946

[35] Zhang K H , Luo W H , Zhong Y R , Ma L , Liu W , Li H D . Adversarial spatio-temporal learning for video deblurring. IEEE Transactions on Image Processing, 2019, 28( 1): 291– 301

[36] Zhang K H, Luo W H, Zhong Y R, Ma L, Stenger B, Liu W, Li H D. Deblurring by realistic blurring. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2020, 2737−2746

[37] Reinhard E , Adhikhmin M , Gooch B , Shirley P . Color transfer between images. IEEE Computer Graphics and Applications, 2001, 21( 5): 34– 41

[38] Lee B B , Martin P R , Grünert U . Retinal connectivity and primate vision. Progress in Retinal and Eye Research, 2010, 29 : 622– 639

[39] Gao S B, Yang K F, Li C Y, Li Y J. A color constancy model with double-opponency mechanisms. In: Proceedings of IEEE International Conference on Computer Vision. 2013, 929−936

[40] Li Y N , Miao Q G , Liu R Y , Song J F , Quan Y N , Huang Y H . A multi-scale fusion scheme based on haze-relevant features for single image dehazing. Neurocomputing, 2018, 283 : 73– 86

[41] Zuiderveld K. Contrast Limited Adaptive Histogram Equalization. Academic Press Professional, Inc., 1994, 474−485

[42] Gijsenij A , Gevers T , Van De Weijer J . Computational color constancy: survey and experiments. IEEE Transactions on Image Processing, 2011, 20( 9): 2475– 2489

[43] Gijsenij A , Gevers T , Van De Weijer J . Improving color constancy by photometric edge weighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34( 5): 918– 929

[44] Yu T , Song K , Miao P , Yang G , Yang H , Chen C . Nighttime single image dehazing via pixel-wise alpha blending. IEEE Access, 2019, 7 : 114619– 114630

[45] Xu Y , Wen J , Fei L , Zhang Z . Review of video and image defogging algorithms and related studies on image restoration and enhancement. IEEE Access, 2016, 4 : 165–

Acknowledgements

This work was supported by Higher Education Scientific Research Project of Ningxia (NGY2017009).

RIGHTS & PERMISSIONS

2022 Higher Education Press