Infrared and visible image fusion algorithm based on Retinex-enhanced multiscale decomposition

Abstract

Aiming at the problems of poor contrast of fusion results, blurring of target margins and loss of background detail information under low illumination conditions of traditional infrared and visible image fusion algorithms, an infrared and visible image fusion algorithm based on multi-scale decomposition with Retinex enhancement was proposed. Firstly, the single-scale Retinex(SSR) algorithm information enhancement process was performed on the weak visible image using Retinex. Secondly, the source image was multi-scale decomposed using cross bilateral filtering to successively obtain the image information of the base layer image and the detail layer, and the fusion method combining the absolute value maximization strategy and guided filtering was used for the base layer image, and the fusion method of constructing weight map and significant map was used for the detail layer image. Finally, the processed base layer image and detail layer image were weighted to obtain the fused image. From the subjective analysis, the proposed method could effectively extract and fuse the important information in the source image, and obtain the image with high fusion quality and natural and clear visual effect. From the objective evaluation, the average accuracy of the proposed method on AG, SF, CE, and FMI was optimal when compared quantitatively with various fusion results.

Keywords

image fusion / infrared and visible images / Retinex algorithm / multiscale decomposition / guided filtering

References

[1] QI H S, RONG C Z, XIAO L M, et al. Infrared and visible image fusion algorithm based on dual-tree complex wavelet transform and guided filtering. Communications Technology, 2019, 52(2): 330-336.

[2] HUO X, ZOU Y, CHEN Y, et al. Dual-scale decomposition and saliency analysis based infrared and visible image fusion. Journal of Image and Graphics, 2021, 26(12): 2813-2825.

[3] XU S P, LIN Z Y, ZHANG G Z, et al. Low-illumination image enhancement algorithm using a hybrid implementation strategy of deep learning and image fusion. Journal of Electronics, 2021, 49(1): 72.

[4] LI Z, HU H M, ZHANG W, et al. Spectrum characteristics preserved visible and near-infrared image fusion algorithm. IEEE Transactions on Multimedia, 2021, 23: 306-319.

[5] ZHAO F, ZHAO W, YAO L, et al. Self-supervised feature adaption for infrared and visible image fusion. Information Fusion, 2021, 76: 189-203.

[6] GAO X Q, LIU G, XIAO G, et al. Infrared and visible image fusion algorithm based on FPDE. Journal of Automation, 2020, 46(4): 186-194.

[7] BAVIRISETTI D P, DHULI R. Fusion of infrared and visible sensor images based on anisotropic diffusion and karhunen-loeve transform. IEEE Sensors Journal, 2016, 16(1): 203-209.

[8] SHREYAMSHA KUMAR B K. Image fusion based on pixel significance using cross bilateral filter. Signal, Image and Video Processing, 2015, 9(5): 1193-1204.

[9] MA Q, ZHU B, ZHANG H W. A dual-band image fusion method bbased on VGG network. Laser and Infrared, 2019, 49(11): 1374-1380.

[10] PRABHAKAR K R, SRIKAR V S, BABU R V. Deep fuse: A deep unsupervised approach for exposure fusion with extreme exposure image pairs//IEEE International Conference on Computer Vision, October 22-29, 2017, Venice, Italy. New York: IEEE, 2017: 4724-4732

[11] LIU Y, CHEN X, CHENG J, et al. Infrared and visible image fusion with convolutional neural networks. International Journal of Wavelets, Multiresolution and Information Processing, 2018, 16(3): 1850018.

[12] LIU G, LIN Z, YU Y. Robust subspace segmentation by low-rank representation//27th International Conference on Machine Learning, Haifa, Israel, New York: ACM, 2010: 663-670.

[13] LIU G, YAN S. Latent low-rank representation for subspace segmentation and feature extraction//2011 International Conference on Computer Vision, November 6-13, 2011, Barcelona, Spain. New York: IEEE, 2011: 1615-1622.

[14] LI H, WU X J. Infrared and visible image fusion using latent low-rank representation. arXiv Preprint, 2018, arXiv:1804.08992.

[15] LI J, LI S J, DUAN X H, et al. Infrared image enhancement based on retinex and probability nonlocal means filtering. Acta Photonica Sinica, 2020, 49(4): 410003

[16] KONG L J, ZHANG M M. Fusion algorithm of low visible light and infrared image based on retinex. Packaging Engineering, 2020, 41(19): 237-244.

[17] CUI Z Y, ZHANG S H. Image enhancement algorithm based on multi-scale retinex and bilateral filter. Laser Journal, 2015, 36(4): 90-93.

[18] QU H C, WANG Y P, GAO J K, et al. Mode adaptive infrared and visible image fusion. Infrared Technology, 2022, 44(3): 268-276.

[19] LIU F. Research of image dehazing and hardware implementation. Hefei: University of Science and Technology of China, 2014.

[20] CHEN G Q. Research on multi-sensor image fusion technology based on multi-scale analysis. Jilin: Jilin University, 2015.

[21] ZHANG X, YE P, XIAO G. VIFB: a visible and infrared image fusion benchmark//IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, June 13-19, 2020, Seattle, WA, USA. New York: IEEE, 2020: 468-478.