Research on a multi-dimensional image information fusion algorithm based on NSCT transform

Abstract

Traditional inspection cameras determine targets and detect defects by capturing images of their light intensity, but in complex environments, the accuracy of inspection may decrease. Information based on polarization of light can characterize various features of a material, such as the roughness, texture, and refractive index, thus improving classification and recognition of targets. This paper uses a method based on noise template threshold matching to denoise and preprocess polarized images. It also reports on design of an image fusion algorithm, based on NSCT transform, to fuse light intensity images and polarized images. The results show that the fused image improves both subjective and objective evaluation indicators, relative to the source image, and can better preserve edge information and help to improve the accuracy of target recognition. This study provides a reference for the comprehensive application of multi-dimensional optical information in power inspection.

Key words： Power inspection; Object detection; Polarization imaging; Image fusion; Image denoising

References

1 Tyo, J.S., Goldstein, D.L., Chenault, D.B., Shaw, J.A.: Review of passive imaging polarimetry for remote sensing applications. Appl. Opt. 45(22), 5453–5469 (2006)

2 Schechner, Y.Y., Narasimhan, S.G., Nayar, S.K.: Polarization-based vision through haze. Appl. Opt. 42(3), 511–525 (2003)

3 Liu, F., Cao, L., Shao, X., Han, P., Bin, X.: Polarimetric dehazing utilizing spatial frequency segregation of images. Appl. Opt. 54(27), 8116–8122 (2015)

4 Cao, N., Liu, W., Zhang, Y.: Quantitative study of improvements of the imaging contrast and imaging range by the polarization technique. Acta Physica Sinica 49(1), 61–66 (2000)

5 Terrier, P., Devlaminck, V., Charbois, J.M.: Segmentation of rough surfaces using a polarization imaging system. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 25(2), 423–430 (2008)

6 Anna, G., Goudail, F., Dolfi, D.: Optimal discrimination of multiple regions with an active polarimetric imager. Opt. Express 19(25), 25367–25378 (2011)

7 Du, W., Jia, W., Zhang, Z., Wang, C.: Optimization of the infrared Stokes imaging polarimeter. SPIE (2017) In: Applied Optics and Photonics China (AOPC2017), 2017, Beijing, China

8 Liu, L.: Research on multispectral image fusion and its evaluation methods. Dissertation for the Master Degree. Chengdu: University of Electronic Science and Technology of China (2012)

9 Wang, J.: Research and application of multispectral image fusion. Dissertation for the Master Degree. Wuhan: Wuhan Institute of Technology (2007)

10 Hu, G., Liu, Z., Xu, X., Gao, R.: Research and recent development of image fusion at pixel level. Overview of security standard in Internet of Vehicles 25(3), 650–655(2008).

11 Hong, R.: Research on multi-source image fusion algorithm and application. Dissertation for the Doctoral Degree. Hefei: University of Science and Technology of China (2008)

12 Li, H.: Research on multi-sensor image fusion algorithm. Dissertation for the Doctoral Degree. Xi’an: Northwestern Polytechnical University (2006)

13 Zhang, J., Fang, Y.: Algorithm and evaluation for polarization image fusion based on edge information. Opto-Electronic Eng. 34(11), 78–81 (2007)

14 Zhang, J., Fang, Y.: Novel image fusion algorithm for multiband polarimetric image based on visible light. Acta Opt. Sin. 28(6), 1067–1072 (2008)

15 Zeng, H., Gu, G., He, W., Yang, W.: An adaptive fusion algorithm for visible polarization images. Acta Photonica Sinica 40(1), 132–135 (2011)

16 Shen, X., Liu, J., Gao, M.: Polarizing image fusion algorithm based on wavelet-contourlet transform. Infrared Technol. 42(2), 182–189 (2020)

17 Jiang, Z., Han, Y., Xie, R., Ren, S.: Research on an infrared polarized image fusion algorithm based on NSST transform. J. Optoelectronics Laser 31(11), 1140–1148 (2020)

18 Shi, G., Tuo, H., Wang, F., Yuan, H., Jia, R.: An adaptive selective fusion method for infrared polarization based on DWT. Ship Electron. Eng. 41(04), 174–177 (2021)

19 Yang, W., Wang, X., Zhao, H., Zhang, G.: Research on an improved underwater polarization image fusion algorithm. J. Changchun Univ. Sci. Technol. (Natural Science Edition) 44(4), 43–49 (2021)

20 Wang, F., Zhang, Y., Wang, F., Zhu, D.: Photovoltaic cell electroluminescence polarization image fusion and defect detection. Electron. Meas. Technol. 45(19), 143–149 (2022)

21 Wang, C., Xu, S.: Underwater polarization image fusion method based on Retinex and wavelet transform. Appl. Laser 42(08), 116–122 (2022)

22 Chen, J., Chen, Y., Li, Y., Bai, X.: Fusion of infrared intensity and polarized images based on structure and decomposition. Infrared Technol. 45(3), 257–265 (2023)

23 Meng, J., Ren, W., Yu, R., Wu, D., Zhang, R., Xie, Y., Wang, J.: Contrast enhanced color polarization image fusion. Optik 284, 170935 (2023)

24 Gao, Y., Yu, J., Zhang, X.: Underwater evidence detection method based on polarization fusion image. Infrared Technol. 45(9), 962–968 (2023)

25 Yang, T., Wang, X.: Face image enhancement based on polarization image fusion. Laser J. 44(3), 148–152 (2023)

26 Li, Y., Zhang, P., Zeng, Y., Yang, J., Zhou, Q., Jiang, X., Wang, M.: Re-mote sensing measurement by full-Stokes-vector based on opto-lectronic modulator. Infrared Laser Eng. 39(2), 335–338 (2010)

27 Raman, C.V.: Christiaan Huyghens and the wave theory of light. Proc. Indian Acad. Sci. Sect. A Phys. Sci. 49(4), 185–192 (1959)

28 Zhao, H., Li, H., Lin, X.: Statically modulated spectral polarization imaging system. Spectrosc. Spectral Anal. 35(4), 1129–1133 (2015)

29 Xue, P., Wang, Z., Zhang, R.: Highly efficient measurement technology based on hyper-spectro polarimetric imaging. Chin. J. Lasers 43(8), 269–276 (2016)

30 Ma, X.: Research on development and related application of high performance pixel polarization camera. Dissertation for the Doctoral Degree. Hefei: University of Science and Technology of China (2019)

31 Pezzaniti, J.L., Chenault, D.B.: A division of aperture MWIR imaging polarimeter. Proc. SPIE Int. Soc. Opt. Eng. 44(3), 515–533 (2005)

32 Chen, Y., Zhu, Z., Liang, Z., Iannucci, L.E., Lake, S.P., Gruev, V.: Analysis of signal-to-noise ratio of angle of polarization and degree of polarization. OSA Continuum 4(5), 1461 (2021)

33 Gilboa, E., Cunningham, J.P., Nehorai, A., Gruev, V.: Image interpolation and denoising for division of focal plane sensors using Gaussian processes. Opt. Express 22(12), 15277–15291 (2014)

34 Miao, S., Fan, C., Wen, G., Gao, J., Zhao, G.: Research on denoising method for polarization degree image and polarization angle image of dim and weak targets. Acta Photonica Sinica 50(07), 108–120 (2021)

35 Qiao, J.: Research on image fusion technology based on polarization imaging. Dissertation for the Master Degree. Changchun: Changchun University of Science and Technology, 2017.

36 Tan, W., Tiwari, P., Pandey, H.M., Moreira, C., Jaiswal, A.K.: Multimodal medical image fusion algorithm in the era of big data. Neural Comput & Appl (2020).

37 Li, Y., Sun, Y., Huang, X., Qi, G., Zheng, M., Zhu, Z.: An image fusion method based on sparse representation and sum modified- Laplacian in NSCT domain. Entropy (Basel) 20(7), 522 (2018)

38 Zhu, Z., Zheng, M., Qi, G., Wang, D., Xiang, Y.: A phase congruency and local Laplacian energy based multi-modality medical image fusion method in NSCT domain. IEEE Access 7, 20811–20824 (2019)