Test method of laser paint removal based on multi-modal feature fusion

Abstract

Laser cleaning is a highly nonlinear physical process for solving poor single-modal (e.g., acoustic or vision) detection performance and low inter-information utilization. In this study, a multi-modal feature fusion network model was constructed based on a laser paint removal experiment. The alignment of heterogeneous data under different modals was solved by combining the piecewise aggregate approximation and gramian angular field. Moreover, the attention mechanism was introduced to optimize the dual-path network and dense connection network, enabling the sampling characteristics to be extracted and integrated. Consequently, the multi-modal discriminant detection of laser paint removal was realized. According to the experimental results, the verification accuracy of the constructed model on the experimental dataset was 99.17%, which is 5.77% higher than the optimal single-modal detection results of the laser paint removal. The feature extraction network was optimized by the attention mechanism, and the model accuracy was increased by 3.3%. Results verify the improved classification performance of the constructed multi-modal feature fusion model in detecting laser paint removal, the effective integration of acoustic data and visual image data, and the accurate detection of laser paint removal.

Keywords

laser cleaning / multi-modal fusion / image processing / deep learning

References

[1] JiaX, ZhangY, ChenY, et al. . Laser cleaning of slots of chrome-plated die [J]. Optics & Laser Technology, 2019, 119: 105659

[2] LiG, GaoW, ZhangL, et al. . The quality improvement of laser rubber removal for laminated metal valves [J]. Optics & Laser Technology, 2021, 139106785

[3] ChengJ, FangS, LiuD, et al. . Laser cleaning technology of metal surface and its application [J]. Applied Laser, 2018, 38(6): 1028-1037(in Chinese)

[4] GUO Nai-hao, WANG Jing-xuan, XIANG Xia. Study on laser cleaning process of sol-gel film optical surface [J]. Laser Technology, 2020(2): 156–160. (in Chinese)

[5] LeiZ, TianZ, ChenY. Laser cleaning technology in industrial fields [J]. Laser & Optoelectronics Progress, 2018, 55(3): 030005

[6] ZhangX, ChenY. Research progress and prospect of application of different types laser in laser cleaning technology [J]. Hot Working Technology, 2016, 45837-40(in Chinese)

[7] DingY, XueY, PangJ, et al. . Advances in in situ monitoring technology for laser processing [J]. Scientia Sinica Physica, Mechanica & Astronomica, 2019, 49(4): 044201

[8] KhedrA, PapadakisV, PouliP, et al. . The potential use of plume imaging for real-time monitoring of laser ablation cleaning of stonework [J]. Applied Physics B, 2011, 1052485-492

[9] ShiT, ZhouL, WangC, et al. . Automatic start detection system based on machine vision [J]. China Laser, 2019, 46(4): 83-89(in Chinese)

[10] ZhangM, DaiH, ZhengY, et al. . Research on laser cleaning detection of train paint coating based on color conversion [J]. Applied Laser, 2020, 40(4): 644-648(in Chinese)

[11] CaiY, CheungN H. Photoacoustic monitoring of the mass removed in pulsed laser ablation [J]. Microchemical Journal, 2011, 97(2): 109-112

[12] PapanikolaouA, TserevelakisG J, MelessanakiK, et al. . Development of a hybrid photoacoustic and optical monitoring system for the study of laser ablation processes upon the removal of encrustation from stonework [J]. Opto-Electronic Advances, 2020, 3(2): 19003701-19003711

[13] RamachandramD, TaylorG W. Deep multimodal learning: A survey on recent advances and trends [J]. IEEE Signal Processing Magazine, 2017, 34(6): 96-108

[14] BaltrušaitisT, AhujaC, MorencyL P. Multimodal machine learning: A survey and taxonomy [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(2): 423-443

[15] SunY, JiaZ, ZhuH. Overview of multimodal deep learning [J]. Computer Engineering and Application, 2020, 56(21): 1-10

[16] HeJ, ZhangC, LiX, et al. . A review of research on multimodal fusion technology for deep learning [J]. Computer Engineering, 2020, 46(5): 1-11(in Chinese)

[17] RasiwasiaN, CostaP J, CovielloE, et al. A new approach to cross-modal multimedia retrieval [C], 2010, New York, ACM Press: 251-260

[18] LiuY, FengX, ZhouZ. Multimodal video classification with stacked contractive autoencoders [J]. Signal Processing, 2016, 120: 761-766

[19] HabibianA, MensinkT, SnoekC G M. Video2vec embeddings recognize events when examples are scarce [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(10): 2089-2103

[20] LecunY, BengioY, HintonG. Deep learning [J]. Nature, 2015, 521(7553): 436-444

[21] LUO Yong-peng, WANG Lin-kun, GUO Xu, et al. Structural damage identification based on GAF-CNN using single sensor data [J]. Vibration. Test and Diagnosis, 2022, 42(1): 169–176, 202–203. DOI: https://doi.org/10.16450/j.cnki.issn.1004-6801.2022.01.026. (in Chinese)

[22] JiangJ, GuoM, YangS. Fault diagnosis method of rolling bearing based on GAF and DenseNet [J]. Industrial and Mining Automation, 2021, 47(8): 84-89(in Chinese)

[23] LiH. Method of dimensionality reduction and feature representation for time series [J]. Control and Decision, 2015, 30(3): 441-447

[24] HuangZ, LiW, LiJ, et al. . Dual-path attention network for single image super-resolution [J]. Expert Systems with Applications, 2021, 169: 114450

[25] WOO S, PARK J, LEE J Y, et al. CBAM: Convolutional block attention module [C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 3–19. DOI: https://doi.org/10.1007/978-3-030-01234-2_1.

[26] HuangG, LiuZ, Van Der MaatenL, et al. Densely connected convolutional networks [C], 2017, Honolulu, HI, USA, IEEE: 2261-2269

[27] ChenB, ZhaoT, LiuJ, et al. . Multipath feature recalibration DenseNet for image classification [J]. International Journal of Machine Learning and Cybernetics, 2021, 12(3): 651-660