Vision-based welding quality detection of steel bridge components in complex construction environments

Abstract

Currently, welding quality detection remains dependent on manual operation, while the increase in the span and intricacy of steel bridges has rendered the conventional method of detection insufficient to fulfill the engineering requirements. This paper presents a systematic study of welding quality detection of steel bridges based on fusion of point clouds and images in complex construction environments. (1) A welding detection system is developed that could filter out stray light and capture weld images. (2) This paper enhances the centerline extraction method in 3D reconstruction, which could effectively filter out noise interference and precisely reconstruct weld contours. The contour dimensions of both filler and cover welds are identified through feature point extraction, with an estimated detection error under 0.6%. (3) This paper optimizes the feature extraction of the Faster R-CNN network based on the appearance feature and detection need of welding defects, resulting in an improvement of 28.3 in mAP. Experimental results demonstrate that the proposed welding quality detection is both efficient and accurate, and is capable of meeting the requirements of actual steel bridge construction.

References

[1] Yuan D, Cui C, Zhang Q, et al.. Fatigue damage evaluation of welded joints in steel bridge based on meso-damage mechanics. Int J Fatigue, 2022, 161. 106898

[2] Li C, Ji B, Fu Z, Yao Y. Stress concentration factor-based hot-spot stress prediction model for the web-gap weld of main girder in steel truss bridge. KSCE J Civ Eng, 2024, 28(1): 262-274.

[3] Jiang X, Cao D, Qiang X, Xu C. Study on fatigue performance of steel bridge welded joints considering initial defects. J Constr Steel Res, 2024, 212. 108369

[4] Zhang C, Cao D, You W, et al.. Fatigue failure of welded details in steel bridge pylons. Eng Fail Anal, 2021, 127. 105530

[5] Dong J, Tang D, Wu D, et al.. Research status and prospect of laser vision weld Seam tracking image processing under strong noise. Electr Weld Mach, 2022, 52: 1-16.

[6] Chen S, Liu J, Chen B, Suo X. Universal fillet weld joint recognition and positioning for robot welding using structured light. Robot Comput-Integr Manuf, 2022, 74. 102279

[7] Stavridis J, Papacharalampopoulos A, Stavropoulos P. Quality assessment in laser welding: a critical review. Int J Adv Manuf Technol, 2018, 94: 1825-1847.

[8] Xu Y, Wang Z. Visual sensing technologies in robotic welding: Recent research developments and future interests. Sens and Actuators A: Phys, 2021, 320. 112551

[9] Xu F, Hou Z, Xiao R, et al.. A novel welding path generation method for robotic multi-layer multi-pass welding based on weld seam feature point. Measur, 2023, 216. 112910

[10] Liang Y, Xu Z. Intelligent inspection of appearance quality for precast concrete components based on improved YOLO model and multi-source data. Eng, Constr and Archit Manag, 2023.

[11] Xu F, Xu Y, Zhang H, Chen S. Application of sensing technology in intelligent robotic arc welding: A review. J Manuf Process, 2022, 79: 854-880.

[12] Jasiuniene E, Mazeika L, Samaitis V, et al.. Ultrasonic non-destructive testing of complex titanium/carbon fibre composite joints. Ultrason, 2019, 95: 13-21.

[13] Jin S, Wang Z, Luo Z. TOFD detection of shallow subsurface defects in aluminum alloy thin plates by half-skip mode-converted wave. Chin J Aeronaut, 2024, 37: 454-463.

[14] Noguchi Y, Tsunokai M, Nakata K, Takeda N. Applicability of eddy current technique in in-bore NDT tool for ITER hydraulic pipe welds. Fusion Eng Des, 2019, 146: 2571-2576.

[15] Xie F, Wu X, Zhang H, Zhang X. Research on pulsed eddy current automatic detection technology for weld defects of pressure vessel cylinder. Meas, 2021, 176. 109081

[16] Mery D, Riffo V, Zscherpel U, et al.. GDXray: The database of x-ray images for nondestructive testing. J Nondestruct Eval, 2015, 34: 42.

[17] Abdelkader R, Ramou N, Khorchef M, et al.. Segmentation of x-ray image for welding defects detection using an improved Chan-Vese model. Mater Today: Proc, 2021, 42: 2963-2967.

[18] Begemann F, Muller J, Unglaub J, et al.. Welding under service conditions – Resulting weld quality and fatigue strength assessment. Eng Fail Anal, 2024, 160. 108142

[19] Trimech M, Annan CD, Walbridge S, Maljaars J. Fatigue behaviour of butt-lap friction stir welded joints used with aluminum bridge decks. Struct, 2023, 51: 1795-1805.

[20] Li D, Chen Q, Wang H, et al.. Deep learning-based acoustic emission data clustering for crack evaluation of welded joints in field bridges. Autom Constr, 2024, 165. 105540

[21] Gu WP, Xiong ZY, Wan W. Autonomous seam acquisition and tracking system for multi-pass welding based on vision sensor. Int J Adv Manuf Technol, 2013, 69: 451-460.

[22] Liu J, Jiao T, Li S, et al.. Automatic seam detection of welding robots using deep learning. Autom Constr, 2022, 143. 104582

[23] Jing O, Lai Z, Qinghong W, et al.. Weld-seam identification and model reconstruction of remanufacturing blade based on three-dimensional vision. Adv Eng Inform, 2021, 49. 101300

[24] Chu HH, Wang ZY. A vision-based system for post-welding quality measurement and defect detection. Int J Adv Manuf Technol, 2016, 86: 3007-3014.

[25] Han Y, Fan J, Yang X. A structured light vision sensor for on-line weld bead measurement and weld quality inspection. Int J Adv Manuf Technol, 2020, 106: 2065-2078.

[26] Wang N, Zhong K, Shi X, Zhang X. A robust weld seam recognition method under heavy noise based on structured-light vision. Robot Comput-Integr Manuf, 2020, 61. 101821

[27] Deng L, Lei T, Wu C, et al.. A weld seam feature real-time extraction method of three typical welds based on target detection. Meas, 2023, 207. 112424

[28] Jia N, Li Z, Ren J, Wang Y, Yang L. A 3D reconstruction method based on grid laser and gray scale photo for visual inspection of welds. Opt & Laser Technol, 2019, 119. 105648

[29] Zhang G, Shi Y, Gu Y, Fan D. Welding torch attitude-based study of human welder interactive behavior with weld pool in GTAW. Robot and Comput-Integr Manuf, 2017, 48: 145-156.

[30] Liu Z, Cao Y, Wang Y, Wang W. Computer vision-based concrete crack detection using U-net fully convolutional networks. Autom Constr, 2019, 104: 129-139.

[31] Bacioiu D, Melton G, Papaelias M, Shaw R. Automated defect classification of Aluminium 5083 TIG welding using HDR camera and neural networks. J Manuf Process, 2019, 45: 603-613.

[32] Gu J, Wang Z, Kuen J, et al.. Recent advances in convolutional neural networks. Pattern Recognit, 2018, 77: 354-377.

[33] Zhu Ge, Liu, . Deep Learning-Based Classification of Weld Surface Defects. Appl Sci, 2019, 9: 3312.

[34] Jiao W, Wang Q, Cheng Y, Zhang Y. End-to-end prediction of weld penetration: A deep learning and transfer learning based method. J Manuf Process, 2021, 63: 191-197.

[35] Yang L, Jiang H. Weld defect classification in radiographic images using unified deep neural network with multi-level features. J Intell Manuf, 2021, 32: 459-469.

[36] Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell, 2017, 39: 1137-1149.

[37] Mansour RF, Escorcia-Gutierrez J, Gamarra M, et al.. Intelligent video anomaly detection and classification using faster RCNN with deep reinforcement learning model. Image Vis Comput, 2021, 112. 104229

[38] Wang J, Mu C, Mu S, et al.. Welding seam detection and location: Deep learning network-based approach. Int J Press Vessel Pip, 2023, 202. 104893

[39] Zhu X, Hu H, Lin S, Dai J (2018) Deformable ConvNets V2: More deformable, better results. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) :9300–9308. https://doi.org/10.1109/CVPR.2019.00953

[40] Hu D, Anantrasirichai N (2022) Object Recognition in Atmospheric Turbulence Scenes. 2023 31st European Signal Processing Conference (EUSIPCO) :561–565. https://doi.org/10.23919/EUSIPCO58844.2023.10289748

[41] He K, Zhang X, Ren S, Sun J (2015) Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) :770–778. https://doi.org/10.1109/CVPR.2016.90

[42] Woo S, Park J, Lee JY, Kweon IS (2018) CBAM: Convolutional Block Attention Module. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer Vision -- ECCV 2018. ECCV 2018. Lecture Notes in Computer Science(), vol 11211, pp 3–19. https://doi.org/10.1007/978-3-030-01234-2_1

[43] Dung CV, Sekiya H, Hirano S, et al.. A vision-based method for crack detection in gusset plate welded joints of steel bridges using deep convolutional neural networks. Autom Constr, 2019, 102: 217-229.

Funding

National Natural Science Foundation of China(72071043)

RIGHTS & PERMISSIONS

The Author(s)