Automated identification of steel weld defects, a convolutional neural network improved machine learning approach

Abstract

This paper proposes a machine-learning-based methodology to automatically classify different types of steel weld defects, including lack of the fusion, porosity, slag inclusion, and the qualified (no defects) cases. This methodology solves the shortcomings of existing detection methods, such as expensive equipment, complicated operation and inability to detect internal defects. The study first collected percussed data from welded steel members with or without weld defects. Then, three methods, the Mel frequency cepstral coefficients, short-time Fourier transform (STFT), and continuous wavelet transform were implemented and compared to explore the most appropriate features for classification of weld statuses. Classic and convolutional neural network-enhanced algorithms were used to classify, the extracted features. Furthermore, experiments were designed and performed to validate the proposed method. Results showed that STFT achieved higher accuracies (up to 96.63% on average) in the weld status classification. The convolutional neural network-enhanced support vector machine (SVM) outperformed six other algorithms with an average accuracy of 95.8%. In addition, random forest and SVM were efficient approaches with a balanced trade-off between the accuracies and the computational efforts.

Graphical abstract

Keywords

steel weld / machine learning / convolutional neural network / weld defect detection / classification task / percussion

References

[1] Connolly G D, Lowe M J S, Rokhlin S I, Temple J A G, Thompson D O, Chimenti D E. Synthetically focused imaging techniques in simulated austenitic steel welds using an ultrasonic phased array. American Institute of Physics, 2010, 1211(1): 871–878

[2] Afshari D, Sedighi M, Karimi M R, Barsoum Z. Prediction of residual stresses in resistance spot weld. Aircraft Engineering, 2016, 88(4): 492–497

[3] Du T, Sun J, Fu S, Zhang C, Gao Q. Research on ultrasonic flaw detection of steel weld in spatial grid structure. IOP Conference Series: Materials Science and Engineering, 2017, 216: 012016

[4] Hu S, Wang W, Alam M S, Ke K. Life-cycle benefits estimation of self-centering building structures. Engineering Structures, 2023, 284: 115982

[5] Nacereddine N, Ziou D, Hamami L. Fusion-based shape descriptor for weld defect radiographic image retrieval. International Journal of Advanced Manufacturing Technology, 2013, 68(9−12): 2815–2832

[6] Sani S, Ismail M P, Mohd S, Masenwat N A, Amran T S T, Amin M S M, Ahmad M R. Design and development of PC-based TOFD ultrasonic scanning system for welds inspection. In: proceedings of AIP Conference. New York: AIP Publishing, 2017, 1802: 050015

[7] Zhang L, Zhang Y, Dai B, Chen B, Li Y. Welding defect detection based on local image enhancement. IET Image Processing, 2019, 13(13): 2647–2658

[8] Sun J, Li C, Wu X, Palade V, Fang W. An effective method of weld defect detection and classification based on machine vision. IEEE Transactions on Industrial Informatics, 2019, 15(12): 6322–6333

[9] Malarvel M, Singh H. An autonomous technique for weld defects detection and classification using multi-class support vector machine in X-radiography image. Optik (Stuttgart), 2021, 231(10): 166342

[10] Li Y, Gao X, Zhang Y, You D, Wang C. Detection model of invisible weld defects by magneto-optical imaging at rotating magnetic field directions. Optics & Laser Technology, 2020, 121: 105772

[11] Gao X, Zhou X, Wang C, Ma N, You D. Skin depth and detection ability of magneto-optical imaging for weld defects in alternating magnetic field. Journal of Manufacturing Systems, 2020, 55: 44–55

[12] Dorafshan S, Maguire M, Collins W. Infrared thermography for weld inspection: Feasibility and application. Infrastructures, 2018, 3(4): 45

[13] Xu Z, Wu M, Fan W. Sparse-based defect detection of weld feature guided waves with a fusion of shear wave characteristics. Measurement, 2021, 174: 109018

[14] Zeng W, Cai F, Wang F, Miao L, You F, Yao F. Finite element simulation of laser-generated ultrasonic waves for quantitative detection of internal defects in welds. Optik (Stuttgart), 2020, 221: 165361

[15] Salamon J, Bello J P. Deep convolutional neural networks and data augmentation for environmental sound classification. IEEE Signal Processing Letters, 2017, 24(3): 279–283

[16] Kim Y, Sa J, Chung Y, Park D, Lee S. Resource-efficient pet dog sound events classification using LSTM-FCN based on time-series data. Sensors (Basel), 2018, 18(11): 4019

[17] Fernando T, Ghaemmaghami H, Denman S, Sridharan S, Hussain N, Fookes C. Heart sound segmentation using bidirectional LSTMs with attention. IEEE Journal of Biomedical and Health Informatics, 2020, 24(6): 1601–1609

[18] Lim S J, Jang S J, Lim J Y, Ko J H. Classification of snoring sound based on a recurrent neural network. Expert Systems with Applications, 2019, 123: 237–245

[19] Sujono A, Santoso B, Endra W. Sound vibration signal processing for detection and identification detonation (knock) to optimize performance Otto engine. AIP Publishing, 2016, LLC: 030003

[20] Bourke S, Nunes D, Stafford F, Hurley G, Graham I. Percussion of the chest re-visited: A comparison of the diagnostic value of ausculatory and conventional chest percussion. Irish Journal of Medical Science, 1989, 158(4): 82–84

[21] Ayodele K P, Ogunlade O, Olugbon O J, Akinwale O B, Kehinde L O. A medical percussion instrument using a wavelet-based method for archivable output and automatic classification. Computers in Biology and Medicine, 2020, 127: 104100

[22] Wang F, Song G. Looseness detection in cup-lock scaffolds using percussion-based method. Automation in Construction, 2020, 118: 103266

[23] Chen D, Montano V, Huo L, Fan S, Song G. Detection of subsurface voids in concrete-filled steel tubular (CFST) structure using percussion approach. Construction & Building Materials, 2020, 262: 119761

[24] Wang F, Song G, Mo Y. Shear loading detection of through bolts in bridge structures using a percussion-based one-dimensional memory-augmented convolutional neural network. Computer-Aided Civil and Infrastructure Engineering, 2020, 36(3): 289–301

[25] Zhou Y, Wang S, Zhou M, Chen H, Yuan C, Kong Q. Percussion-based bolt looseness identification using vibration-guided sound reconstruction. Structural Control and Health Monitoring, 2022, 29(2): e2876

[26] Hu S, Zhu S, Alam M S, Wang W. Machine learning-aided peak and residual displacement-based design method for enhancing seismic performance of steel moment-resisting frames by installing self-centering braces. Engineering Structures, 2022, 271: 114935

[27] Hu S, Qiu C, Zhu S. Machine learning-driven performance-based seismic design of hybrid self-centering braced frames with SMA braces and viscous dampers. Smart Materials and Structures, 2022, 31(10): 105024

[28] Hu S, Wang W, Alam M S, Zhu S, Ke K. Machine learning-aided peak displacement and floor acceleration-based design of hybrid self-centering braced frames. Journal of Building Engineering, 2023, 72: 106429

[29] Hu S, Qiu C, Zhu S. Floor acceleration control of self-centering braced frames using viscous dampers. Journal of Building Engineering, 2023, 74: 105944

[30] Raza A, Mehmood A, Ullah S, Ahmad M, Choi GS, On BW. Heartbeat sound signal classification using deep learning. Sensors, 2019, 19(21): 4819

[31] Wang P, Lim C S, Chauhan S, Foo J Y A, Anantharaman V. Phonocardiographic signal analysis method using a modified hidden Markov model. Annals of Biomedical Engineering, 2007, 35(3): 367–374

[32] ZhouZ. Machine Learning. Beijing: Tsinghua University Publishing House Co., Ltd., 2016

[33] RobertJ. Manipulate audio with a simple and easy high level interface. 2022. (available at the website of GitHub)

[34] Chagas P, Souza L, Araújo I, Aldeman N, Duarte A, Angelo M, dos-Santos W L C, Oliveira L. Classification of glomerular hypercellularity using convolutional features and support vector machine. Artificial Intelligence in Medicine, 2020, 103: 101808

[35] Ma Y, Xie Q, Liu Y, Xiong S. A weighted KNN-based automatic image annotation method. Neural Computing & Applications, 2020, 32(11): 6559–6570

[36] NairVHinton G E. Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning. Haifa: International Machine Learning Society, 2010, 807–814

[37] KrizhevskyASutskever IHintonG E. ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 2012, 25(2)

[38] He K, Zhang X, Ren S, Sun J. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904–1916

Electronic Supplementary Material

Supplementary material is available in the online version of this article at https://doi.org/10.1007/s11709-024-1045-7 and is accessible for authorized users.

Acknowledgements

The authors gratefully acknowledge the support of Shanghai Pinlan Data Technology Co., Ltd., and Open Fund of Shanghai Key Laboratory of Engineering Structure Safety, SRIBS (No. 2021-KF-06).

Competing interests

The authors declare that they have no competing interests.

RIGHTS & PERMISSIONS

2024 Higher Education Press