Investigation of crack segmentation and fast evaluation of crack propagation, based on deep learning

Abstract

Identifying crack and predicting crack propagation are critical processes for the risk assessment of engineering structures. Most traditional approaches to crack modeling are faced with issues of high computational costs and excessive computing time. To address this issue, we explore the potential of deep learning (DL) to increase the efficiency of crack detection and forecasting crack growth. However, there is no single algorithm that can fit all data sets well or can apply in all cases since specific tasks vary. In the paper, we present DL models for identifying cracks, especially on concrete surface images, and for predicting crack propagation. Firstly, SegNet and U-Net networks are used to identify concrete cracks. Stochastic gradient descent (SGD) and adaptive moment estimation (Adam) algorithms are applied to minimize loss function during iterations. Secondly, time series algorithms including gated recurrent unit (GRU) and long short-term memory (LSTM) are used to predict crack propagation. The experimental findings indicate that the U-Net is more robust and efficient than the SegNet for identifying crack segmentation and achieves the most outstanding results. For evaluation of crack propagation, GRU and LSTM are used as DL models and results show good agreement with the experimental data.

Graphical abstract

Keywords

deep learning / crack segmentation / crack propagation / encoder−decoder / recurrent neural network

References

[1] LemaitreJ. A Course on Damage Mechanics. Berlin: Springer Science & Business Media, 2012

[2] Graybeal B A, Phares B M, Rolander D D, Moore M, Washer G. Visual inspection of highway bridges. Journal of Nondestructive Evaluation, 2002, 21(3): 67–83

[3] Chatzi E N, Hiriyur B, Waisman H, Smyth A W. Experimental application and enhancement of the XFEM–GA algorithm for the detection of flaws in structures. Computers & Structures, 2011, 89(7−8): 556–570

[4] Vu-Bac N, Nguyen-Xuan H, Chen L, Bordas S, Kerfriden P, Simpson R, Liu G, Rabczuk T. A node-based smoothed extended finite element method (NS-XFEM) for fracture analysis. Computer Modeling in Engineering & Sciences, 2011, 73: 331–356

[5] Zou Q, Cao Y, Li Q, Mao Q, Wang S. Cracktree: Automatic crack detection from pavement images. Pattern Recognition Letters, 2012, 33(3): 227–238

[6] Butcher J B, Day C, Austin J, Haycock P, Verstraeten D, Schrauwen B. Defect detection in reinforced concrete using random neural architectures. Computer-Aided Civil and Infrastructure Engineering, 2014, 29(3): 191–207

[7] Yagi K, Tanaka S, Kawahara T, Nihei K, Okada H, Osawa N. Evaluation of crack propagation behaviors in a T-shaped tubular joint employing tetrahedral FE modeling. International Journal of Fatigue, 2017, 96: 270–282

[8] Nguyen-Thanh V M, Zhuang X, Nguyen-Xuan H, Rabczuk T, Wriggers P. A virtual element method for 2D linear elastic fracture analysis. Computer Methods in Applied Mechanics and Engineering, 2018, 340: 366–395

[9] Wang Q, Ji B, Fu Z, Ye Z. Evaluation of crack propagation and fatigue strength of rib-to-deck welds based on effective notch stress method. Construction & Building Materials, 2019, 201: 51–61

[10] Rabczuk T, Belytschko T. Cracking particles: A simplified meshfree method for arbitrary evolving cracks. International Journal for Numerical Methods in Engineering, 2004, 61(13): 2316–2343

[11] Rabczuk T, Belytschko T. A three-dimensional large deformation meshfree method for arbitrary evolving cracks. Computer Methods in Applied Mechanics and Engineering, 2007, 196(29−30): 2777–2799

[12] Oliveira H, Correia P L. Automatic road crack detection and characterization. IEEE Transactions on Intelligent Transportation Systems, 2013, 14(1): 155–168

[13] Vu-Bac N, Nguyen-Xuan H, Chen L, Lee C K, Zi G, Zhuang X, Liu G R, Rabczuk T. A phantom-node method with edge-based strain smoothing for linear elastic fracture mechanics. Journal of Applied Mathematics, 2013, 2013: 978026

[14] Adhikari R, Moselhi O, Bagchi A. Image-based retrieval of concrete crack properties for bridge inspection. Automation in Construction, 2014, 39: 180–194

[15] Shi Y, Cui L, Qi Z, Meng F, Chen Z. Automatic road crack detection using random structured forests. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(12): 3434–3445

[16] Minh H L, Sang-To T, Abdel Wahab M, Cuong-Le T. A new metaheuristic optimization based on K-means clustering algorithm and its application for structural damage identification. Knowledge-Based Systems, 2022, 251: 109189

[17] Ho T T, Kim T, Kim W J, Lee C H, Chae K J, Bak S H, Kwon S O, Jin G Y, Park E K, Choi S. A 3D-CNN model with CT-based parametric response mapping for classifying COPD subjects. Scientific Reports, 2021, 11(1): 34

[18] ParkS STranV TDoanN PHwangK B. Evaluation of damage level for ground settlement using the convolutional neural network. In: Proceedings of CIGOS 2021, Emerging Technologies and Applications for Green Infrastructure. Singapore: Springer Singapore, 2022, 1261–1268

[19] Ho T T, Kim G T, Kim T, Choi S, Park E K. Classification of rotator cuff tears in ultrasound images using deep learning models. Medical & Biological Engineering & Computing, 2022, 60(5): 1269

[20] Ogunjinmi P D, Park S S, Kim B, Lee D E. Rapid post-earthquake structural damage assessment using convolutional neural networks and transfer learning. Sensors, 2022, 22(9): 3471

[21] JeonH MNguyenV DJeonJ W. Pedestrian detection based on deep learning. In: Proceedings of IECON 2019—45th Annual Conference of the IEEE Industrial Electronics Society. New York: IEEE, 2019, 144–151

[22] Quang Dinh V, Munir F, Azam S, Yow K C, Jeon M. Transfer learning for vehicle detection using two cameras with different focal lengths. Information Sciences, 2020, 514: 71–87

[23] Park S S, Tran V T, Lee D E. Application of various yolo models for computer vision-based real-time pothole detection. Applied Sciences, 2021, 11(23): 11229

[24] Wang G, Li W, Zuluaga M A, Pratt R, Patel P A, Aertsen M, Doel T, David A L, Deprest J, Ourselin S, Vercauteren T. Interactive medical image segmentation using deep learning with image-specific fine tuning. IEEE Transactions on Medical Imaging, 2018, 37(7): 1562–1573

[25] Minaee S, Boykov Y Y, Porikli F, Plaza A J, Kehtarnavaz N, Terzopoulos D. Image segmentation using deep learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(7): 3523–3542

[26] le Hien Nguyen D, Thi Thanh Do D, Lee J, Rabczuk T, Nguyen-Xuan H. Forecasting damage mechanics by deep learning. Computers, Materials & Continua, 2019, 61(3): 951–977

[27] Schwarzer M, Rogan B, Ruan Y, Song Z, Lee D Y, Percus A G, Chau V T, Moore B A, Rougier E, Viswanathan H S, Srinivasan G. Learning to fail: Predicting fracture evolution in brittle material models using recurrent graph convolutional neural networks. Computational Materials Science, 2019, 162: 322–332

[28] Wang J, Zheng Y, Luo R, Ma J, Peng Y, Aslam S, Jia W. Prediction method of three-dimensional crack propagation path based on deep learning application. Advanced Engineering Materials, 2021, 23(4): 2001043

[29] Zou Q, Zhang Z, Li Q, Qi X, Wang Q, Wang S. DeepCrack: Learning hierarchical convolutional features for crack detection. IEEE Transactions on Image Processing, 2019, 28(3): 1498–1512

[30] Liu Z, Cao Y, Wang Y, Wang W. Computer vision-based concrete crack detection using U-net fully convolutional networks. Automation in Construction, 2019, 104: 129–139

[31] Anitescu C, Atroshchenko E, Alajlan N, Rabczuk T. Artificial neural network methods for the solution of second order boundary value problems. Computers, Materials & Continua, 2019, 59(1): 345

[32] Thi Thanh Do D, Lee J, Nguyen-Xuan H. Fast evaluation of crack growth path using time series forecasting. Engineering Fracture Mechanics, 2019, 218: 106567

[33] Hsu Y C, Yu C H, Buehler M J. Using deep learning to predict fracture patterns in crystalline solids. Matter, 2020, 3(1): 197–211

[34] Goswami S, Anitescu C, Chakraborty S, Rabczuk T. Transfer learning enhanced physics informed neural network for phase-field modeling of fracture. Theoretical and Applied Fracture Mechanics, 2020, 106: 102447

[35] Chakraborty A, Anitescu C, Zhuang X, Rabczuk T. Domain adaptation based transfer learning approach for solving PDEs on complex geometries. Engineering with Computers, 2022, 38(5): 4569

[36] Yang X, Li H, Yu Y, Luo X, Huang T, Yang X. Automatic pixel-level crack detection and measurement using fully convolutional network. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(12): 1090–1109

[37] Fukushima K, Miyake S, Ito T. Neocognitron: A neural network model for a mechanism of visual pattern recognition. IEEE Transactions on Systems, Man, and Cybernetics, 1983, SMC-13(5): 826–834

[38] Waibel A, Hanazawa T, Hinton G, Shikano K, Lang K J. Phoneme recognition using time-delay neural networks. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1989, 37(3): 328–339

[39] LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86(11): 2278–2324

[40] GoodfellowIBengioYCourvilleA. Deep Learning. Cambridge, MA: MIT Press, 2016

[41] Badrinarayanan V, Kendall A, Cipolla R. SegNet: A deep convolutional encoder−decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481–2495

[42] SimonyanKZissermanA. Very deep convolutional networks for large-scale image recognition. 2014, arXiv: 1409.1556

[43] RonnebergerOFischerPBroxT. U-Net: Convolutional networks for biomedical image segmentation. In: Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Cham, 2015, 234–241

[44] DiPietroRHagerG D. Deep learning: RNNs and LSTM. In: Proceedings of the Handbook of Medical Image Computing and Computer Assisted Intervention. Amsterdam: Elsevier, 2020, 503–519

[45] Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation, 1997, 9(8): 1735–1780

[46] ChoKvanMerriënboer BGulcehreCBahdanauDBougaresFSchwenkHBengioY. Learning phrase representations using RNN encoder−decoder for statistical machine translation. 2014, arXiv: 1406.1078

[47] Greff K, Srivastava R K, Koutník J, Steunebrink B R, Schmidhuber J. LSTM: A search space odyssey. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(10): 2222–2232

[48] MaY DLiuQQianZ B. Automated image segmentation using improved PCNN model based on cross-entropy. In: Proceedings of the 2004 International Symposium on Intelligent Multimedia, Video and Speech Processing. New York: IEEE, 2004, 743–746

[49] SudreC HLiWVercauterenTOurselinSJorge CardosoM. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Proceedings of the Deep learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Cham: Springer Cham, 2017, 240–248

[50] SalehiS S MErdogmusDGholipourA. Tversky loss function for image segmentation using 3D fully convolutional deep networks. In: Proceedings of the International Workshop on Machine Learning in Medical Imaging. Cham: Springer Cham, 2017, 379–387

[51] AbrahamNKhanN M. A novel focal Tversky loss function with improved attention U-Net for lesion segmentation. In: Proceedings of the 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). New York: IEEE, 2019, 683–687

[52] BermanMTrikiA RBlaschkoM B. The Lovász-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks. In: Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2018, 4413–4421

[53] Robbins H, Monro S. A stochastic approximation method. Annals of Mathematical Statistics, 1951, 22(3): 400–407

[54] KingmaD PBaJ. Adam: A method for stochastic optimization. 2014, arXiv: 1412.6980

[55] BertelsJEelbodeTBermanMVandermeulenDMaesFBisschopsRBlaschkoM B. Optimizing the dice score and Jaccard index for medical image segmentation: Theory and practice. In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Cham, 2019, 92–100

[56] BotchkarevA. Performance metrics (error measures) in machine learning regression, forecasting and prognostics: Properties and typology. 2018, arXiv: 1809.03006

[57] Liu Y, Yao J, Lu X, Xie R, Li L. DeepCrack: A deep hierarchical feature learning architecture for crack segmentation. Neurocomputing, 2019, 338: 139–153

[58] Winkler B, Hofstetter G, Niederwanger G. Experimental verification of a constitutive model for concrete cracking. Proceedings of the Institution of Mechanical Engineers, Part L: Journal of Materials: Design and Applications, 2001, 215(2): 75–86

[59] Wang H, Zhang W, Sun F, Zhang W. A comparison study of machine learning based algorithms for fatigue crack growth calculation. Materials, 2017, 10(5): 543

[60] IoffeSSzegedyC. Batch normalization: Accelerating deep network training by reducing internal covariate shift. 2015, arXiv: 1502.03167

[61] AgarapA F. Deep learning using rectified linear units (ReLU). 2018, arXiv: 1803.08375

[62] Paris P, Erdogan F. A critical analysis of crack propagation laws. Journal of Basic Engineering, 1963, 85(4): 528–533

[63] Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 2014, 15: 1929–1958

[64] ZhuJ YParkTIsolaPEfrosA A. Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV). New York: IEEE, 2017, 2242–2251

Acknowledgements

The first author would like to thank European Commission H2020-MSCA-RISE BESTOFRAC project for research funding.

Funding note

Open Access funding enabled and organized by Projekt DEAL.

Open Access

This article is licensed under a Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Competing interests

The authors declare that they have no competing interests.

RIGHTS & PERMISSIONS

2024 The Author(s). This article is published with open access at link.springer.com and journal.hep.com.cn