Comparative analysis of twelve transfer learning models for the prediction and crack detection in concrete dams, based on borehole images

Abstract

Disaster-resilient dams require accurate crack detection, but machine learning methods cannot capture dam structural reaction temporal patterns and dependencies. This research uses deep learning, convolutional neural networks, and transfer learning to improve dam crack detection. Twelve deep-learning models are trained on 192 crack images. This research aims to provide up-to-date detecting techniques to solve dam crack problems. The finding shows that the EfficientNetB0 model performed better than others in classifying borehole concrete crack surface tiles and normal (undamaged) surface tiles with 91% accuracy. The study’s pre-trained designs help to identify and to determine the specific locations of cracks.

Graphical abstract

Keywords

concrete dam / borehole closed-circuit television / deep learning models / crack detection / water resources management

References

[1] Li Y, Bao T, Xu B, Shu X, Zhou Y, Du Y, Wang R, Zhang K. A deep residual neural network framework with transfer learning for concrete dams patch-level crack classification and weakly-supervised localization. Measurement, 2022, 188: 110641

[2] Mulligan M, van Soesbergen A, Sáenz L. GOODD, a global dataset of more than 38,000 georeferenced dams. Scientific Data, 2020, 7(1): 31

[3] EmbankmentsTechnical CommitteeDamsSlopes. Remote Sensing for Monitoring Embankments, Dams, and Slopes: Recent Advances. Reston, VA: American Society of Civil Engineers, 2021

[4] Li Y, Bao T, Shu X, Chen Z, Gao Z, Zhang K. A hybrid model integrating principal component analysis, fuzzy C-means, and Gaussian process regression for dam deformation prediction. Arabian Journal for Science and Engineering, 2021, 46(5): 4293–4306

[5] Feng D, Feng M Q. Computer vision for SHM of civil infrastructure: From dynamic response measurement to damage detection—A review. Engineering Structures, 2018, 156: 105–117

[6] Kim H, Ahn E, Shin M, Sim S H. Crack and noncrack classification from concrete surface images using machine learning. Structural Health Monitoring, 2019, 18(3): 725–738

[7] Chen B, Zhang H, Wang G, Huo J, Li Y, Li L. Automatic concrete infrastructure crack semantic segmentation using deep learning. Automation in Construction, 2023, 152: 104950

[8] Dong C Z, Catbas F N. A review of computer vision-based structural health monitoring at local and global levels. Structural Health Monitoring, 2021, 20(2): 692–743

[9] Meng Q, Xue H, Song H, Zhuang X, Rabczuk T. Rigid-block DEM modeling of mesoscale fracture behavior of concrete with random aggregates. Journal of Engineering Mechanics, 2023, 149(2): 04022114

[10] Ishfaque M, Dai Q, Haq N, Jadoon K, Shahzad S M, Janjuhah H T. Use of recurrent neural network with long short-term memory for seepage prediction at Tarbela Dam, KP, Pakistan. Energies, 2022, 15(9): 3123

[11] Mauludin L M, Budiman B A, Santosa S P, Zhuang X, Rabczuk T. Numerical modeling of microcrack behavior in encapsulation-based self-healing concrete under uniaxial tension. Journal of Mechanical Science and Technology, 2020, 34(5): 1847–1853

[12] Mauludin L M, Zhuang X, Rabczuk T. Computational modeling of fracture in encapsulation-based self-healing concrete using cohesive elements. Composite Structures, 2018, 196: 63–75

[13] Quayum M S, Zhuang X, Rabczuk T. Computational model generation and RVE design of self-healing concrete. Frontiers of Structural and Civil Engineering, 2015, 9(4): 383–396

[14] Zhu H, Wu X, Luo Y, Jia Y, Wang C, Fang Z, Zhuang X, Zhou S. Prediction of early compressive strength of ultrahigh-performance concrete using machine learning methods. International Journal of Computational Methods, 2023, 20(8): 2141023

[15] Chen J S, Lin K Y, Young S Y. Effects of crack width and permeability on moisture-induced damage of pavements. Journal of Materials in Civil Engineering, 2004, 16(3): 276–282

[16] Khan S U R, Zhao M, Asif S. Hybrid-NET: A fusion of DenseNet169 and advanced machine learning classifiers for enhanced brain tumor diagnosis. International Journal of Imaging Systems and Technology, 2024, 34(1): e22975

[17] Al-Masni M A, Kim D H, Kim T S. Multiple skin lesions diagnostics via integrated deep convolutional networks for segmentation and classification. Computer Methods and Programs in Biomedicine, 2020, 190: 105351

[18] HuangGLiuZvan der MaatenL. Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE Computer Society, 2017, 4700–4708

[19] HowardA GZhuMChenB. Mobilenets: Efficient convolutional neural networks for mobile vision applications. 2017, arXiv: 1704.04861

[20] HeKZhangXRenS. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV: IEEE Computer Society, 2016, 770–778

[21] CholletF. Deep learning with separable convolutions. 2016, arXiv: 1610.2357

[22] QuangN H. Automatic skin lesion analysis towards melanoma detection. In: Proceedings of 2017 21st Asia Pacific Symposium on Intelligent and Evolutionary Systems (IES). Hanoi: Springer, 2017, 106–111

[23] ShahinA HKamalAElattarM A. Deep ensemble learning for skin lesion classification from dermoscopic images. In: Prodeeings of 2018 9th Cairo International Biomedical Engineering Conference (CIBEC). Cairo: IEEE, 2018, 150–153

[24] SzegedyCIoffeSVanhouckeV. Inception-V4, inception-ResNet and the impact of residual connections on learning. In: Proceedings of the AAAI Conference on Artificial Intelligence. San Francisco, CA: AAAI Press, 2017

[25] TanMLeQ. Efficientnet: Rethinking model scaling for convolutional neural networks. In: Proceedings of International Conference on Machine Learning. Beach, CA: PMLR, 2019, 6105–6114

[26] ChhabraMKumarR. A smart healthcare system based on classifier DenseNet 121 model to detect multiple diseases. In: Proceedings of Second MRCN 2021. Singapore: Springer Nature Singapore, 2022, 297–312

[27] Ashqar B A, Abu-Naser S S. Identifying images of invasive hydrangea using pre-trained deep convolutional neural networks. International Journal of Academic Engineering Research, 2019, 3(3): 28–36

[28] Wang M, Wang E, Liu X, Wang Z, Wang C. Influence of neural network structure on rock intelligent classification based on structural and tectonic features of rocks. Rock Mechanics and Rock Engineering, 2022, 55(9): 5415–5432

[29] Li Y, Wang P, Feng Q, Ji X, Jin D, Gong J. Landslide detection based on shipborne images and deep learning models: A case study in the three gorges reservoir area in China. Landslides, 2023, 20(3): 547–558

[30] TolstayaEEgorovA. Segmentation of seismic images. In: Processing of 13th International Conference in Computer and Graphic Visualization, Computer Vision and Image Processing. Hangzhou: IEEE, 2021, 564–570

[31] Jeon J, Lee J, Shin D, Park H. Development of dam safety management system. Advances in Engineering Software, 2009, 40(8): 554–563

[32] Zhang H, Yang G, Li H, Du W, Wang J. Pixel-wise detection algorithm for crack structural reconstruction based on rock CT images. Automation in Construction, 2023, 152: 104895

[33] Li Y, Bao T, Huang X, Chen H, Xu B, Shu X, Zhou Y, Cao Q, Tu J, Wang R, Zhang K. Underwater crack pixel-wise identification and quantification for dams via lightweight semantic segmentation and transfer learning. Automation in Construction, 2022, 144: 104600

[34] StrickerREisenbachMSesselmannM. Improving visual road condition assessment by extensive experiments on the extended gaps dataset. In: Proceedings of 2019 International Joint Conference on Neural Networks (IJCNN). Budapest: IEEE, 2019, 1–8

[35] Li Z. Global sensitivity analysis of the static performance of concrete gravity dam from the viewpoint of structural health monitoring. Archives of Computational Methods in Engineering, 2021, 28(3): 1611–1646

[36] Dong J, Wang N, Fang H, Hu Q, Zhang C, Ma B, Ma D, Hu H. Innovative method for pavement multiple damages segmentation and measurement by the Road-Seg-CapsNet of feature fusion. Construction and Building Materials, 2022, 324: 126719

[37] Kang D, Benipal S S, Gopal D L, Cha Y J. Hybrid pixel-level concrete crack segmentation and quantification across complex backgrounds using deep learning. Automation in Construction, 2020, 118: 103291

[38] Xiang Y, Sheng J B, Wang L, Cai Y B, Meng Y, Cai W. Research progresses on equipment technologies used in safety inspection, repair, and reinforcement for deepwater dams. Science China. Technological Sciences, 2022, 65(5): 1059–1071

[39] Kov’ari K, Peter G. Continuous strain monitoring in the rock foundation of a large gravity dam. Rock Mechanics and Rock Engineering, 1983, 16(3): 157–171

[40] Li Y, Bao T, Gong J, Shu X, Zhang K. The prediction of dam displacement time series using STL, extra-trees, and stacked LSTM neural network. IEEE Access: Practical Innovations, Open Solutions, 2020, 8: 94440–94452

[41] Wang W, Wang M, Li H, Zhao H, Wang K, He C, Wang J, Zheng S, Chen J. Pavement crack image acquisition methods and crack extraction algorithms: A review. Journal of Traffic and Transportation Engineering, 2019, 6(6): 535–556

[42] Li Y, Bao T, Gao Z, Shu X, Zhang K, Xie L, Zhang Z. A new dam structural response estimation paradigm powered by deep learning and transfer learning techniques. Structural Health Monitoring, 2022, 21(3): 770–787

[43] Li Y, Bao T, Shu X, Gao Z, Gong J, Zhang K. Data-driven crack behavior anomaly identification method for concrete dams in long-term service using offline and online change point detection. Journal of Civil Structural Health Monitoring, 2021, 11(5): 1449–1460

[44] Ishfaque M, Salman S, Jadoon K Z, Danish A A K, Bangash K U, Dai Q W. Understanding the effect of hydro-climatological parameters on dam seepage using shapley additive explanation (SHAP): A case study of earth-fill Tarbela dam, Pakistan. Water, 2022, 14(17): 2598

[45] Yamaguchi T, Hashimoto S. Fast crack detection method for large-size concrete surface images using percolation-based image processing. Machine Vision and Applications, 2010, 21(5): 797–809

[46] Chun P, Izumi S, Yamane T. Automatic detection method of cracks from concrete surface imagery using two-step light gradient boosting machine. Computer-Aided Civil and Infrastructure Engineering, 2021, 36(1): 61–72

[47] Hsieh Y A, Tsai Y J. Machine learning for crack detection: Review and model performance comparison. Journal of Computing in Civil Engineering, 2020, 34(5): 04020038

[48] Shirhatti V, Borthakur A, Ray S. Effect of reference scheme on power and phase of the local field potential. Neural Computation, 2016, 28(5): 882–913

[49] Pathirage C S N, Li J, Li L, Hao H, Liu W, Ni P. Structural damage identification based on autoencoder neural networks and deep learning. Engineering Structures, 2018, 172: 13–28

[50] Ni F, Zhang J, Chen Z. Pixel-level crack delineation in images with convolutional feature fusion. Structural Control and Health Monitoring, 2019, 26(1): e2286

[51] Kim B, Cho S. Image-based concrete crack assessment using mask and region-based convolutional neural network. Structural Control and Health Monitoring, 2019, 26(8): e2381

[52] Huyan J, Li W, Tighe S, Xu Z, Zhai J. CrackU-net: A novel deep convolutional neural network for pixelwise pavement crack detection. Structural Control and Health Monitoring, 2020, 27(8): e2551

[53] Zhao C, Ding D, Du Z, Shi Y, Su G, Yu S. Analysis of perception accuracy of roadside millimeter-wave radar for traffic risk assessment and early warning systems. International Journal of Environmental Research and Public Health, 2023, 20(1): 879

[54] Zhao C, Song A, Zhu Y, Jiang S, Liao F, Du Y. Data-driven indoor positioning correction for infrastructure-enabled autonomous driving systems: A lifelong framework. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(4): 3908–3921

[55] Zhao C, Song A, Du Y, Yang B. TrajGAT: A map-embedded graph attention network for real-time vehicle trajectory imputation of roadside perception. Transportation Research Part C, Emerging Technologies, 2022, 142: 103787

[56] Cha Y J, Choi W, Suh G, Mahmoudkhani S, Büyüköztürk O. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(9): 731–747

[57] Huyan J, Li W, Tighe S, Zhai J, Xu Z, Chen Y. Detection of sealed and unsealed cracks with complex backgrounds using deep convolutional neural network. Automation in Construction, 2019, 107: 102946

[58] Du Y, Pan N, Xu Z, Deng F, Shen Y, Kang H. Pavement distress detection and classification based on YOLO network. International Journal of Pavement Engineering, 2021, 22(13): 1659–1672

[59] Liang J, Chen B, Shao C, Li J, Wu B. Time reverse modeling of damage detection in underwater concrete beams using piezoelectric intelligent modules. Sensors, 2020, 20(24): 7318

[60] Chen J, Xiong F, Zhu Y, Yan H. A crack detection method for underwater concrete structures using sensing-heating system with porous casing. Measurement, 2021, 168: 108332

[61] Si J, Xiong W, Zhong D, Yan A, Wang P, Liu Z. Piezoelectric-based damage-depth monitoring method for underwater energy-relief blasting technique. Journal of Civil Structural Health Monitoring, 2021, 11(2): 251–264

[62] KönigJJenkinsM DBarrieP. A convolutional neural network for pavement surface crack segmentation using residual connections and attention gating. In: Proceedings of 2019 IEEE International Conference on Image Processing (ICIP). Taipei, China: IEEE, 2019, 1460–1464

[63] Zhang J, Zhang J. An improved nondestructive semantic segmentation method for concrete dam surface crack images with high resolution. Mathematical Problems in Engineering, 2020, 2020: 1–14

[64] Dung C V, Sekiya H, Hirano S, Okatani T, Miki C. A vision-based method for crack detection in gusset plate welded joints of steel bridges using deep convolutional neural networks. Automation in Construction, 2019, 102: 217–229

[65] Sward D, Monk J, Barrett N. A systematic review of remotely operated vehicle surveys for visually assessing fish assemblages. Frontiers in Marine Science, 2019, 6: 134

[66] Capocci R, Dooly G, Omerdić E, Coleman J, Newe T, Toal D. Inspection-class remotely operated vehicles—A review. Journal of Marine Science and Engineering, 2017, 5(1): 13

[67] Lund-Hansen L C, Juul T, Eskildsen T D, Hawes I, Sorrell B, Melvad C, Hancke K. A low-cost remotely operated vehicle (ROV) with an optical positioning system for under-ice measurements and sampling. Cold Regions Science and Technology, 2018, 151: 148–155

[68] GuoHZhuangXRabczukT. A deep collocation method for the bending analysis of Kirchhoff plate. 2021, arXiv: 2102.02617

[69] Samaniego E, Anitescu C, Goswami S, Nguyen-Thanh V M, Guo H, Hamdia K, Zhuang X, Rabczuk T. An energy approach to the solution of partial differential equations in computational mechanics via machine learning: Concepts, implementation and applications. Computer Methods in Applied Mechanics and Engineering, 2020, 362: 112790

[70] Zhuang X, Guo H, Alajlan N, Zhu H, Rabczuk T. Deep autoencoder based energy method for the bending, vibration, and buckling analysis of Kirchhoff plates with transfer learning. European Journal of Mechanics-A/Solids, 2021, 87: 104225

[71] Guo H, Zhuang X, Chen P, Alajlan N, Rabczuk T. Stochastic deep collocation method based on neural architecture search and transfer learning for heterogeneous porous media. Engineering with Computers, 2022, 38(6): 5173–5198

[72] Guo H, Zhuang X, Fu X. Physics-informed deep learning for three-dimensional transient heat transfer analysis of functionally graded materials. Computational Mechanics, 2023, 72(3): 1–12

[73] Guo H, Zhuang X, Alajlan N, Rabczuk T. Physics-informed deep learning for melting heat transfer analysis with model-based transfer learning. Computers & Mathematics with Applications, 2023, 143: 303–317

[74] KwasigrochAMikołajczykAGrochowskiM. Deep neural networks approach to skin lesions classification—A comparative analysis. In: Proceedings of 2017 22nd International Conference on Methods and Models in Automation and Robotics (MMAR). Miedzyzdroje: IEEE, 2017, 1069–1074

[75] BonaccorsoG. Machine Learning Algorithms. London: Packt Publishing Ltd., 2017

[76] SaitoTRehmsmeierM. Basic evaluation measures from the confusion matrix. 2017. Available at the website of Classeval

Acknowledgements

The authors wish to express their gratitude to several entities for their contributions to this research. This research was supported by Youth Science and Technology Fund (No. B240201122) (Muhammad Ishfaque) under the Post-Doctoral research program of Hohai University, Nanjing, Jiangsu Province of China. This work was also supported by the National Natural Science Foundation of China (Grant Nos. 61972136, 41874148, and 42174178), the Natural Science and Foundation of Hubei Province (No. 2020CFB497), the Hubei Provincial Department of Education Outstanding Youth Scientific Innovation Team Support Foundation (Nos. T201410 and T2020017), the Natural Science Foundation of Education Department of Hubei Province (No. B2020149), the Science and Technology Research Project of the Education Department of Hubei Province (No. Q20222704), and the Natural Science Foundation of Xiaogan City (Nos. XGKJ2022010095 and XGKJ2022010094). The funding is a foreign expert project of Henan Province (No. HNGD2023027).

Competing interests

The authors declare that they have no competing interests.

RIGHTS & PERMISSIONS

2024 Higher Education Press