A deep neural network combined with a two-stage ensemble model for detecting cracks in concrete structures

Abstract

Detection of cracks in concrete structures is critical for their safety and the sustainability of maintenance processes. Traditional inspection techniques are costly, time-consuming, and inefficient regarding human resources. Deep learning architectures have become more widespread in recent years by accelerating these processes and increasing their efficiency. Deep learning models (DLMs) stand out as an effective solution in crack detection due to their features such as end-to-end learning capability, model adaptation, and automatic learning processes. However, providing an optimal balance between model performance and computational efficiency of DLMs is a vital research topic. In this article, three different methods are proposed for detecting cracks in concrete structures. In the first method, a Separable Convolutional with Attention and Multi-layer Enhanced Fusion Network (SCAMEFNet) deep neural network, which has a deep architecture and can provide a balance between the depth of DLMs and model parameters, has been developed. This model was designed using a convolutional neural network, multi-head attention, and various fusion techniques. The second method proposes a modified vision transformer (ViT) model. A two-stage ensemble learning model, deep feature-based two-stage ensemble model (DFTSEM), is proposed in the third method. In this method, deep features and machine learning methods are used. The proposed approaches are evaluated using the Concrete Cracks Image Data set, which the authors collected and contains concrete cracks on building surfaces. The results show that the SCAMEFNet model achieved an accuracy rate of 98.83%, the ViT model 97.33%, and the DFTSEM model 99.00%. These findings show that the proposed techniques successfully detect surface cracks and deformations and can provide practical solutions to real-world problems. In addition, the developed methods can contribute as a tool for BIM platforms in smart cities for building health.

Graphical abstract

Keywords

concrete cracks image dataset / crack detection / depthwise separable convolution / multi-scale feature fusion / SCAMEFNet deep neural network / two-stage ensemble learning model

References

[1] Spencer B F Jr, Hoskere V, Narazaki Y. Advances in computer vision-based civil infrastructure inspection and monitoring. Engineering, 2019, 5(2): 199–222

[2] Andrushia A D, Anand N, Neebha T M, Naser M Z, Lubloy E. Autonomous detection of concrete damage under fire conditions. Automation in Construction, 2022, 140: 104364

[3] Han X, Zhao Z, Chen L, Hu X, Tian Y, Zhai C, Wang L, Huang X. Structural damage-causing concrete cracking detection based on a deep-learning method. Construction & Building Materials, 2022, 337: 127562

[4] Asadi Shamsabadi E, Xu C, Rao A S, Nguyen T, Ngo T, Dias-da-Costa D. Vision transformer-based autonomous crack detection on asphalt and concrete surfaces. Automation in Construction, 2022, 140: 104316

[5] Rao A S, Nguyen T, Palaniswami M, Ngo T. Vision-based automated crack detection using convolutional neural networks for condition assessment of infrastructure. Structural Health Monitoring, 2021, 20(4): 2124–2142

[6] Liu F, Liu J, Wang L. Deep learning and infrared thermography for asphalt pavement crack severity classification. Automation in Construction, 2022, 140: 104383

[7] Katsigiannis S, Seyedzadeh S, Agapiou A, Ramzan N. Deep learning for crack detection on masonry façades using limited data and transfer learning. Journal of Building Engineering, 2023, 76: 107105

[8] Russel N S, Selvaraj A. MultiScaleCrackNet: A parallel multiscale deep CNN architecture for concrete crack classification. Expert Systems with Applications, 2024, 249: 123658

[9] Nyathi M A, Bai J, Wilson I D. Deep learning for concrete crack detection and measurement. Metrology, 2024, 4(1): 66–81

[10] Shashidhar R, Manjunath D, Shanmukha S M. CrackSpot: Deep learning for automated detection of structural cracks in concrete infrastructure. Asian Journal of Civil Engineering, 2024, 25(1): 1079–1090

[11] Bhalaji Kharthik K S, Onyema E M, Mallik S, Siva Prasad B V V, Qin H, Selvi C, Sikha O K. Transfer learned deep feature based crack detection using support vector machine: A comparative study. Scientific Reports, 2024, 14(1): 14517

[12] RashidTMokji M MRasheedM. Cracked concrete surface classification in low-resolution images using a convolutional neural network. Journal of Optics, 2024

[13] Rabczuk T, Ren H, Zhuang X. A nonlocal operator method for partial differential equations with application to electromagnetic waveguide problem. Computers, Materials & Continua, 2019, 59(1): 31–55

[14] Samaniego E, Anitescu C, Goswami S, Nguyen-Thanh V M, Guo H, Hamdia K, Zhuang X, Rabczuk T. An energy approach to the solution of partial differential equations in computational mechanics via machine learning: Concepts, implementation and applications. Computer Methods in Applied Mechanics and Engineering, 2020, 362: 112790

[15] Hong G, Chen X, Chen J, Zhang M, Ren Y, Zhang X. A multi-scale gated multi-head attention depthwise separable CNN model for recognizing COVID-19. Scientific Reports, 2021, 11(1): 18048

[16] VaswaniAShazeer NParmarNUszkoreitJJonesL GomezA NKaiser LPolosukhinI. Attention is all you need. In: Advances in neural information processing systems 30 (NIPS 2017). Long Beach, CA: NIPS, 2017, 5998–6008

[17] Cheng S, Liu Y. Research on transportation mode recognition based on multi-head attention temporal convolutional network. Sensors, 2023, 23(7): 3585

[18] DosovitskiyABeyerLKolesnikov AWeissenbornDZhaiXUnterthiner TDehghaniMMindererMHeigoldG GellySUszkoreit JHoulsbyN. An image is worth 16 × 16 words: Transformers for image recognition at scale. 2020, arXiv: 2010.11929

[19] Azad R, Kazerouni A, Heidari M, Aghdam E K, Molaei A, Jia Y, Jose A, Roy R, Merhof D. Advances in medical image analysis with vision transformers: A comprehensive review. Medical Image Analysis, 2024, 91: 103000

[20] Hou S, Liu Y, Yang Q. Real-time prediction of rock mass classification based on TBM operation big data and stacking technique of ensemble learning. Journal of Rock Mechanics and Geotechnical Engineering, 2022, 14(1): 123–143

[21] ReisH CTurk VBozkurtM FYigitS N. Concrete Cracks Image Dataset (CCID). 2024 (available at the website of Mendeley Data v1)

[22] ZophBVasudevan VShlensJLeQ V. Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. Salt Lake City: USA IEEE/CVF, 2018, 8697–8710

[23] SzegedyCVanhoucke VIoffeSShlensJWojnaZ. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition. Las Vegas: USA IEEE/CVF, 2016, 2818–2826

[24] Szegedy C, Ioffe S, Vanhoucke V, Alemi A. Inception-v4, inception-resnet and the impact of residual connections on learning. Proceedings of the AAAI Conference on Artificial Intelligence. San Francisco, CA: AAAI Press, 2017, 31(1): 4278–4284

[25] HowardA GZhu MChenBKalenichenkoDWangW WeyandTAndreetto MAdamH. Mobilenets: Efficient convolutional neural networks for mobile vision applications. 2017, arXiv: 1704.04861v1

[26] SandlerMHoward AZhuMZhmoginovAChenL C. Mobilenetv2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. Salt Lake City: USA IEEE/CVF, 2018, 4510–4520

[27] HuangGLiu ZVan Der MaatenLWeinbergerK Q. Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. Honolulu: USA IEEE/CVF, 2017, 4700–4708

[28] HeKZhangX RenSSunJ. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. Las Vegas: USA IEEE/CVF, 2016, 770–778

[29] HeKZhangX RenSSunJ. Identity mappings in deep residual networks. In: Proceedings of Computer Vision-ECCV 2016. Cham: Springer International Publishing, 2016, 630–645

[30] TanMLeQ. Efficientnet: Rethinking model scaling for convolutional neural networks. 2019, arXiv: 1905.11946

[31] AbadiMBarham PChenJChenZDavisA DeanJDevin MGhemawatSIrvingGIsardM, . TensorFlow: A System for Large-Scale Machine Learning. 2016, arXiv: 1605.08695

[32] KingmaD PBa J. Adam: A method for stochastic optimization. 2014, arXiv: 1412.6980

RIGHTS & PERMISSIONS

Higher Education Press