Automated classification of civil structure defects based on convolutional neural network

Abstract

Today, the most commonly used civil infrastructure inspection method is based on a visual assessment conducted by certified inspectors following prescribed protocols. However, the increase in aggressive environmental and load conditions, coupled with the achievement of many structures of the life-cycle end, has highlighted the need to automate damage identification and satisfy the number of structures that need to be inspected. To overcome this challenge, this paper presents a method for automating concrete damage classification using a deep convolutional neural network. The convolutional neural network was designed after an experimental investigation of a wide number of pretrained networks, applying the transfer-learning technique. Training and validation were conducted using a database built with 1352 images balanced between “undamaged”, “cracked”, and “delaminated” concrete surfaces. To increase the network robustness compared to images in real-world situations, different image configurations have been collected from the Internet and on-field bridge inspections. The GoogLeNet model, with the highest validation accuracy of approximately 94%, was selected as the most suitable network for concrete damage classification. The results confirm that the proposed model can correctly classify images from real concrete surfaces of bridges, tunnels, and pavement, resulting in an effective alternative to the current visual inspection techniques.

Graphical abstract

Keywords

concrete structure / infrastructures / visual inspection / convolutional neural network / artificial intelligence

References

[1] Anitescu C, Atroshchenko E, Alajlan N, Rabczuk T. Artificial neural network methods for the solution of second order boundary value problems. Computers, Materials & Continua, 2019, 59(1): 345–359

[2] Guo H, Zhuang X, Rabczuk T. A deep collocation method for the bending analysis of Kirchhoff plate. Computers, Materials & Continua, 2019, 59(2): 433–456

[3] Fan Z, Wu Y, Lu J, Li W. Automatic pavement crack detection based on structured prediction with the convolutional neural network. arXiv preprint arXiv: 1802.02208, 2018

[4] Zhang L, Yang F, Daniel Zhang Y, Zhu Y J. Road crack detection using deep convolutional neural network. In: IEEE International Conference on Image Processing 2016. Phoenix, AZ: ICIP, 2016, 3708–3712

[5] Kim B, Cho S. Automated vision-based detection of cracks on concrete surfaces using a deep learning technique. Sensors (Basel), 2018, 18(10): 3452–3469

[6] Hung P, Su N, Diep V. Surface classification of damaged concrete using deep convolutional neural network. Pattern Recognition and Image Analysis, 2019, 29(4): 676–687

[7] Zhu J, Song J. An intelligent classification model for surface defects on cement concrete bridges. Applied Sciences (Basel, Switzerland), 2020, 10(3): 972–990

[8] Feng C, Zhang H, Wang S, Li Y, Wang H, Yan F. Structural damage detection using deep convolutional neural network and transfer learning. KSCE Journal of Civil Engineering, 2019, 23(10): 4493–4502

[9] Song Q, Wu Y, Xin X, Yang L, Yang M, Chen H, Liu C, Hu M, Chai X, Li J. Real-time tunnel crack analysis system via deep learning. IEEE Access: Practical Innovations, Open Solutions, 2019, 7: 64186–64197

[10] Makantasis K, Protopapadakis E, Doulamis A, Doulamis N, Loupos C. Doulamis Nikolaos. Deep convolutional neural networks for efficient vision based tunnel inspection. In: IEEE International Conference on Intelligent Computer Communication and Processing 2015. Cluj-Napoca: Romania ICCP, 2015, 335–342

[11] Patterson B, Leone G, Pantoja M, Behrouzi A. Deep learning for automated image classification of seismic damage to built infrastructure. In: Proceedings of the 11th National Conference in Earthquake Engineering 2018. Los Angeles, CA: Earthquake Engineering Research Institute, 2018

[12] Gulgec N S, Takac M, Pakzad S N. Structural damage detection using convolutional neural networks. In: Barthorpe R, Platz R, Lopez I, et al., eds. Model Validation and Uncertainty Quantification. Conference Proceedings of the Society for Experimental Mechanics Series. Cham: Springer, 2017, 331–337

[13] Cha Y J, Choi W, Suh G, Mahmoudkhani S, Büyüköztürk O. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(9): 731–747

[14] Soukup D, Huber-Mörk R. Convolutional neural networks for steel surface defect detection from photometric stereo images. In: Bebis G, et al., eds. Advances in Visual Computing. ISVC 2014. Lecture Notes in Computer Science. Cham: Springer, 2014, 668–677

[15] Li J, Su Z, Geng J, Yin Y. Real-time detection of steel strip surface defects based on improved YOLO detection network. IFAC-PapersOnLine, 2018, 51(21): 76–81

[16] Rawat W, Wang Z. Deep convolutional neural networks for image classification: A comprehensive review. Neural Computation, 2017, 29(9): 1–98

[17] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015, 521(7553): 436–444

[18] Nair V, Hinton G E. Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning 2010. Haifa: Israel ICML-10, 2010, 807–814

[19] Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg A C, Fei F L. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 2015, 115(3): 211–252

[20] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. Communications of the ACM, 2017, 60(6): 84–90

[21] Iandola F N, Han S, Moskewicz M W, Ashraf K, Dally W J, Keutzer K. SqueezeNet: AlexNet-level accuracy with 50X fewer parameters and<0.5 MB model size. 2017, arxiv.org/abs/1602.07360

[22] Zhang X, Zhou X, Lin M, Sun J. ShuffleNet: An extremely efficient convolutional neural network for mobile devices. In: Conference on Computer Vision and Pattern Recognition 2018. Salt Lake City: USA IEEE/CVF, 2018, 6848–6856

[23] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2016. Las Vegas, 2016, 770–778.

[24] Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: USA IEEE, 2015, 1–9

[25] Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L C. MobileNetV2: Inverted residuals and linear bottlenecks. In: Conference on Computer Vision and Pattern Recognition 2018. Salt Lake City: USA IEEE/CVF, 2018, 4510–4520

[26] Howard A G, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H. MobileNets: Efficient convolutional neural networks for mobile vision applications. 2017, arXiv:1704.04861

[27] Zoph B, Vasudevan V, Shlens J, Le Q V. Learning transferable architectures for scalable image recognition. In: Conference on Computer Vision and Pattern Recognition 2018. Salt Lake City: USA IEEE/CVF, 2018, 8697–8710

RIGHTS & PERMISSIONS

2021 Higher Education Press