Pixel-level crack segmentation of tunnel lining segments based on an encoder–decoder network

Abstract

Regular detection and repair for lining cracks are necessary to guarantee the safety and stability of tunnels. The development of computer vision has greatly promoted structural health monitoring. This study proposes a novel encoder–decoder structure, CrackRecNet, for semantic segmentation of lining segment cracks by integrating improved VGG-19 into the U-Net architecture. An image acquisition equipment is designed based on a camera, 3-dimensional printing (3DP) bracket and two laser rangefinders. A tunnel concrete structure crack (TCSC) image data set, containing images collected from a double-shield tunnel boring machines (TBM) tunnel in China, was established. Through data preprocessing operations, such as brightness adjustment, pixel resolution adjustment, flipping, splitting and annotation, 2880 image samples with pixel resolution of 448 × 448 were prepared. The model was implemented by Pytorch in PyCharm processed with 4 NVIDIA TITAN V GPUs. In the experiments, the proposed CrackRecNet showed better prediction performance than U-Net, TernausNet, and ResU-Net. This paper also discusses GPU parallel acceleration effect and the crack maximum width quantification.

Graphical abstract

Keywords

tunnel lining segment / crack detection / semantic segmentation / convolutional neural network / encoder–decoder structure

References

[1] Lei M F, Liu L H, Shi C H, Tan Y, Lin Y X, Wang W D. A novel tunnel-lining crack recognition system based on digital image technology. Tunnelling and Underground Space Technology, 2021, 108: 103724

[2] YangYWang L FZhangY FHanX J. Multi-feature fusion based classification algorithm of surface disease image of concrete structure. Journal of Chang’an University: Natural Science Edition, 2021, 41(3): 64−74 (in Chinese)

[3] Zhao S, Shadabfar M, Zhang D M, Chen J Y, Huang H W. Deep learning-based classification and instance segmentation of leakage-area and scaling images of shield tunnel linings. Structural Control and Health Monitoring, 2021, 28(6): e2732

[4] Yao Y, Tung S T E, Glisic B. Crack detection and characterization techniques—An overview. Structural Control and Health Monitoring, 2014, 21(12): 1387–1413

[5] Liu Y R, Hou S K, Li C Y, Zhou H W, Jin F, Qin P X, Yang Q. Study on support time in double-shield TBM tunnel based on self-compacting concrete backfilling material. Tunnelling and Underground Space Technology, 2020, 96: 103212

[6] Koch C, Georgieva K, Kasireddy V, Akinci B, Fieguth P. A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure. Advanced Engineering Informatics, 2015, 29(2): 196–210

[7] Liu J, Yang X, Lau S, Wang X, Luo S, Lee V C S, Ding L. Automated pavement crack detection and segmentation based on two-step convolutional neural network. Computer-Aided Civil and Infrastructure Engineering, 2020, 35(11): 1291–1305

[8] Valença J, Julio E. MCrack-Dam: the scale-up of a method to assess cracks on concrete dams by image processing. The case study of Itaipu Dam, at the Brazil–Paraguay border. Journal of Civil Structural Health Monitoring, 2018, 8(5): 857–866

[9] Wang Y, Zhang J Y, Liu J X, Zhang Y, Chen Z P, Li C G, Yan R B. Research on crack detection algorithm of the concrete bridge based on image processing. Procedia Computer Science, 2019, 154: 610–616

[10] Ying L, Salari E. Beamlet transform-based technique for pavement crack detection and classification. Computer-Aided Civil and Infrastructure Engineering, 2010, 25(8): 572–580

[11] Abdel-Qader I, Abudayyeh O, Kelly M E. Analysis of edge-detection techniques for crack identification in bridges. Journal of Computing in Civil Engineering, 2003, 17(4): 255–263

[12] OliveiraHCorreia P L. Automatic road crack segmentation using entropy and image dynamic thresholding. In: Proceedings of the 17th European Signal Processing Conference. Glasgow: IEEE, 2009: 622–626

[13] Xu Y, Li D, Xie Q, Wu Q, Wang J. Automatic defect detection and segmentation of tunnel surface using modified Mask R-CNN. Measurement, 2021, 178: 109316

[14] Feng C, Zhang H, Wang H, Wang S, Li Y. Automatic pixel-level crack detection on dam surface using deep convolutional network. Sensors, 2020, 20(7): 2069

[15] Shi Y, Cui L, Qi Z, Meng F, Chen Z. Automatic road crack detection using random structured forests. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(12): 3434–3445

[16] NohYKooD KangY MPark DLeeD. Automatic crack detection on concrete images using segmentation via fuzzy C-means clustering. In: Proceedings of the International conference on applied system innovation (ICASI). Sapporo: IEEE, 2017: 877–880

[17] Li G, Zhao X, Du K, Ru F, Zhang Y. Recognition and evaluation of bridge cracks with modified active contour model and greedy search-based support vector machine. Automation in Construction, 2017, 78: 51–61

[18] Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Li F F. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 2015, 115(3): 211–252

[19] Park S, Bang S, Kim H, Kim H. Patch-based crack detection in black box images using convolutional neural networks. Journal of Computing in Civil Engineering, 2019, 33(3): 04019017

[20] ZhangCChang C CJamshidiM. Simultaneous pixel-level concrete defect detection and grouping using a fully convolutional model. Structural Health Monitoring, 2021, 20(4): 147592172098543

[21] Cha Y J, Choi W, Suh G, Mahmoudkhani S, Büyüköztürk O. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(9): 731–747

[22] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015, 521(7553): 436–444

[23] Chen J, He Y. A novel U-shaped encoder–decoder network with attention mechanism for detection and evaluation of road cracks at pixel level. Computer-Aided Civil and Infrastructure Engineering, 2022, 37(13): 1721–1736

[24] Zhou Z, Zhang J, Gong C. Hybrid semantic segmentation for tunnel lining cracks based on Swin Transformer and convolutional neural network. Computer-Aided Civil and Infrastructure Engineering, 2023, 38(17): 2491–2510

[25] XuYFanY L LiH. Lightweight semantic segmentation of complex structural damage recognition for actual bridges. Structural Health Monitoring, 2023, 22(5): 14759217221147015

[26] ArafinPBillah A MIssaA. Deep learning-based concrete defects classification and detection using semantic segmentation. Structural Health Monitoring, 2023: 14759217231168212

[27] Chen L, Yao H, Fu J, Ng C T. The classification and localization of crack using lightweight convolutional neural network with CBAM. Engineering Structures, 2023, 275: 115291

[28] ZhangLYang FZhangY DZhuY J. Road crack detection using deep convolutional neural network. In: Proceedings of the 2016 IEEE international conference on image processing (ICIP). Phoenix: IEEE, 2016: 3708–3712

[29] Zou Q, Zhang Z, Li Q, Qi X, Wang Q, Wang S. Deepcrack: Learning hierarchical convolutional features for crack detection. IEEE Transactions on Image Processing, 2018, 28(3): 1498–1512

[30] Bang S, Park S, Kim H, Kim H. Encoder-decoder network for pixel-level road crack detection in black-box images. Computer-Aided Civil and Infrastructure Engineering, 2019, 34(8): 713–727

[31] O’Brien D, Osborne J A, Perez-Duenas E, Cunningham R, Li Z L. Automated crack classification for the CERN underground tunnel infrastructure using deep learning. Tunnelling and Underground Space Technology, 2023, 131: 104668

[32] Dang L M, Wang H X, Li Y F, Park Y, Oh C, Nguyen T N, Moon H. Automatic tunnel lining crack evaluation and measurement using deep learning. Tunnelling and Underground Space Technology, 2022, 124: 104472

[33] Feng S J, Feng Y, Zhang X L, Chen Y H. Deep learning with visual explanations for leakage defect segmentation of metro shield tunnel. Tunnelling and Underground Space Technology, 2023, 136: 105107

[34] Xue Y D, Jia F, Cai X Y, Shadabfar M, Huang H W. An optimization strategy to improve the deep learning-based recognition model of leakage in shield tunnels. Computer-Aided Civil and Infrastructure Engineering, 2022, 37(3): 386–402

[35] ManjunathaPMasri S FNakanoAWellfordL C. CrackDenseLinkNet: A deep convolutional neural network for semantic segmentation of cracks on concrete surface images. Structural Health Monitoring, 2023: 14759217231173305

[36] Que Y, Dai Y, Ji X, Leung A K, Chen Z, Jiang Z L, Tang Y C. Automatic classification of asphalt pavement cracks using a novel integrated generative adversarial networks and improved VGG model. Engineering Structures, 2023, 277: 115406

[37] Guo L, Li R, Jiang B, Shen X. Automatic crack distress classification from concrete surface images using a novel deep-width network architecture. Neurocomputing, 2020, 397: 383–392

[38] Xu H, Su X, Wang Y, Cai H, Cui K, Chen X. Automatic bridge crack detection using a convolutional neural network. Applied Sciences, 2019, 9(14): 2867

[39] Huyan J, Li W, Tighe S, Xu Z C, Zhai J Z. CrackU-net: A novel deep convolutional neural network for pixelwise pavement crack detection. Structural Control and Health Monitoring, 2020, 27(8): e2551

[40] Zhang K, Cheng H D, Zhang B. Unified approach to pavement crack and sealed crack detection using preclassification based on transfer learning. Journal of Computing in Civil Engineering, 2018, 32(2): 04018001

[41] HouS KOu Z GQinP XWangY LLiuY R. Image-based crack recognition of tunnel lining using residual U-Net convolutional neural network. In: Proceedings of the IOP Conference Series: Earth and Environmental Science. Jakarta: IOP Publishing, 2021: 072001

[42] Attard L, Debono C J, Valentino G, Di Castro M. Tunnel inspection using photogrammetric techniques and image processing: A review. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 144: 180–188

[43] Ai Q, Yuan Y, Bi X L. Acquiring sectional profile of metro tunnels using charge-coupled device cameras. Structure and Infrastructure Engineering, 2016, 12(9): 1065–1075

[44] Zhao S, Zhang D M, Huang H W. Deep learning-based image instance segmentation for moisture marks of shield tunnel lining. Tunnelling and Underground Space Technology, 2020, 95: 103156

[45] Ai Q, Yuan Y. Rapid acquisition and identification of structural defects of metro tunnel. Sensors, 2019, 19(19): 4278

[46] Zhou M L, Cheng W, Huang H W, Chen J Y. A novel approach to automated 3D spalling defects inspection in railway tunnel linings using laser intensity and depth information. Sensors, 2021, 21(17): 5725

[47] IglovikovVShvets A. Ternausnet: U-net with vgg11 encoder pre-trained on imagenet for image segmentation. 2018, arXiv:1801.05746

[48] BenzCDebus PHaH KRodehorstV. Crack segmentation on UAS-based imagery using transfer learning. In: Proceedings of the 2019 International Conference on Image and Vision Computing New Zealand (IVCNZ). Dunedin: IEEE, 2019: 1–6

[49] JoginMMadhulika M SDivyaG DMeghanaR KApoorvaS. Feature extraction using convolution neural networks (CNN) and deep learning. In: Proceedings of the 2018 3rd IEEE international conference on recent trends in electronics, information & communication technology (RTEICT). Bangalore: IEEE, 2018: 2319–2323

[50] SimonyanKZisserman A. Very deep convolutional networks for large-scale image recognition. 2014, arXiv:1409.1556

[51] NairVHinton G E. Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10). Madison, WI: ACM, 2010: 807–814

[52] Swiderska-Chadaj Z, de Bel T, Blanchet L, Baidoshvili A, Vossen D, van der Laak J, Litjens G. Impact of rescanning and normalization on convolutional neural network performance in multi-center, whole-slide classification of prostate cancer. Scientific Reports, 2020, 10(1): 1–14

[53] IoffeSSzegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. International conference on machine learning. Stockholm: ICML, 2015: 448–456

[54] WuYHeK. Group normalization. In: Proceedings of the European conference on computer vision (ECCV), Munich: Springer, 2018: 3–19

[55] RonnebergerOFischerPBroxT. U-Net: convolutional Networks for biomedical image segmentation. In: Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munish: Springer, 2015: 234–241

[56] Deng L Y. The Cross-Entropy method: a unified approach to combinatorial optimization, Monte-Carlo simulation, and machine learning. Technometrics, 2006, 48(1): 147–148

[57] Qu Z, Mei J, Liu L, Zhou D Y. Crack detection of concrete pavement with cross-entropy loss function and improved VGG16 network model. IEEE Access : Practical Innovations, Open Solutions, 2020, 8: 54564–54573

[58] Li C Y, Hou S K, Liu Y R, Qin P X, Yang Q. Analysis on the crown convergence deformation of surrounding rock for double-shield TBM tunnel based on advance borehole monitoring and inversion analysis. Tunnelling and Underground Space Technology, 2020, 103: 103513

[59] Wang W J, Su C. Semi-supervised semantic segmentation network for surface crack detection. Automation in Construction, 2021, 128: 103786

[60] Bejani M M, Ghatee M. A systematic review on overfitting control in shallow and deep neural networks. Artificial Intelligence Review, 2021, 54(8): 1–48

[61] Shorten C, Khoshgoftaar T M. A survey on image data augmentation for deep learning. Journal of Big Data, 2019, 6(1): 1–48

[62] Yang X B, Chen R, Zhang F Q, Zhang L, Fan X J, Ye Q L, Fu L Y. Pixel-level automatic annotation for forest fire image. Engineering Applications of Artificial Intelligence, 2021, 104: 104353

[63] Coelho F, Neto J P. A method for regularization of evolutionary polynomial regression. Applied Soft Computing, 2017, 59: 223–228

[64] Ohsaki M, Wang P, Matsuda K, Katagiri S, Watanabe H, Ralescu A. Confusion-matrix-based kernel logistic regression for imbalanced data classification. IEEE Transactions on Knowledge and Data Engineering, 2017, 29(9): 1806–1819

[65] Ren Y, Huang J, Hong Z, Lu W, Yin J, Zou L, Shen X. Image-based concrete crack detection in tunnels using deep fully convolutional networks. Construction & Building Materials, 2020, 234: 117367

[66] Sokolova M, Lapalme G. A systematic analysis of performance measures for classification tasks. Information Processing & Management, 2009, 45(4): 427–437

[67] Liu Z Q, Wang Y Z, Hua X G, Zhu H P, Zhu Z W. Optimization of wind turbine TMD under real wind distribution countering wake effects using GPU acceleration and machine learning technologies. Journal of Wind Engineering and Industrial Aerodynamics, 2021, 208: 104436

[68] Dong S, Zhao P, Lin X, Kaeli D. Exploring GPU acceleration of Deep Neural Networks using Block Circulant Matrices. Parallel Computing, 2020, 100: 102701

[69] Wang J J, Liu Y F, Nie X, Mo Y L. Deep convolutional neural networks for semantic segmentation of cracks. Structural Control and Health Monitoring, 2022, 29(1): e2850

[70] RongY. The research on the key technology of the crack controlling of reinforced concrete lining of undersea tunnel. Dissertation for the Doctoral Degree. Shanghai: Toingji University, 2007 (in Chinese)

[71] Yang X C, Li H, Yu Y T, Luo X C, Huang T, Yang X. Automatic pixel-level crack detection and measurement using fully convolutional network. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(12): 1090–1109

[72] SunFChoiY K YuYWangW. Medial meshes for volume approximation. 2013, arXiv:1308.3917

[73] Huang H W, Li Q T, Zhang D M. Deep learning based image recognition for crack and leakage defects of metro shield tunnel. Tunnelling and Underground Space Technology, 2018, 77: 166–176

RIGHTS & PERMISSIONS

Higher Education Press