High-precision segmentation and quantification of tunnel lining crack using an improved DeepLabV3+

Abstract

Current semantic segmentation models have limitations in addressing tunnel lining crack, such as high complexity, misidentification, or inability to detect tiny cracks in specific practical scenarios, which is crucial for precise assessment of tunnel lining health. We developed a novel approach called EDeepLab, aiming to achieve a higher precision detection and segmentation of lining surface crack. EDeepLab improves upon the original DeepLabV3+ framework by replacing its backbone network with an optimized lightweight EfficientNetV2. The amount of EfficientNetV2 block computation is reduced and a self-designed shallow feature fusion module is used to merge the layers to enhance parameter utilization efficiency. Furthermore, the normalization-based attention module and convolutional block attention module attention mechanisms are integrated to classify and process both high and low dimensional information features. This allows for comprehensive utilization of global semantic information and channel information, thereby enhancing the model’s feature extraction capability. Results in constructed metro-tunnel crack dataset demonstrate that the number of parameters is reduced from 144.45 M in the DeepLabV3+ to 99.80 M in the EDeepLab. EDeepLab achieves a mean intersection over union of 84.77%, mean pixel accuracy of 94.96%, and frames per second of 18.52 f/s. The proposed EDeepLab outperforms other models including U-Net, ResNet and fully convolutional networks in the quantitative analysis of tiny cracks and noise interference.

Keywords

Tunnel lining / Deep learning / Surface crack / DeepLabV3+

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

CRediT authorship contribution statement

Zhutian Pan: Writing - review & editing, Writing - original draft, Conceptualization. Xuepeng Zhang: Conceptualization. Yujing Jiang: Validation, Conceptualization. Bo Li: Formal analysis, Conceptualization. Naser Golsanami: Methodology, Conceptualization. Hang Su: Visualization. Yue Cai: Conceptualization.

Declaration of competing interest

Yujing Jiang is an editorial board member for Underground Space and was not involved in the editorial review or the decision to publish this article. All authors declare that there are no competing interests.

Acknowledgement

This work was funded by the National Natural Science Foundation of China (Grant No. 52109132), and Shandong Provincial Natural Science Foundation (Grant No. ZR2020QE270).

References

[1] Ali, R., & Cha, Y. J. (2022). Attention-based generative adversarial network with internal damage segmentation using thermography. Automation in Construction, 141, 104412.

[2] Amjoud, A. B., & Amrouch, M. (2023). Object detection using deep learning, CNNs and vision transformers: a review. IEEE Access, 11, 35479-35516.

[3] Chen, L. C., Zhu, Y. K., Papandreou, G., Schroff, F., & Adam, H. (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European Conference on Computer Vision (ECCV 2018) (pp. 833-851).

[4] Chongqing Transport Commission (2015). JTG H12-2015: Technical specifications of maintenance for highway tunnel. Beijing, China: China Communications Press (in Chinese).

[5] Editorial Department of China Journal of Highway and Transport (2022). Review on China's Traffic Tunnel Engineering Research: 2022. China Journal of Highway and Transport, 35(4), 1-40 (in Chinese).

[6] Feng, C. C., Zhang, H., Wang, H. R., Wang, S., & Li, Y. L. (2020). Automatic pixel-level crack detection on dam surface using deep convolutional network. Sensors, 20(7), 2069.

[7] Gao, H. J., He, P., Chen, Z., & Li, X. Y. (2019). Study on a surrounding rock pressure calculation method for super-large section highway tunnels. Symmetry, 11(9), 1133.

[8] Garcia-Garcia, A., Orts-Escolano, S., Oprea, S. O., Villena-Martinez, V., & Garcia-Rodriguez, J. (2017). A review on deep learning techniques applied to semantic segmentation. preprint. https://arxiv.org/abs/1704.06857.

[9] He, K. M., Zhang, X. Y., Ren, S. Q., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (pp.770-778).

[10] He, K. M., Zhang, X. Y., Ren, S. Q., & Sun, J. (2015). Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9), 1904-1916.

[11] He, S. Q., Xiao, B., Wei, H. J., Huang, S. J., & Chen, T. S. (2023). SVM classifier of cervical histopathology images based on texture and morphological features. Technology and Health Care, 31(1), 69-80.

[12] Hu, H. B., Hu, X. J., & Gong, X. N. (2024). Predicting the strut forces of the steel supporting structure of deep excavation considering various factors by machine learning methods. Underground Space, 18, 114-129.

[13] Jang, K., An, Y. K., Kim, B., & Cho, S. (2021). Automated crack evaluation of a high-rise bridge pier using a ring-type climbing robot. Computer-Aided Civil and Infrastructure Engineering, 36(1), 14-29.

[14] Jiang, Y. J., & Zhang, X. P. (2021). Research on Automatic Detection and Health Assessment of Tunnel Lining. Tunnel Construction, 41(3), 341-348 (in Chinese).

[15] Kang, Q., Chen, E. J., Li, Z. C., Luo, H. B., & Liu, Y. (2023). Attentionbased LSTM predictive model for the attitude and position of shield machine in tunneling. Underground Space, 13, 335-350.

[16] Kong, X. X., & Li, J. (2018). Automated fatigue crack identification through motion tracking in a video stream. In Proceedings of Conference on Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems. (Vol. 10598, pp. 212-219). Denver, Colorado, United States.

[17] König, J., Jenkins, M. D., Mannion, M., Barrie, P., & Morison, G. (2022). hat's Cracking? A Review and Analysis of Deep Learning Methods for Structural Crack Segmentation, Detection and Quantification. preprint. https://arxiv.org/abs/2202.03714.

[18] Lei, M. F., Liu, L. H., Shi, C. H., Tan, Y., Lin, Y. X., & Wang, W. D. (2021). A novel tunnel-lining crack recognition system based on digital image technology. Tunnelling and Underground Space Technology, 108, 103724.

[19] Liang, R. Z., Xia, T. D., Huang, M. S., & Lin, C. G. (2017). Simplified analytical method for evaluating the effects of adjacent excavation on shield tunnel considering the shearing effect. Computers and Geotechnics, 81, 167-187.

[20] Liao, J. H., Yue, Y. H., Zhang, D. J., Tu, W., Cao, R., Zou, Q., & Li, Q. Q. (2022). Automatic tunnel crack inspection using an efficient mobile imaging module and a lightweight CNN. IEEE Transactions on Intelligent Transportation Systems, 23(9), 15190-15203.

[21] Liu, F. Y., Ding, W. Q., Qiao, Y. F., & Wang, L. B. (2024). Transfer learning-based encoder-decoder model with visual explanations for infrastructure crack segmentation: new open database and comprehensive evaluation. Underground Space, 17, 60-81.

[22] Liu, Y. D., Wang, Y. T., Wang, S. W., Liang, T. T., Zhao, Q. J., Tang, Z., & Ling, H. B. (2020). CBNet: a novel composite backbone network architecture for object detection. Proceedings of the AAAI Conference on Artificial Intelligence, 34(7), 11653-11660.

[23] Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 3431-3440).

[24] Mei, R. G., Wang, T. X., Qian, S. P., Zhang, H. H., & Yan, X. H. (2022). Information Mining from Images of Pipeline Based on Knowledge Representation and Reasoning. In Proceedings of the 18th International Conference on Advanced Data Mining and Applications (ADMA 2022) (pp. 142-156).

[25] Merazi-Meksen, T., Boudraa, M., & Boudraa, B. (2014). Mathematical morphology for TOFD image analysis and automatic crack detection. Ultrasonics, 54(6), 1642-1648.

[26] Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. In Proceedings of the International Conference on Medical Image Computing and ComputerAssisted Intervention (MICCAI 2015) (pp. 234-241).

[27] Shahrour, I., Bian, H. B., Xie, X. Y., & Zhang, Z. X. (2021). Smart technology applications for the optimal management of underground facilities. Underground Space, 6(5), 551-559.

[28] Shangguan, Y. L., Wang, Q., Niu, C. C., Wang, K., & Zhang, Y. G. (2012). Engineering Geological Condition Analysis of some Beijing Engineering. Advanced Materials Research, 446-449, 1989-1992.

[29] Shafait, F., Keysers, D., & Breuel, T. M. (2008). Efficient implementation of local adaptive thresholding techniques using integral images. In Proceedings of the 15th Document Recognition and Retrieval Conference (DRR-2008). January 26-31, San Jose, CA, USA.

[30] Somasundaram, K., & Kalavathi, P. (2014). Brain segmentation in magnetic resonance human head scans using multi-seeded region growing. The Imaging Science Journal, 62(5), 273-284.

[31] Sunnetci, K. M., Kaba, E., Çeliker, F. B., & Alkan, A. (2023). Comparative parotid gland segmentation by using ResNet-18 and MobileNetV2 based DeepLab v3+ architectures from magnetic resonance images. Concurrency and Computation: Practice and Experience, 35(1), e7450.

[32] Peng, H. X., Zhong, J. R., Liu, H. N., Li, J., Yao, M. W., & Zhang, X. (2023). ResDense-focal-DeepLabV3+enabled litchi branch semantic segmentation for robotic harvesting. Computers and Electronics in Agriculture, 206, 107691.

[33] Tan, M. X., & Le, Q. V. (2021). EfficientNetV2:Smaller models and faster training. In Proceedings of the 38th International Conference on Machine Learning (PMLR) (pp. 10096-10106).

[34] Tichý T., Brož J., Bělinová Z., & Pirník, R. (2021). Analysis of predictive maintenance for tunnel systems. Sustainability, 3(7), 3977.

[35] Wan, C. F., Xiong, X. B., Wen, B., Gao, S., Fang, D., Yang, C. Q., & Xue, S. T. (2022). Crack detection for concrete bridges with imaged based deep learning. Science Progress, 105(4), 1-21.

[36] Woo, S., Park, J., Lee, J. Y., & Kweon, I. S. (2018). CBAM: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV 2018) (pp. 3-19).

[37] Xi, D. J., Qin, Y., & Wang, Z. W. (2022). Attention Deeplabv3 model and its application into gear pitting measurement. Journal of Intelligent & Fuzzy Systems, 42(4), 3107-3120.

[38] Xiong, L. J., Zhang, D. L., & Zhang, Y. (2020). Water leakage image recognition of shield tunnel via learning deep feature representation. Journal of Visual Communication and Image Representation, 71, 102708.

[39] Xu, H. Y., Su, X., Wang, Y., Cai, H. Y., Cui, K. R., & Chen, X. D. (2019). Automatic bridge crack detection using a convolutional neural network. Applied Sciences, 9(14), 2867.

[40] Xu, L. Z., Wang, Y. D., Dong, A. Q., Zhu, L. Q., Shi, H. M., & Yu, Z. J. (2023). Image-based intelligent detection of typical defects of complex subway tunnel surface. Tunnelling and Underground Space Technology, 140, 105266.

[41] Xue, Y. D., Jia, F., Cai, X. Y., Shadabfar, M., & Huang, H. W. (2021). An optimization strategy to improve the deep learning-based recognition model of leakage in shield tunnels. Computer-Aided Civil and Infrastructure Engineering, 37(3), 386-402.

[42] Xue, Y. D., Shi, P. Z., Jia, F., & Huang, H. W. (2022). 3D reconstruction and automatic leakage defect quantification of metro tunnel based on SfM-Deep learning method. Underground Space, 7(3), 311-323.

[43] Yang, W. C., Yin, R. S., Lei, M. F., Deng, E., & Zhang, P. P. (2018). Vibratory influential zoning for grade-separated tunnels under the load of trains. Geotechnical and Geological Engineering, 36, 723-735.

[44] Ye, Z. J., Yang, Y. Y., & Ye, Y. (2023). Three-dimensional effects of multiple voids behind lining on the mechanical behavior of tunnel structure. Ain Shams Engineering Journal, 14(4), 101949.

[45] Yuan, Y., Jiang, X. M., & Liu, X. (2013). Predictive maintenance of shield tunnels. Tunneling and Underground Space Technology, 38, 69-86.

[46] Zhang, B. W., & Liu, Y. J. (2022). Sub-pixel Level Edge Extraction Technology for Industrial Parts for Smart Manufacturing. In International Conference on Machine Learning for Cyber Security (ML4CS 2022) (pp. 469-483).

[47] Zhao, S., Zhang, D. M., Xue, Y. D., Zhou, M. L., & Huang, H. W. (2021). A deep learning-based approach for refined crack evaluation from shield tunnel lining images. Automation in Construction, 132, 103934.

[48] Zhao, Z. C., Xie, Y. L., Yang, X. H., & Li, Y. Y. (2004). Observation research on the mechanical characteristic of highway tunnel lining in loess. China Journal of Highway and Transport, 17(1), 66-69 (in Chinese).

[49] Zhou, S. L., Canchila, C., & Song, W. (2023). Deep learning-based crack segmentation for civil infrastructure: data types, architectures, and benchmarked performance. Automation in Construction, 146, 104678.

[50] Zhou, Z., Yan, L. B., Zhang, J. J., Zhang, Y. D., Gong, C. J., Yang, H., & Deng, E. (2023a). Automatic segmentation of tunnel lining defects based on multiscale attention and context information enhancement. Construction and Building Materials, 387, 131621.

[51] Zhou, Z., Zhang, J. J., & Gong, C. J. (2023b). Hybrid semantic segmentation for tunnel lining cracks based on Swin Transformer and convolutional neural network. Computer-Aided Civil and Infrastructure Engineering, 38(17), 2491-2510.

[52] Zhou, Z., Zhang, J. J., Gong, C. J., & Wu, W. (2023c). Automatic tunnel lining crack detection via deep learning with generative adversarial network-based data augmentation. Underground Space, 9, 140-154.