Research on hybrid segmentation method and application of multi-category defect detection in tunnel linings based on deep learning

Abstract

During tunnel construction and operation, the timely detection and treatment of lining defects are critical to ensuring the safety and stability of tunnel engineering. To achieve fast and accurate detection of multi-category tunnel lining defects from image data, this paper proposes an improved model based on the KNet segmentation algorithm, integrated with the Swin Transformer and UPerNet models. First, a large-scale multi-category image dataset of tunnel lining defects is constructed by collecting and organizing data from relevant literature and public datasets. Second, the pre-trained KNet-enhanced model is trained on this dataset using transfer learning to improve detection accuracy and generalization performance. The proposed KNet-UPerNet-SwinL model achieves an average improvement of approximately 16.94 percentage points in mean Intersection over Union (mIoU) compared to other classical semantic segmentation methods, demonstrating superior segmentation accuracy and stability. In addition, an in-depth discussion of the model’s decision-making process and the factors influencing its performance is provided through visually interpretable progressive feature analysis and multi-index joint analysis at the application level. Finally, to facilitate practical application, the model is converted into the Open Neural Network Exchange (ONNX) universal format, and a desktop application for tunnel lining defect detection is developed. The detection and recognition of lining defects are successfully implemented in the experimental sections of the Central Yunnan Water Diversion Project, achieving remarkable segmentation results. In summary, this study provides a comprehensive solution that offers practical technical and tool support for multi-category defect detection in tunnel linings and holds significant importance for the safety and stability of tunnel engineering.

Keywords

Tunnel lining defects / Large-scale dataset / Defect segmentation detection / Visual interpretability analysis / Engineering application

References

[1] Y. Xu, D. Li, Q. Xie, Q. Wu, J. Wang, Automatic defect detection and segmentation of tunnel surface using modified Mask R—CNN, Measurement 178 (2021) 109316, doi: 10.1016/j.measurement.2021.109316.

[2] J. Fu, J. Xie, S. Wang, J. Yang, F. Yang, H. Pu, Cracking performance of an operational tunnel lining due to local construction defects, Int. J. Geomech 19 (2019) 04019019, doi: 10.1061/(ASCE)GM.1943—5622.0001371.

[3] F. Ye, N. Qin, X. Liang, A. Ouyang, Z. Qin, E. Su, Analyses of the defects in highway tunnels in China, Tunn. Undergr. Space Technol. 107 (2021) 103658, doi: 10.1016/j.tust.2020.103658.

[4] Z. Zhou, J. Zhang, C. Gong, Automatic detection method of tunnel lining multidefects via an enhanced You Only Look Once network, Comput.—Aided Civ. Infrastruct. Eng. 37 (2022) 762-780, doi: 10.1111/mice.12836.

[5] C. Gong, W. Ding, K.M. Mosalam, S. Günay, K. Soga, Comparison of the structural behavior of reinforced concrete and steel fiber reinforced concrete tunnel segmental joints, Tunn. Undergr. Space Technol. 68 (2017) 38-57, doi: 10.1016/j.tust.2017.05.010.

[6] S. Zhao, D. Zhang, Y. Xue, M. Zhou, H. Huang, A deep learning—based approach for refined crack evaluation from shield tunnel lining images, Autom. Constr. 132 (2021) 103934, doi: 10.1016/j.autcon.2021.103934.

[7] R. Liu, M. Vail, B. Koohbor, C. Zhu, C.S. Tang, H. Xu, X.C. Shi, Desiccation cracking in clay—bottom ash mixtures: insights from crack image analysis and digital image correlation, Bull. Eng. Geol. Environ. 81 (2022) 139, doi: 10.1007/s10064—022—02631—0.

[8] S. Zhao, D.M. Zhang, H.W. Huang, Deep learning—based image instance segmentation for moisture marks of shield tunnel lining, Tunn. Undergr. Space Technol. 95 (2020) 103156, doi: 10.1016/j.tust.2019.103156.

[9] Z. Zhou, J. Zhang, C. Gong, H. Ding, Automatic identification of tunnel leakage based on deep semantic segmentation, Chin. J. Rock Mech. Eng. 41 (2022) 2082-2093, doi: 10.13722/j.cnki.jrme.2022.0016.

[10] Q. Chen, C. Xiong, W. Lv, B. Shen, B. Zeng, J. Li, C. Feng, Z. Hu, F. Zhu, Convolutional neural network with attention module for identification of tunnel seepage, Transp. Res. Rec. 2676 (2022) 112-123, doi: 10.1177/03611981221091774.

[11] J. Liao, Y. Yue, D. Zhang, W. Tu, R. Cao, Q. Zou, Q. Li, Automatic tunnel crack inspection using an efficient mobile imaging module and a lightweight CNN, IEEE Trans. Intell. Transp. Syst. 23 (2022) 15190-15203, doi: 10.1109/TITS.2021.3138428.

[12] F. Liu, L. Wang, UNet—based model for crack detection integrating visual explanations, Constr. Build. Mater. 322 (2022) 126265, doi: 10.1016/j.conbuildmat.2021.126265.

[13] Y. Ren, J. Huang, Z. Hong, W. Lu, J. Yin, L. Zou, X. Shen, Image—based concrete crack detection in tunnels using deep fully convolutional networks, Constr. Build. Mater. 234 (2020) 117367, doi: 10.1016/j.conbuildmat.2019.117367.

[14] R. Battulwar, E. Emami, M. Zare, K. Battulwar, M. Shahsavar, A. Moniri—Morad, J. Sattarvand, Utilizing deep learning for the automated extraction of rock mass features from point clouds, Geotech. Geol. Eng. 42 (2024) 6179-6194, doi: 10.1007/s10706—024—02886—2.

[15] N.S. Russel, A. Selvaraj, MultiScaleCrackNet: A parallel multiscale deep CNN architecture for concrete crack classification, Expert Syst. Appl. 249 (2024) 123658, doi: 10.1016/j.eswa.2024.123658.

[16] Y. Ren, H. Zhang, L. Dong, H. Yang, H. Li, L. Du, Q. Chen, S. Li, DAM—CGNet: Semantic segmentation—based approach for valley—bottom extraction from digital elevation models, Comput. Geosci. 205 (2025) 106012, doi: 10.1016/j.cageo.2025.106012.

[17] J. Liu, Z. Zhao, C. Lv, Y. Ding, H. Chang, Q. Xie, An image enhancement algorithm to improve road tunnel crack transfer detection, Constr. Build. Mater. 348 (2022) 128583, doi: 10.1016/j.conbuildmat.2022.128583.

[18] F. Wei, G. Yao, Y. Yang, Y. Sun, Instance—level recognition and quantification for concrete surface bughole based on deep learning, Autom. Constr. 107 (2019) 102920, doi: 10.1016/j.autcon.2019.102920.

[19] J. Cui, S. Wu, H. Cheng, G. Kui, H. Zhang, M. Hu, P. He, Composite interpretability optimization ensemble learning inversion surrounding rock mechanical parameters and support optimization in soft rock tunnels, Comput. Geotech. 165 (2024) 105877, doi: 10.1016/j.compgeo.2023.105877.

[20] J. Cui, S. Wu, H. Cheng, T. Wang, G. Jiang, S. Pu, Z. Ren, Time series prediction of the surrounding rock displacement of a soft rock tunnel in the central yunnan water diversion project, J. Tsinghua Univ. (Sci. Technol.) 64 (2024) 1215-1225, doi: 10.16511/j.cnki.qhdxxb.2024.26.031.

[21] S. Pu, X. Yang, S. Wu, H. Cheng, T. Xiang, J. Cui, M. Hu, Risk assessment of tunnel construction in complex geological environments based on improved game theory and multi—dimensional cloud model, Bull. Eng. Geol. Environ. 84 (2025) 489, doi: 10.1007/s10064—025—04541—3.

[22] J. Cui, S. Wu, H. Cheng, X. Hou, J. Wang, W. Liu, C. Chu, An interpretable and adaptive tunnel water inflow prediction method using data augmentation and AHP—Enhanced OP—LightGBM, Tunn. Undergr. Space Technol. 171 (2026) 107445, doi: 10.1016/j.tust.2026.107445.

[23] G. Yang, K. Liu, J. Zhang, B. Zhao, Z. Zhao, X. Chen, B.M. Chen, Datasets and processing methods for boosting visual inspection of civil infrastructure: A comprehensive review and algorithm comparison for crack classification, segmentation, and detection, Constr. Build. Mater. 356 (2022) 129226, doi: 10.1016/j.conbuildmat.2022.129226.

[24] Q. Song, Y. Wu, X. Xin, L. Yang, M. Yang, H. Chen, C. Liu, M. Hu, X. Chai, J. Li, Realtime tunnel crack analysis system via deep learning, IEEE Access 7 (2019) 64186-64197, doi: 10.1109/ACCESS.2019.2916330.

[25] H. Huang, Q. Li, D. Zhang, Deep learning based image recognition for crack and leakage defects of metro shield tunnel, Tunn. Undergr. Space Technol. 77 (2018) 166-176, doi: 10.1016/j.tust.2018.04.002.

[26] S. Qin, T. Qi, B. Lei, X. Huang, Batched—image detection model and deployment method for tunnel lining defects using line—scan cameras based on experiments study, Tunn. Undergr. Space Technol. 142 (2023) 105428, doi: 10.1016/j.tust.2023.105428.

[27] D. Tabernik, M. Šuc, D. Skočaj, Automated detection and segmentation of cracks in concrete surfaces using joined segmentation and classification deep neural network, Constr. Build. Mater. 408 (2023) 133582, doi: 10.1016/j.conbuildmat.2023.133582.

[28] Q. Zhou, Z. Qu, Y.X. Li, F.R. Ju, Tunnel crack detection with linear seam based on mixed attention and multiscale feature fusion, IEEE Trans. Instrum. Meas. 71 (2022) 1-11, doi: 10.1109/TIM.2022.3184351.

[29] D. Dais, İ.E. Bal, E. Smyrou, V. Sarhosis, Automatic crack classification and segmentation on masonry surfaces using convolutional neural networks and transfer learning, Autom. Constr. 125 (2021) 103606, doi: 10.1016/j.autcon.2021.103606.

[30] Y. Xue, X. Cai, M. Shadabfar, H. Shao, S. Zhang, Deep learning—based automatic recognition of water leakage area in shield tunnel lining, Tunn. Undergr. Space Technol. 104 (2020) 103524, doi: 10.1016/j.tust.2020.103524.

[31] Q. Zou, Z. Zhang, Q. Li, X. Qi, Q. Wang, Wang S. DeepCrack, Learning hierarchical convolutional features for crack detection, IEEE Trans. Image Process. 28 (2019) 1498-1512, doi: 10.1109/TIP.2018.2878966.

[32] Y. Xu, S. Ma, X. Yao, Z. Zhang, Q. Yan, A consumer—grade multi—sensor deep learning approach for tunnel leakage detection, Geotech. Geol. Eng. 43 (2025) 276, doi: 10.1007/s10706—025—03221—z.

[33] S.J. Feng, Y. Feng, X.L. Zhang, Y.H. Chen, Deep learning with visual explanations for leakage defect segmentation of metro shield tunnel, Tunn. Undergr. Space Technol. 136 (2023) 105107, doi: 10.1016/j.tust.2023.105107.

[34] Y. Xue, F. Jia, X. Cai, M. Shadabfar, H. Huang, An optimization strategy to improve the deep learning—based recognition model of leakage in shield tunnels, Comput.—Aided Civ. Infrastruct. Eng. 37 (2022) 386-402, doi: 10.1111/mice.12731.

[35] Z. Qi, D. Liu, J. Zhang, J. Chen, Micro—concrete crack detection of underwater structures based on convolutional neural network, Mach. Vis. Appl. 33 (2022) 74, doi: 10.1007/s00138—022—01327—5.

[36] W. Zhang, J. Pang, K. Chen, C.C. Loy, K—Net: Towards unified image segmentation, 2021, doi: 10.48550/arXiv.2106.14855.

[37] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, B. Guo, Swin Transformer: Hierarchical vision Transformer using shifted windows, 2021, doi: 10.48550/arXiv.2103.14030.

[38] T. Xiao, Y. Liu, B. Zhou, Y. Jiang, J. Sun, Unified perceptual parsing for scene understanding, 2018, doi: 10.48550/arXiv.1807.10221.

[39] J. Bai, S. Wang, Q. Xu, J. Zhu, Z. Li, K. Lai, X. Liu, Z. Chen, Intelligent regional subsurface predictions based on limited borehole data and interpretability stacking technique of ensemble learning, Bull. Eng. Geol. Environ. 83 (2024) 272, doi: 10.1007/s10064—024—03758—y.

[40] N.H. Quang, H. Lee, S. Ahn, G. Kim, Enhanced wildfire detection and semantic segmentation via fine—tuned deep learning models, Earth Sci. Inform. 18 (2025) 397, doi: 10.1007/s12145—025—01863—4.

[41] A. Garcia—Garcia, S. Orts—Escolano, S. Oprea, V. Villena—Martinez, P. Martinez—Gonzalez, J. Garcia—Rodriguez, A survey on deep learning techniques for image and video semantic segmentation, Appl. Soft Comput. 70 (2018) 41-65, doi: 10.1016/j.asoc.2018.05.018.

[42] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, 2015, doi: 10.48550/arXiv.1512.03385.

[43] H.Y. Wu, C.L. Kuo, C.Y. Lin, W.T. Chu, Deep learning in the advanced core sample porosity determination with XCT image, Earth Sci. Inform. 18 (2024) 17, doi: 10.1007/s12145—024—01602—1.

[44] M. Wang, W. Guo, F. Yang, B. Yan, Y. Xu, J. Jiang, J. Huang, Rock thin section image classification in low data scenarios using few—shot learning, Comput. Geosci. 203 (2025) 105962, doi: 10.1016/j.cageo.2025.105962.

[45] J. Long, E. Shelhamer, T. Darrell, Fully Convolutional Networks for Semantic Segmentation, 2015, doi: 10.48550/arXiv.1411.4038.

[46] H. Zhao, J. Shi, X. Qi, X. Wang, J. Jia, Pyramid scene parsing network, 2017, doi: 10.48550/arXiv.1612.01105.

[47] L.C. Chen, G. Papandreou, F. Schroff, H. Adam, Rethinking atrous convolution for semantic image segmentation, 2017, doi: 10.48550/arXiv.1706.05587.

[48] H. Huang, Q. Li, Image recognition for water leakage in shield tunnel based on deep learning, Chin. J. Rock Mech. Eng. 36 (2017) 2861-2871, doi: 10.13722/j.cnki.jrme.2017.0552.

[49] Z. Zhou, J. Zhang, C. Gong, H. Ding, Automatic identification of tunnel leakage based on deep semantic segmentation, Chin. J. Rock Mech. Eng. 41 (2022) 2082-2093, doi: 10.13722/j.cnki.jrme.2022.0016.

[50] Z. Zhou, J. Zhang, C. Gong, W. Wu, Automatic tunnel lining crack detection via deep learning with generative adversarial network—based data augmentation, Undergr. Space 9 (2023) 140-154, doi: 10.1016/j.undsp.2022.07.003.

[51] X. Liu, Z. Hong, W. Shi, X. Guo, Image—processing—based subway tunnel crack detection system, Sensors 23 (2023) 6070, doi: 10.3390/s23136070.

[52] X. He, Z. Tang, Y. Deng, G. Zhou, Y. Wang, L. Li, UAV—based road crack object—detection algorithm, Autom. Constr. 154 (2023) 105014, doi: 10.1016/j.autcon.2023.105014.

[53] L. Xiong, D. Zhang, Y. Zhang, Water leakage image recognition of shield tunnel via learning deep feature representation, J. Vis. Commun. Image Represent. 71 (2020) 102708, doi: 10.1016/j.jvcir.2019.102708.

[54] C. Tang, Y.F. Liu, B.L. Li, L. Tang, J.S. Fan, Automated subway tunnel lining crack classification and detection based on two—step sequential convolutional neural network, J. Civ. Struct. Health Monit. (2024), doi: 10.1007/s13349—024—00900—y.

[55] W. Zhang, Z. Zhang, D. Qi, Y. Liu, Automatic crack detection and classification method for subway tunnel safety monitoring, Sensors 14 (2014) 19307-19328, doi: 10.3390/s141019307.

[56] H. Huang, Z. Wu, H. Shen, A three—stage detection algorithm for automatic crackwidth identification of fine concrete cracks, J. Civ. Struct. Health Monit. 14 (2024) 1373-1382, doi: 10.1007/s13349—024—00797—7.

[57] Z. Zhou, J. Zhang, C. Gong, Hybrid semantic segmentation for tunnel lining cracks based on Swin Transformer and convolutional neural network, Comput.—Aided Civ. Infrastruct. Eng. 38 (2023) 2491-2510, doi: 10.1111/mice.13003.

[58] Z. Zhou, L. Yan, J. Zhang, Y. Zheng, C. Gong, H. Yang, E. Deng, Automatic segmentation of tunnel lining defects based on multiscale attention and context information enhancement, Constr. Build. Mater. 387 (2023) 131621, doi: 10.1016/j.conbuildmat.2023.131621.

[59] J. Wu, X. Zhang, Tunnel crack detection method and crack image processing algorithm based on improved retinex and deep learning, Sensors 23 (2023) 9140, doi: 10.3390/s23229140.

[60] C. Wu, H. Huang, L. Zhang, J. Chen, Y. Tong, M. Zhou, Towards automated 3D evaluation of water leakage on a tunnel face via improved GAN and self—attention DL model, Tunn. Undergr. Space Technol. 142 (2023) 105432, doi: 10.1016/j.tust.2023.105432.

[61] D. Joshi, T.P. Singh, G. Sharma, Automatic surface crack detection using segmentation—based deep—learning approach, Eng. Fract. Mech. 268 (2022) 108467, doi: 10.1016/j.engfracmech.2022.108467.

[62] H. Huang, S. Zhao, D. Zhang, J. Chen, Deep learning—based instance segmentation of cracks from shield tunnel lining images, Struct. Infrastruct. Eng. 18 (2022) 183-196, doi: 10.1080/15732479.2020.1838559.

[63] D. Kang, S.S. Benipal, D.L. Gopal, Y.J. Cha, Hybrid pixel—level concrete crack segmentation and quantification across complex backgrounds using deep learning, Autom. Constr. 118 (2020) 103291, doi: 10.1016/j.autcon.2020.103291.

[64] Y. Pan, G. Zhang, L. Zhang, A spatial—channel hierarchical deep learning network for pixel—level automated crack detection, Autom. Constr. 119 (2020) 103357, doi: 10.1016/j.autcon.2020.103357.

[65] X. Miao, J. Wang, Z. Wang, Q. Sui, Y. Gao, P. Jiang, Automatic recognition of highway tunnel defects based on an improved U—Net model, IEEE Sens. J. 19 (2019) 11413-11423, doi: 10.1109/JSEN.2019.2934897.

[66] Z. Liu, Y. Cao, Y. Wang, W. Wang, Computer vision—based concrete crack detection using U—net fully convolutional networks, Autom. Constr. 104 (2019) 129-139, doi: 10.1016/j.autcon.2019.04.005.

[67] S. Zhao, M. Shadabfar, D. Zhang, J. Chen, H. Huang, Deep learning—based classification and instance segmentation of leakage—area and scaling images of shield tunnel linings, Struct. Control Health Monit. 28 (2021) e2732, doi: 10.1002/stc.2732.

[68] R.P.K. Poudel, S. Liwicki, R. Cipolla, Fast—SCNN: Fast semantic segmentation network, 2019, doi: 10.48550/arXiv.1902.04502.

[69] E. Xie, W. Wang, Z. Yu, A. Anandkumar, J.M. Alvarez, P. Luo, SegFormer: Simple and efficient design for semantic segmentation with transformers (2021), doi: 10.48550/arXiv.2105.15203.

[70] Z. Chen, X. Liu, J. Yang, E. Little, Y. Zhou, Deep learning—based method for SEM image segmentation in mineral characterization, an example from Duvernay Shale samples in Western Canada Sedimentary Basin, Comput. Geosci. 138 (2020) 104450, doi: 10.1016/j.cageo.2020.104450.

[71] O. Ronneberger, P. Fischer, T. Brox, U—Net: Convolutional networks for biomedical image segmentation (2015), doi: 10.48550/arXiv.1505.04597.

[72] Santos V. Silva dos, E. Gloaguen, S. Tirdad, Lithological mapping using Spatially Constrained Bayesian Network (SCB—Net): A deep learning model for generating fielddata—constrained predictions with uncertainty evaluation using remote sensing data, Comput. Geosci. 204 (2025) 105964, doi: 10.1016/j.cageo.2025.105964.

[73] L.C. Chen, Y. Zhu, G. Papandreou, F. Schroff, H. Adam, Encoder—decoder with atrous separable convolution for semantic image segmentation, 2018, doi: 10.48550/arXiv.1802.02611.

[74] H. Zhao, J. Shi, X. Qi, X. Wang, J. Jia, Pyramid scene parsing network, 2017, doi: 10.48550/arXiv.1612.01105.

[75] L.C. Chen, G. Papandreou, F. Schroff, H. Adam, Rethinking atrous convolution for semantic image segmentation, 2017, doi: 10.48550/arXiv.1706.05587.

[76] B. Cheng, I. Misra, A.G. Schwing, A. Kirillov, R. Girdhar, Maskedattention mask transformer for universal image segmentation, 2022, doi: 10.48550/arXiv.2112.01527.

[77] Q. Wang, J. Hu, P. Liu, G. Sun, Intelligent recognition of shale fracture network images based on transfer learning, Earth Sci. Inform. 17 (2024) 797-812, doi: 10.1007/s12145—023—01202—5.

[78] R. Fu, Q. Hu, X. Dong, Y. Guo, Y. Gao, B. Li, Axiom—based Grad—CAM: Towards accurate visualization and explanation of CNNs, 2020, doi: 10.48550/arXiv.2008.02312.

[79] H. Wang, L. Li, Y.Y. Jiao, X.R. Ge, S.C. Li, A relationship—based and object—oriented software for monitoring management during geotechnical excavation, Adv. Eng. Softw. 71 (2014) 34-45, doi: 10.1016/j.advengsoft.2014.02.001.

[80] X. Wang, S. Wu, L. Han, J. Wang, J. Cui, Y. Shen, Slope stability prediction based on GSOEM—SV: A mobile application practicably deploy in engineering verification, Adv. Eng. Softw. 192 (2024) 103648, doi: 10.1016/j.advengsoft.2024.103648.