Rapid acquisition and surface defects recognition based on panoramic image of small-section hydraulic tunnel

Abstract

Small-section hydraulic tunnels are characterized by small spaces and various section forms, under complex environments, which makes it difficult to carry out an inspection by the mobile acquisition equipment. To resolve these problems, an arbitrarily adjustable camera module deployment method and the corresponding automatic image acquisition equipment with multi-area array cameras are proposed and developed. Such method enables the acquisition of full-length surface images of the hydraulic tunnels with different cross-section forms and diameters by a one-way travel, and the overlap rate and accuracy of the acquired image sets meet the requirements of three-dimensional reconstruction and panoramic image generation. In addition, to improve the speed and accuracy of traditional algorithms for tunnel surface defects detection, this paper proposes an improved YOLOv5s-DECA model. The algorithm introduces DenseNet to optimize the backbone feature extraction network and incorporates an efficient channel attention ECA module to make a better extraction of features of defects. The experimental results show that mAP, and F1-score of YOLOv5-DECA are 73.4% and 74.6%, respectively, which are better than the common model in terms of accuracy and robustness. The proposed YOLOv5-DECA has great detection performance for targets with variable shapes and can solve the problem of classification imbalance in surface defects. Then, by combining YOLOv5-DECA with the direction search algorithm, a “point-ring-section” method is established to allow rapid identification of common surface defects by detecting them layer by layer with the bottom image of the stitched panorama as the seed. The presented method in this paper effectively solves the problem that a single image fails to show the overall distribution of the defects and their accurate positioning in a whole large tunnel section and the effective features of defects in an excessively large panoramic image size are difficult to be captured by the neural network. Field applications demonstrated that the presented method is adequate for high-precision and intelligent surface defect detection and positioning for different small-section hydraulic tunnels such as circular, arch-wall, and box-shaped hydraulic tunnels.

Keywords

Small-section hydraulic tunnels / Image acquisition / Panorama image / Accurate positioning

CRediT authorship contribution statement

Haoyu Wang: Writing - original draft, Software, Methodology. Jichen Xie: Investigation. Jinyang Fu: Writing - review & editing, Supervision, Project administration. Cong Zhang: Data curation. Dingping Chen: Methodology. Zhiheng Zhu: Methodology. Xuesen Zhang: Funding acquisition.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgement

This research was funded by the Hunan Provincial Natural Science Foundation Project (Grant No. 2023JJ30672), the Science and Technology Research and Development Program Project of China Railway Group Limited (Grant No. 2021-Special-08(A)), the Science and Technology Research and Development Plan Project of China National Railway Group Co. Ltd. (Grant No. L2022G003), and the Open Foundation of National Engineering Laboratory for High-speed Railway Construction (No. HSR202005).

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

[1] Cabo, C., Ordóñez, C., & Argüelles-Fraga, R. (2017). An algorithm for optimizing terrestrial laser scanning in tunnels. Automation in Construction, 83, 163-168.

[2] Cacciari, P. P., & Futai, M. M. (2016). Mapping and characterization of rock discontinuities in a tunnel using 3D terrestrial laser scanning. Bulletin of Engineering Geology and the Environment, 75(1), 223-237.

[3] Cao, M. T. (2023). Drone-assisted segmentation of tile peeling on building facades using a deep learning model. Journal of Building Engineering, 80, 108063.

[4] Chen, Z., Wang, Q., He, Q., Yu, T., Zhang, M., & Wang, P. (2022). CUFuse: Camera and ultrasound data fusion for rail defect detection. IEEE Transactions on Intelligent Transportation Systems, 23(11), 21971-21983.

[5] Dang, C., Wang, Z., He, Y., Wang, L., Cai, Y., Shi, H., & Jiang, J. (2023). The Accelerated Inference of a Novel Optimized YOLOv5-LITE on Low-Power Devices for Railway Track Damage Detection. IEEE Access, 11, 134846-134865.

[6] Deng, J., Lu, Y., & Lee, V. C. S. (2021). Imaging-based crack detection on concrete surfaces using You Only Look Once network. Structural Health Monitoring, 20(2), 484-499.

[7] Du, Y., Pan, N., Xu, Z., Deng, F., Shen, Y., & Kang, H. (2021). Pavement distress detection and classification based on YOLO network. International Journal of Pavement Engineering, 22(13), 1659-1672.

[8] Duan, S., Zhang, M., Qiu, S., Xiong, J., Zhang, H., Li, C., Jiang, Q., & Kou, Y. (2024). Tunnel lining crack detection model based on improved YOLOv5. Tunnelling and Underground Space Technology, 147, 105713.

[9] Furukawa, Y., Curless, B., Seitz, S. M., & Szeliski, R. (2010). Towards internet-scale multi-view stereo. In Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, CA, USA (pp.4700-4708).

[10] Gavilán, M., Sánchez, F., Ramos, J. A., & Marcos, O. (2013). Mobile inspection system for high-resolution assessment of tunnels. In The 6th International Conference on Structural Health Monitoring of Intelligent Infrastructure (pp.10).

[11] Guo, F., Qian, Y., Liu, J., & Yu, H. (2023). Pavement crack detection based on transformer network. Automation in Construction, 145, 104646.

[12] Hawley, C. J., & Gräbe, P. J. (2022). Water leakage mapping in concrete railway tunnels using LiDAR generated point clouds. Construction and Building Materials, 361, 129644.

[13] Hu, H., Li, Z., He, Z., Wang, L., Cao, S., & Du, W. (2024). Road surface crack detection method based on improved YOLOv5 and vehiclemounted images. Measurement, 229, 114443.

[14] Hu, J., Shen, L., & Sun, G. (2018). Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp.7132-7141). Salt Lake City, UT, USA. IEEE.

[15] Hu, J., Shen, L., & Sun, G. (2018). Squeeze-and-excitation networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp.7132-7141). Salt Lake City, UT, USA.

[16] Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017a). Densely connected convolutional networks. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2261-2269). Honolulu, HI, USA.

[17] Huang, H., Sun, Y., Xue, Y., & Wang, F. (2017b). Inspection equipment study for subway tunnel defects by grey-scale image processing. Advanced Engineering Informatics, 32, 188-201.

[18] Huber-Mörk, R., Nölle, M., Oberhauser, A., & Fischmeister, E. (2010). Statistical rail surface classification based on 2D and 21/2D image analysis. In International Conference on Advanced Concepts for Intelligent Vision Systems (pp.50-61). Berlin, Heidelberg: Springer.

[19] Li, S., Gu, X., Xu, X., Xu, D., Zhang, T., Liu, Z., & Dong, Q. (2021). Detection of concealed cracks from ground penetrating radar images based on deep learning algorithm. Construction and Building Materials, 273, 121949.

[20] Li, S., Zhao, X., & Zhou, G. (2019). Automatic pixel-level multiple damage detection of concrete structure using fully convolutional network. Computer-Aided Civil and Infrastructure Engineering, 34(7), 616-634.

[21] Liu, J., Zhao, Z., Lv, C., Ding, Y., Chang, H., & Xie, Q. (2022). An image enhancement algorithm to improve road tunnel crack transfer detection. Construction and Building Materials, 348, 128583.

[22] Liu, X., Wang, Y., Yu, Z., Zhu, L. Q., Shi, H. M., & Huang, H. T. (2018). Research on cracks image detection system for subway tunnel. In Proceedings of the 2018 IEEE 4th Information Technology and Mechatronics Engineering Conference (ITOEC) (pp. 188-192), Chongqing, China.

[23] Özaslan, T., Mohta, K., Keller, J., Mulgaonkar, Y., & Taylor, C. J. (2016). Towards fully autonomous visual inspection of dark featureless dam penstocks using MAVs. In Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 4998-5005), Daejeon, Korea. (South).

[24] Park, G., Lee, M., Jang, H., & Kim, C. (2021). Thermal anomaly detection in walls via CNN-based segmentation. Automation in Construction, 125, 103627.

[25] Pham, V., Pham, C., & Dang, T. (2020, December). Road damage detection and classification with detectron2 and faster Rr-CNNcnn. In 2020 IEEE International Conference on Big Data (Big Data) (pp. 55925601). Atlanta, GA, USA. IEEE.

[26] Pratibha, K., Mishra, M., Ramana, G. V., & Lourenço, P. B. (2023). Deep Learning-Based YOLO Network Model for Detecting Surface Cracks During Structural Health Monitoring. In International Conference on Structural Analysis of Historical Constructions (pp.179-187). Cham: Springer Nature Switzerland.

[27] Protopapadakis, E., Stentoumis, C., Doulamis, N., Doulamis, A., Loupos, K., Makantasis, K., Kopsiaftis, G., & Amditis, A. (2016). Autonomous robotic inspection in tunnels. Remote Sensing and Spatial Information Sciences, III-5(5), 167-174.

[28] Qin, H., Zhang, D., Tang, Y., & Wang, Y. (2021). Automatic recognition of tunnel lining elements from GPR images using deep convolutional networks with data augmentation. Automation in Construction, 130, 103830.

[29] Rasmus, A., Berglund, M., Honkala, M., Valpola, H., & Raiko, T. (2015). Semi-supervised learning with ladder networks. Advances in neural information processing systems, arXiv:1507.0267228. (pre-print).

[30] Sano, N., Ito, T., Baba, K., Nishimura, K., Yoshitake, I., & Kanagawa, K. (2006). Investigation of cracks in lining concrete by using sequential image analysis. Journal of Construction Management and Engineering, 62(3), 558-566.

[31] Sekar, A., & Perumal, V. (2021). Automatic road crack detection and classification using multi-tasking faster RCNN. Journal of Intelligent & Fuzzy Systems, 41(6), 6615-6628.

[32] Situ, Z., Teng, S., Feng, W., Zhong, Q., Chen, G., Su, J., & Zhou, Q. (2023). A transfer learning-based YOLO network for sewer defect detection in comparison to classic object detection methods. Developments in the Built Environment, 15, 100191.

[33] Snavely, N., Seitz, S. M., & Szeliski, R. (2006). Photo tourism: exploring Exploring photo collections in 3D. ACM Transactions on Graphics, 25 (3), 835-846.

[34] Wang, H., Wang, S., Feng, C., Li, Y. L., Chen, Y. C., & Zhang, H. (2019). Diversion tunnel defects inspection and identification using an automated robotic system. In Proceedings of the 2019 Chinese Automation Congress. Hangzhou, China (pp.5863-5868). Hangzhou, China.

[35] Wang, J., Wei, Z., Zhang, T., & Zeng, W. (2016). Deeply-fused nets. arxiv preprint arXxiv:1605.07716. (pre-print).

[36] Wang, P. Y., Wang, S. H., & Jierula, A. (2021). Automatic identification and location of tunnel lining cracks. Advances in Civil Engineering, 2021(1), 8846442.

[37] Wang, Y., Liao, W., Dong, A., Xu, L., Zhu, L., Shi, H., & Yu, Z. (2024). High-speed acquisition and intelligent tunnel surface defects recognition. Tunnelling and Underground Space Technology, 144, 105572.

[38] Xing, J., Liu, Y., & Zhang, G. Z. (2023). Improved YOLOV5-based UAV pavement crack detection. IEEE Sensors Journal, 23(14), 15901-15909.

[39] Xu, Y., Li, D., Xie, Q., Wu, Q., & Wang, J. (2021). Automatic defect detection and segmentation of tunnel surface using modified Mask RCNN. Measurement, 178(1), 109316.

[40] Yin, Z. R., Lei, Z. Z., Zheng, A., Zhu, J. S., & Liu, X. Z. (2023). Automatic detection and association analysis of multiple surface defects on shield subway tunnels. Sensors, 23(16), 7106.

[41] Yu, C., Deng, H. G., & Feng, Y. X. (2020). Effects of faster region-based convolutional neural network on the detection efficiency of rail defects under machine vision. In 2020 IEEE 5th Information Technology and Mechatronics Engineering Conference (ITOEC) (pp. 1377-1380). Chongqing, China. IEEE.

[42] Yu, C., Geng, D. X., Huang, Z. J., Zhu, Z. H., & Shi, Y. F. (2020b). Refinement identification and evaluation of tunnel lining cracks. Journal of Highway and Transportation Research and Development (English Edition), 14(1), 75-83.

[43] Zhang, E., Shao, L., & Wang, Y. (2023a). Unifying transformer and convolution for dam crack detection. Automation in Construction, 147, 104712.

[44] Zhang, J., Qian, S., & Tan, C. (2023b). Automated bridge crack detection method based on lightweight vision models. Complex & Intelligent Systems, 9(2), 1639-1652.

[45] Zhang, Y., Zuo, Z., Xu, X., Wu, J., Zhu, J., Zhang, H., Wang, J., & Tian, Y. (2022). Road damage detection using UAV images based on multilevel attention mechanism. Automation in Construction, 144, 104613.

[46] Zhou, Y., Wang, S., Mei, X., Yin, W., Lin, C., Hu, Q., & Mao, Q. (2017). Railway tunnel clearance inspection method based on 3D point cloud from mobile laser scanning. Sensors, 17(9), 2055.

[47] Zhou, Z., Zhang, J., & Gong, C. (2022). Automatic detection method of tunnel lining multi-defects via an enhanced You Only Look Once network. Computer-Aided Civil and Infrastructure Engineering, 37(6), 762-780.

[48] Zhu, J., Zhao, D., & Luo, X. (2024). Evaluating the optimised YOLObased defect detection method for subsurface diagnosis with ground penetrating radar. Road Materials and Pavement Design, 25(1), 186-203.

[49] Zhu, Z. H., Fu, J. Y., Yang, J. S., & Zhang, X. M. (2016). Panoramic image stitching for arbitrarily shaped tunnel lining inspection. Com-puter-Aided Civil and Infrastructure Engineering, 31(12), 936-953.