A hybrid review of sewer inspection tools and automated CCTV image analysis techniques

Abstract

Maintaining the integrity of sewage networks is crucial for sustainable urban development. Despite extensive research on inspection tools, machine learning applications, and condition assessment for sewer defects, a holistic review of these elements remains absent. This paper addresses this gap by presenting a comprehensive review within a unified framework, employing a mixed-method approach that includes bibliometric, scientometric, and systematic analyses. Our findings reveal that integrating in-pipe and out-pipe inspection methods enhances outcomes. The current literature identifies modified RegNet, dilation segmentation with conditional random field (DilaSeg-CRF), you only look once (YOLO) models, and faster region-based convolutional neural network (Faster R-CNN) as effective algorithms for classification, segmentation, and object detection, both on-site and off-site, respectively. However, machine learning is an evolving field, and future algorithms may surpass these models. Identifying key challenges, we propose recommendations aimed at advancing research and enhancing replicability: notably, the expansion of international research collaborations, particularly in underrepresented regions such as the Middle East, Africa, Asia, and South America; applying the latest version of YOLOv11 in object detection; and investigating defect patterns in polyvinyl chloride (PVC) sewer and rehabilitated pipes using advanced diagnostic methods. This review anticipates aiding policymakers in adopting informed strategies, thereby contributing to the development of smarter, more sustainable cities.

Keywords

Sewer pipelines / Scientometric analysis / Automated defects detection / Digital technologies / Sustainable drainage systems

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

CRediT authorship contribution statement

Mohamed Nashat: Writing - original draft. Tarek Zayed: Supervision, Writing - review & editing.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgement

This work was supported by the Research Grants Council of the University Grants Committee (Grant No. RGC-15209022) and the General Research Fund (Grant No. GRF-15202524) in Hong Kong, China.

References

[1] Alejo, D., Chataigner, F., Serrano, D., Merino, L., & Caballero, F. (2021). Into the dirt: Datasets of sewer networks with aerial and ground platforms. Journal of Field Robotics, 38 (1), 105-120.

[2] Alqahtani, F. K., Alsharef, A., Hommadi, G. M., & Alammari, M. A. (2023). Assessment framework for the maintainability of sewer pipeline systems. Applied Sciences, 13 (21), 11828.

[3] Alshami, A., Elsayed, M., Ali, E., Eltoukhy, A. E. E., & Zayed, T. (2023a). Harnessing the power of ChatGPT for automating systematic review process: Methodology, case study, limitations, and future directions. Systems, 11 (7), 351.

[4] Alshami, A., Elsayed, M., Ali, E., Eltoukhy, A. E. E., & Zayed, T. (2023b). Monitoring blockage and overflow events in small-sized sewer network using contactless flow sensors in Hong Kong: Problems, causes, and proposed solution. IEEE Access, 11, 87131-87149.

[5] Bahnsen, C. H., Clement, A., Larsen, H. C. Ø., & Moeslund, T. B. (2023). Measuring the interior of in-use sewage pipes using 3D vision. Automation in Construction, 151.

[6] Caradot, N., Riechel, M., Rouault, P., Caradot, A., Lengemann, N., Eckert, E., Ringe, A., Clemens, F., & Cherqui, F. (2020). The influence of condition assessment uncertainties on sewer deterioration modelling. Structure and Infrastructure Engineering, 16 (2), 287-296.

[7] Chen, Y., Sharifuzzaman, S. A. S. M., Wang, H., Li, Y., Minh Dang, L., Song, H.-K., & Moon, H. (2023). Deep learning based underground sewer defect classification using a modified RegNet. Computers, Materials and Continua, 75 (3), 5451-5469.

[8] Chuang, T.-Y., & Sung, C.-C. (2020). Learning and SLAM based decision support platform for sewer inspection. Remote Sensing, 12 (6).

[9] Dang, L. M., Kyeong, S., Li, Y., Wang, H., Nguyen, T. N., & Moon, H. (2021). Deep learning-based sewer defect classification for highly imbalanced dataset. Computers and Industrial Engineering, 161.

[10] Dang, L. M., Wang, H., Li, Y., Nguyen, L. Q., Nguyen, T. N., Song, H.-K., & Moon, H. (2023). Lightweight pixel-level semantic segmentation and analysis for sewer defects using deep learning. Construction and Building Materials, 371.

[11] Dang, L. M., Wang, H., Li, Y., Nguyen, T. N., & Moon, H. (2022). DefectTR: End-to-end defect detection for sewage networks using a transformer. Construction and Building Materials, 325.

[12] Fang, X., Li, Q., Zhu, J., Chen, Z., Zhang, D., Wu, K., Ding, K., & Li, Q. (2022). Sewer defect instance segmentation, localization, and 3D reconstruction for sewer floating capsule robots. Automation in Construction, 142.

[13] Fu, G., Jin, Y., Sun, S., Yuan, Z., & Butler, D. (2022). The role of deep learning in urban water management: A critical review. Water Research, 223.

[14] Guo, W., Zhang, X., Zhang, D., Chen, Z., Zhou, B., Huang, D., & Li, Q. (2022). Detection and classification of pipe defects based on pipe-extended feature pyramid network. Automation in Construction, 141.

[15] Hassan, S. I., Dang, L. M., Mehmood, I., Im, S., Choi, C., Kang, J., Park, Y.-S., & Moon, H. (2019). Underground sewer pipe condition assessment based on convolutional neural networks. Automation in Construction, 106.

[16] Haurum, J. B., Madadi, M., Escalera, S., & Moeslund, T. B. (2022). Multi-scale hybrid vision transformer and Sinkhorn tokenizer for sewer defect classification. Automation in Construction, 144.

[17] Haurum, J. B., & Moeslund, T. B. (2020). A survey on image-based automation of CCTV and SSET sewer inspections. Automation in Construction, 111.

[18] He, J., Hou, Z., Zhu, D., Li, Z., & Li, Z. (2022a). Automatic accurate longitudinal location of structural defects in sewer pipes via monocular ranging. Applied Optics, 61 (27), 7899-7911.

[19] He, M., Zhao, Q., Gao, H., Zhang, X., & Zhao, Q. (2022b). Image segmentation of a sewer based on deep learning. Sustainability (Switzerland), 14 (11).

[20] Hu, C., Dong, B., Shao, H., Zhang, J., & Wang, Y. (2023). Toward purifying defect feature for multilabel sewer defect classification. IEEE Transactions on Instrumentation and Measurement, 72.

[21] Jain, R. K., Das, A., Mukherjee, A., Ray, D. N., & Karmakar, P. (2021). Experimental performance of robotic inspection system for underground pipelines. Journal of The Institution of Engineers (India): Series C, 102 (3), 683-703.

[22] Jia, P., Liao, Y., Zhao, Q., He, M., & Guo, M. (2023). Defect severity assessment model for sewer pipeline based on automated pipe calibration. Journal of Pipeline Systems Engineering and Practice, 14 (3).

[23] Karabij, M., Yin, X., Bouferguene, A., & Al-Hussein, M. (2022). Image processing-based framework for determining deterioration in sewer pipe defects. Journal of Infrastructure Systems, 28 (4).

[24] Kumar, S. S., Abraham, D., & Choi, J. (2022). A hybrid framework for mining spatial characteristics of sewer defects from inspection databases. Urban Water Journal, 19 (1), 51-61.

[25] Kumar, S. S., Wang, M., Abraham, D. M., Jahanshahi, M. R., Iseley, T., & Cheng, J. C. P. (2020). Deep learning-based automated detection of sewer defects in CCTV videos. Journal of Computing in Civil Engineering, 34 (1).

[26] Li, D., Cong, A., & Guo, S. (2019). Sewer damage detection from imbalanced CCTV inspection data using deep convolutional neural networks with hierarchical classification. Automation in Construction, 101, 199-208.

[27] Li, D., Xie, Q., Yu, Z., Wu, Q., Zhou, J., & Wang, J. (2021). Sewer pipe defect detection via deep learning with local and global feature fusion. Automation in Construction, 129.

[28] Li, M., Li, M., Ren, Q., Liu, H., & Liu, C. (2023). Intelligent identification and classification of sewer pipeline network defects based on improved RegNetY network. Journal of Civil Structural Health Monitoring, 13 (2-3), 547-560.

[29] Li, Y., Wang, H., Dang, L. M., Jalil Piran, M., & Moon, H. (2022). A robust instance segmentation framework for underground sewer defect detection. Measurement: Journal of the International Measurement Confederation, 190.

[30] Liu, H., Yue, Y., Liu, C., Spencer, B. F., & Cui, J. (2023). Automatic recognition and localization of underground pipelines in GPR B-scans using a deep learning model. Tunnelling and Underground Space Technology, 134.

[31] Ma, D., Liu, J., Fang, H., Wang, N., Zhang, C., Li, Z., & Dong, J. (2021). A Multi-defect detection system for sewer pipelines based on StyleGAN-SDM and fusion CNN. Construction and Building Materials, 312.

[32] Ma, S., Elshaboury, N., Ali, E., & Zayed, T. (2024a). Proactive exfiltration severity management in sewer networks: A hyperparameter optimization for two-tiered machine learning prediction. Tunnelling and Underground Space Technology, 144.

[33] Ma, S., Zayed, T., Xing, J., & Shao, Y. (2024b). A state-of-the-art review for the prediction of overflow in urban sewer systems. Journal of Cleaner Production, 434, 139923.

[34] Makana, L. O., Shepherd, W. J., Tait, S., Rogers, C. D. F., Metje, N., Boxall, J. B., & Schellart, A. N. A. (2022). Future inspection and deterioration prediction capabilities for buried distributed water infrastructure. Journal of Pipeline Systems Engineering and Practice, 13 (3).

[35] Makris, K. F., Langeveld, J., & Clemens, F. H. L. R. (2020). A review on the durability of PVC sewer pipes: Research vs. practice. Structure and Infrastructure Engineering, 16 (6), 880-897.

[36] Meijer, D., Scholten, L., Clemens, F., & Knobbe, A. (2019). A defect classification methodology for sewer image sets with convolutional neural networks. Automation in Construction, 104, 281-298.

[37] Mezhoud, C., Berreksi, A., Bedjou, A., & Bosseler, B. (2022). Prioritization of maintenance work in wastewater networks using decision support methods. Journal of Water Sanitation and Hygiene for Development, 12 (2), 186-199.

[38] Moradi, S., Zayed, T., & Golkhoo, F. (2019). Review on computer aided sewer pipeline defect detection and condition assessment. Infrastructures, 4 (1).

[39] Moradi, S., Zayed, T., Nasiri, F., & Golkhoo, F. (2020). Automated anomaly detection and localization in sewer inspection videos using proportional data modeling and deep learning-based text recognition. Journal of Infrastructure Systems, 26 (3).

[40] Myrans, J., Everson, R., & Kapelan, Z. (2019). Automated detection of fault types in CCTV sewer surveys. Journal of Hydroinformatics, 21, 153-163.

[41] Oh, C., Dang, L. M., Han, D., & Moon, H. (2022). Robust sewer defect detection with text analysis based on deep learning. IEEE Access, 10, 46224-46237.

[42] Pan, F., Tang, D., Guo, X., & Pan, S. (2021). Defect identification of pipeline ultrasonic inspection based on multi-feature fusion and multi-criteria feature evaluation. International Journal of Pattern Recognition and Artificial Intelligence, 35 (11).

[43] Pan, G., Zheng, Y., Guo, S., & Lv, Y. (2020). Automatic sewer pipe defect semantic segmentation based on improved U-Net. Automation in Construction, 119.

[44] Salihu, C., Hussein, M., Mohandes, S. R., & Zayed, T. (2022). Towards a comprehensive review of the deterioration factors and modeling for sewer pipelines: A hybrid of bibliometric, scientometric, and meta-analysis approach. Journal of Cleaner Production, 351, 131460.

[45] Situ, Z., Teng, S., Feng, W., Zhong, Q., Chen, G., Su, J., & Zhou, Q. (2023). A transfer learning-based YOLO network for sewer defect detection in comparison to classic object detection methods. Developments in the Built Environment, 15.

[46] Situ, Z., Teng, S., Liao, X., Chen, G., & Zhou, Q. (2024). Real-time sewer defect detection based on YOLO network, transfer learning, and channel pruning algorithm. Journal of Civil Structural Health Monitoring, 14 (1), 41-57.

[47] Siu, C., Wang, M., & Cheng, J. C. P. (2022). A framework for synthetic image generation and augmentation for improving automatic sewer pipe defect detection. Automation in Construction, 137.

[48] Tan, Y., Cai, R., Li, J., Chen, P., & Wang, M. (2021). Automatic detection of sewer defects based on improved you only look once algorithm. Automation in Construction, 131.

[49] Tugeumwolachot, T., Seki, H., Tsuji, T., & Hiramitsu, T. (2021). Development of a compact sewerage robot with multi-DOF cutting tool. Artificial Life and Robotics, 26 (4), 404-411.

[50] Wang, M., & Cheng, J. C. P. (2020). A unified convolutional neural network integrated with conditional random field for pipe defect segmentation. Computer-Aided Civil and Infrastructure Engineering, 35 (2), 162-177.

[51] Wang, M., Kumar, S. S., & Cheng, J. C. P. (2021a). Automated sewer pipe defect tracking in CCTV videos based on defect detection and metric learning. Automation in Construction, 121.

[52] Wang, M., Luo, H., & Cheng, J. C. P. (2021b). Towards an automated condition assessment framework of underground sewer pipes based on closed-circuit television (CCTV) images. Tunnelling and Underground Space Technology, 110.

[53] Wang, M., & Yin, X. (2022). Construction and maintenance of urban underground infrastructure with digital technologies. Automation in Construction, 141.

[54] Wang, N., Ma, D., Du, X., Li, B., Di, D., Pang, G., & Duan, Y. (2024). An automatic defect classification and segmentation method on three-dimensional point clouds for sewer pipes. Tunnelling and Underground Space Technology, 143.

[55] Wang, T., Li, Y., Zhai, Y., Wang, W., & Huang, R. (2023). A sewer pipeline defect detection method based on improved YOLOv5. Processes, 11 (8).

[56] Wang, Y., Li, P., & Li, J. (2022). The monitoring approaches and non-destructive testing technologies for sewer pipelines. Water Science and Technology, 85 (10), 3107-3121.

[57] Xia, Z., Guo, S., Sun, D., Lv, Y., Li, H., & Pan, G. (2023). Structure-aware dehazing of sewer inspection images based on monocular depth cues. Computer-Aided Civil and Infrastructure Engineering, 38 (6), 762-778.

[58] Xie, Q., Li, D., Xu, J., Yu, Z., & Wang, J. (2019). Automatic detection and classification of sewer defects via hierarchical deep learning. IEEE Transactions on Automation Science and Engineering, 16 (4), 1836-1847.

[59] Xu, M., Yang, F., & Yan, R. (2023). Design and implementation of three-channel drainage pipeline ground penetrating radar device. Sensors, 23 (9).

[60] Yang, K., Fang, H., Liu, H., Li, B., Zhang, X., Xia, Y., & Zhai, K. (2023). An intelligent model to predict the mechanical properties of defected concrete drainage pipes. International Journal of Mechanical Sciences, 260.

[61] Ye, X., Zuo, J., Li, R., Wang, Y., Gan, L., Yu, Z., & Hu, X. (2019). Diagnosis of sewer pipe defects on image recognition of multi-features and support vector machine in a southern Chinese city. Frontiers of Environmental Science and Engineering, 13 (2).

[62] Yin, X., Chen, Y., Bouferguene, A., Zaman, H., Al-Hussein, M., & Kurach, L. (2020a). A deep learning-based framework for an automated defect detection system for sewer pipes. Automation in Construction, 109.

[63] Yin, X., Chen, Y., Bouferguene, A., Zaman, H., Al-Hussein, M., & Russell, R. (2020b). Data-driven framework for modeling productivity of closed-circuit television recording process for sewer pipes. Journal of Construction Engineering and Management, 146 (8).

[64] Yin, X., Ma, T., Bouferguene, A., & Al-Hussein, M. (2021). Automation for sewer pipe assessment: CCTV video interpretation algorithm and sewer pipe video assessment (SPVA) system development. Automation in Construction, 125.

[65] Yu, Z., Li, X., Sun, L., Zhu, J., & Lin, J. (2024). A composite transformer-based multi-stage defect detection architecture for sewer pipes. Computers, Materials and Continua, 78 (1), 435-451.

[66] Yussuf, A. N., Weerasinghe, N. P., Chen, H., Hou, L., Herath, D., Rashid, M., Zhang, G., & Setunge, S. (2024). Leveraging deep learning techniques for condition assessment of stormwater pipe network. Journal of Civil Structural Health Monitoring, 15, 619-633.

[67] Zhang, J., Liu, X., Zhang, X., Xi, Z., & Wang, S. (2023a). Automatic detection method of sewer pipe defects using deep learning techniques. Applied Sciences (Switzerland), 13 (7).

[68] Zhang, X., Zhang, J., Tian, L., Liu, X., & Wang, S. (2023b). A lightweight method for detecting sewer defects based on improved YOLOv5. Applied Sciences (Switzerland), 13 (15).

[69] Zhao, M., Fang, Z., Ding, N., Li, N., Su, T., & Qian, H. (2023). Quantitative detection technology for geometric deformation of pipelines based on LiDAR. Sensors, 23 (24).

[70] Zhou, B., Li, B., Lan, W., Tian, C., & Yao, W. (2024a). SDH-FCOS: An efficient neural network for defect detection in urban underground pipelines. Computers, Materials and Continua, 78 (1), 633-652.

[71] Zhou, Q., Situ, Z., Feng, W., Liu, H., Liao, X., Zhang, J., Ge, X., & Chen, G. (2024b). Deep learning, geometric characterization and hydrodynamic modeling for assessing sewer defect impacts on urban flooding: A case study in Guangzhou, China. Journal of Environmental Management, 351.

[72] Zhou, Q., Situ, Z., Teng, S., & Chen, G. (2021a). Convolutional neural networks-based model for automated sewer defects detection and classification. Journal of Water Resources Planning and Management, 147 (7).

[73] Zhou, Q., Situ, Z., Teng, S., & Chen, G. (2023). Comparative effectiveness of data augmentation using traditional approaches versus StyleGANs in automated sewer defect detection. Journal of Water Resources Planning and Management, 149 (9).

[74] Zhou, Q., Situ, Z., Teng, S., Chen, W., Chen, G., & Su, J. (2022a). Comparison of classic object-detection techniques for automated sewer defect detection. Journal of Hydroinformatics, 24 (2), 406-419.

[75] Zhou, Q., Situ, Z., Teng, S., Liu, H., Chen, W., & Chen, G. (2022b). Automatic sewer defect detection and severity quantification based on pixel-level semantic segmentation. Tunnelling and Underground Space Technology, 123.

[76] Zhou, Y., Ji, A., & Zhang, L. (2022c). Sewer defect detection from 3D point clouds using a transformer-based deep learning model. Automation in Construction, 136.

[77] Zounemat-Kermani, M., Alizamir, M., Yaseen, Z. M., & Hinkelmann, R. (2021). Concrete corrosion in wastewater systems: Prediction and sensitivity analysis using advanced extreme learning machine. Frontiers of Structural and Civil Engineering, 15 (2), 444-460.

[78] Zuo, X., Dai, B., Shan, Y., Shen, J., Hu, C., & Huang, S. (2020). Classifying cracks at sub-class level in closed circuit television sewer inspection videos. Automation in Construction, 118.