An enhanced method of CNNs by incorporating the clustering-guided block for concrete crack recognition

Abstract

Concrete cracking poses a significant threat to the safety and stability of crucial infrastructure such as bridges, roads, and building structures. Recognizing and accurately measuring the morphology of cracks is essential for assessing the structural integrity of these elements. This paper introduces a novel Crack Segmentation method known as CG-CNNs, which combines a Clustering-guided (CG) block with a Convolutional Neural Network (CNN). The innovative CG block operates by categorizing extracted image features into K groups, merging these features, and then simultaneously feeding the augmented features and original image into the CNN for precise crack image segmentation. It automatically determines the optimal K value by evaluating the Silhouette Coefficient for various K values, utilizing the grayscale feature value of each cluster centroid as a defining characteristic for each category. To bolster our approach, we curated a dataset of 2500 crack images from concrete structures, employing rigorous pre-processing and data augmentation techniques. We benchmarked our method against three prevalent CNN architectures: DeepLabV3 + , U-Net, and SegNet, each augmented with the CG block. An algorithm specialized for assessing crack edge recognition accuracy was employed to analyze the proposed method's performance. The comparative analysis demonstrated that CNNs enhanced with the CG block exhibited exceptional crack image recognition capabilities and enabled precise segmentation of crack edges. Further investigation revealed that the CG-DeepLabV3 + model excelled, achieving an F1 score of 0.90 and an impressive intersection over union (IoU) value of 0.82. Notably, the CG-DeepLabV3 + model significantly reduced the recognition error for locating crack edges to a mere 2.31 pixels. These enhancements mark a significant advancement in developing accurate algorithms based on deep neural networks for identifying concrete crack edges reliably. In conclusion, our CG-CNNs approach offers a highly accurate method for crack segmentation, which is invaluable for machine-based measurements of cracks on concrete surfaces.

Keywords

K-means clustering-guided / CNNs / DeepLabV3 +  / U-Net / SegNet / Concrete crack recognition / Semantic segmentation / Information and Computing Sciences / Artificial Intelligence and Image Processing / Engineering / Civil Engineering

References

[1] Ahmed NBC, Lahouar S, Souani C, Besbes K. Automatic crack detection from pavement images using fuzzy thresholding. In Proceedings of the 2017 International Conference on Control, Automation and Diagnosis (ICCAD). 2017. 528–537, https://doi.org/10.1109/CADIAG.2017.8075714.

[2] AmhazR, ChambonS, IdierJ, BaltazartV. Automatic crack detection on two-dimensional pavement images: an algorithm based on minimal path selection. IEEE Trans Intell Transport Syst, 2016, 17: 2718-2729.

[3] Arthur D, Vassilvitskii S. k-means++: The advantages of careful seeding. Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, 2007, https://doi.org/10.1137/S0097539704442587.

[4] BadrinarayananV, KendallA, CipollaR. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Machine Intell, 2017, 39: 2481-2495.

[5] Caron M, Bojanowski P, Joulin A, Douze M. Deep clustering for unsupervised learning of visual features. In Proceedings of the Proceedings of the European conference on computer vision (ECCV), 2018, 132–149, https://doi.org/10.48550/arXiv.1807.05520.

[6] ChenLC, PapandreouG, KokkinosI, MurphyK, YuilleAL. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Machine Intell, 2018, 40: 834-848.

[7] ChoiW, ChaYJ. SDDNet: real-time crack segmentation. IEEE Trans Indust Elect, 2020, 67: 8016-8025.

[8] del Río-BarralP, SoilánM, González-CollazoSM, AriasP. Pavement crack detection and clustering via region-growing algorithm from 3D MLS point clouds. Remote Sens, 2022, 14225866.

[9] EjimaT, OhashiT, SuandiSA, LiuZ. Tunnel crack detection and classification system based on image processing. Proc Conf Machine Vision Applic Indust Inspection, 2002, 4959145-152.

[10] FanXN, WuJJ, ShiPF, ZhangXW, XieYJ. A novel automatic dam crack detection algorithm based on local-global clustering. Multimed Tools Applic, 2018, 772026581-26590.

[11] HamishebaharY, GuanH, SoSP, JoJ. A comprehensive review of deep learning-based crack detection approaches. Appl Sci-Basel, 2022, 1231374.

[12] HanC, YangH, YangY. Enhancing pixel-level crack segmentation with visual mamba and convolutional networks. Autom Const, 2024, 168. 105770

[13] HoangDN. Detection of surface crack in building structures using image processing technique with an improved Otsu method for image thresholding. Adv Civil Eng, 2018.

[14] HouSB, ZhouT, LiuYC, DangP, LuHL, ShiHB. Teeth U-Net: a segmentation model of dental panoramic X-ray images for context semantics and contrast enhancement. Computers in Biology and Medicine, 2023, 152: 106296.

[15] JahanshahiMR, KellyJS, MasriSF, SukhatmeGS. A survey and evaluation of promising approaches for automatic image-based defect detection of bridge structures. Struct Infrast Eng, 2009, 5: 455-486.

[16] Jenkins, M.D.; Carr, T.A.; Iglesias, M.I.; Buggy, T.; Morison, G. A deep convolutional neural network for semantic Pixel-Wise segmentation of road and pavement surface cracks. In Proceedings of the 2018 26th European Signal Processing Conference (EUSIPCO), 2018, 2120–2124, https://doi.org/10.23919/EUSIPCO.2018.855328.

[17] JiAK, XueXL, WangYN, LuoXW, XueWR. An integrated approach to automatic pixel-level crack detection and quantification of asphalt pavement. Autom Const, 2020, 114. 103176

[18] KingmaD, AdamBJ. A method for stochastic optimization. Comput Sci, 2015.

[19] LangH, LuJJ, LouYX, ChenSD. Pavement cracking detection and classification based on 3D image using multiscale clustering model. J Comput Civil Eng, 2020, 34504020034.

[20] LiuYH, YaoJ, LuXH, XieRP, LiL. Deep Crack: a deep hierarchical feature learning architecture for crack segmentation. Neurocomputing, 2019, 338: 139-153.

[21] LiuY, YaoJ, LuX, XieR, LiL. DeepCrack: a deep hierarchical feature learning architecture for crack segmentation. Neurocomputing, 2019, 338: 139-153.

[22] LiuHN, XieZ, YuRX, ZhangN. A new three-dimensional fractal dimension model to describe the complexity of concrete pores. J Adv Conc Technol, 2022, 20: 127-138.

[23] MaM, YangL, LiuY, YuH. An attention-based progressive fusion network for pixelwise pavement crack detection. Measurement, 2024, 226. 114159

[24] MaK, ZhaoS, LiuG, MiaoJ, KangJ, WangL, YuQ. Failure and post-failure kinematic disaster processes of columnar rock masses based on 3D discontinuous deformation analysis. Int J Rock Mechanics Min Sci, 2025, 186. 106003

[25] NargesK, VidaM. A critical review and comparative study on image segmentation-based techniques for pavement crack detection. Const Build Mater, 2022, 321. 126162

[26] Oliveira H, Correia PL. Road surface crack detection: improved segmentation with pixel-based refinement. In Proceedings of the 2017 25th European Signal Processing Conference (EUSIPCO), 2017, 2026–2030, https://doi.org/10.23919/EUSIPCO.2017.8081565.

[27] Ouyang A, Luo C, Zhou C. Surface distresses detection of pavement based on digital image processing. Computer and Computing Technologies in Agriculture IV-IFIP Tc 12 Conference, 2011, 347, 268–375, https://doi.org/10.1007/978-3-642-18369-0_42.

[28] QuintanaM, TorresJ, MenéndezJM. A simplified computer vision system for road surface inspection and maintenance. IEEE Trans Intell Transport Syst, 2016, 17: 608-619.

[29] RenZH, FangFZ, YanN, WuY. State of the art in defect detection based on machine vision. Int J Prec Eng Manufact-Green Technol, 2022, 9: 661-691.

[30] RezaieA, AchantaR, GodioM, BeyerK. Comparison of crack segmentation using digital image correlation measurements and deep learning. Const Buil Mater, 2020, 261. 120474

[31] RonnebergerO, FischerP, BroxT. U-Net: Convolutional networks for biomedical image segmentation. Int Conf Med Image Comput Comput-Ass Int Springer, 2015, 9351: 234-241.

[32] RousseeuwPJ. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics, 1987, 20: 53-65.

[33] ShelhamerE, LongJ, DarrellT. Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Machine Intell, 2017, 39: 640-651.

[34] SpencerBF, HoskereV, NarazakiY. Advances in computer vision-based civil infrastructure inspection and monitoring. Engineering, 2019, 5: 199-222.

[35] Yang B, Fu X, Sidiropoulos ND, Hong M. Towards k-means-friendly spaces: simultaneous deep learning and clustering, Proceedings of the 34th International Conference on Machine Learning, 2017, https://doi.org/10.48550/arXiv.1610.04794.

[36] Yang S. Xiao W, Zhang M, Guo S, Zhao J, Shen F. Image Data Augmentation for Deep Learning: a Survey. arXiv, 2023, https://doi.org/10.48550/arXiv.2204.08610.

[37] ZhangXL, HeYL, JinY, QinHL, AzharM, HuangJZ. A robust k-Means clustering algorithm based on observation point mechanism. Complexity, 2020.

[38] ZhangLL, LiaoY, WangGX, ChenJ, WangHB. A multi-scale contextual information enhancement network for crack segmentation. Appl Sci-Basel, 2022, 122111135.

[39] ZhangHB, ZhongX, LiGL, LiuW, LiuJW, JiDH, LiX, WuJG. BCU-Net: Bridging ConvNeXt and U-Net for medical image segmentation. Comput Biol Med, 2023, 159. 106960

Funding

National Natural Science Foundation of China(52208207)

Shaanxi Provincial Department of Transportation, China(21-23K)

Natural Science Basic Research Plan in Shaanxi Province of China(2022JQ-475)

RIGHTS & PERMISSIONS

The Author(s)