Application of SSD-MobileNetV2 for automated defect detection in masonry bridges using AI and IoT

Abstract

This paper presents a novel artificial intelligence (AI) and Internet of Things (IoT) framework for structural health monitoring (SHM) of masonry bridges. The system utilises the Single Shot MultiBox Detector (SSD) MobileNetV2 model within the TensorFlow Object Detection API to automatically detect critical defects such as spalling, section loss, missing masonry units, and open joints. The model achieved a mean Average Precision (mAP) of 87.4% and an F1-score of 0.89, demonstrating its reliable performance in classifying and localising defects. Through detailed analysis using TensorBoard, the study demonstrates the reliable performance of the model in classifying and localising defects, enabling timely maintenance interventions. By automating defect detection and data analysis, this approach improves monitoring efficiency, reduces operational costs, and improves safety compared to traditional manual inspections. The paper also discusses the potential for future optimisation and real-world deployment to support sustainable management of masonry bridge infrastructure.

Keywords

Artificial intelligence / Internet of things / Structural health monitoring / Object detection / Masonry bridges / Bridge inspection / Defect detection / Real-time data analysis

References

[1] AktanAE, CatbasFN, GrimmelsmanKA, TsikosCJ. Issues in infrastructure health monitoring for management. J Eng Mech, 2000, 126(7): 711-724.

[2] Ali M, Zhang L, Yang C (2021) Real-time multiple damage mapping using autonomous uav and deep faster region-based neural networks for gps-denied structures. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), IEEE, pp 1234–1240

[3] Bakos JD (2016) Chapter 2 – multicore and data-level optimization: Openmp and simd. In: Embedded Systems: ARM Programming and Optimization, Elsevier, Oxford, pp 49–103. https://doi.org/10.1016/B978-0-12-800342-8.00002-X

[4] BeslPJ, McKayND. A method for registration of 3-d shapes. IEEE Trans Pattern Anal Mach Intel, 1992, 14(2): 239-256.

[5] BiswasD, SuH, WangC, StevanovicA, WangW. An automatic traffic density estimation using single shot detection (ssd) and mobilenet-ssd. Phys Chem Earth A/B/C, 2019, 110: 176-184.

[6] BradskiG. The opencv library. Dr Dobb’s J Softw Tools Prof Programmer, 2000, 25(11): 120-123

[7] Bs en 1504-9 (2008) Products and systems for the protection and repair of concrete structures—definitions, requirements, quality control and evaluation of conformity—part 9: General principles for the use of products and systems. British Standards Institution

[8] CaoMT, TranQV, NguyenNM, ChangKT. Survey on performance of deep learning models for detecting road damages using multiple dashcam image resources. Adv Eng Inform, 2020, 46101182.

[9] ChaYJ, ChoiWH, BuyukozturkO. Deep learning-based crack damage detection using convolutional neural networks. Comput Aided Civ Infrastruct Eng, 2017, 32(5): 361-378.

[10] ChenY, MedioniG. Object modeling by registration of multiple range images. Image Vision Comput, 1992, 10(3): 145-155.

[11] CortesC, VapnikV. Support-vector networks. Mach Learn, 1995, 20(3): 273-297.

[12] Dais D, Figueiredo E, Costa JS, Silva MAG (2021) Transfer learning and texture features for recognition of the conditions of construction materials with small data sets. J Comput Civ Eng 38(1). https://doi.org/10.1061/JCCEE5.CPENG-5478

[13] FarrarCR, WordenK. An introduction to structural health monitoring. Phil Trans R Soc A Math Phys Eng Sci, 2007, 365(1851): 303-315.

[14] Haykin S (1999) Neural Networks: A Comprehensive Foundation. Prentice Hall

[15] HuangX, ZhangY, WangL. Yolov5-based real-time defect detection for civil infrastructure inspection using uav imagery. J Infrastruct Syst, 2023, 29204023007.

[16] Huang J, Rathod V, Sun C, Zhu M, Korattikara A, Fathi A, Fischer I, Wojna Z, Song Y, Guadarrama S, Murphy K (2017) Speed/accuracy trade-offs for modern convolutional object detectors. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), IEEE, Piscataway, pp 7310–7311

[17] KaartinenE, DunphyK, SadhuA. Lidar-based structural health monitoring: Applications in civil infrastructure systems. Sensors, 2022, 22124610.

[18] KangJ, LeeS, KimJ. Autonomous uavs for structural health monitoring using deep learning and an ultrasonic beacon system with geo-tagging. Struct Control Health Monit, 2018, 2512e2249

[19] KimH, NarazakiY, SpencerBFJr. Automated bridge component recognition using close-range images from unmanned aerial vehicles. Eng Struct, 2023, 274115184.

[20] LeeJH, ParkJJ, YoonH. Automatic bridge design parameter extraction for scan-to-bim. Appl Sci, 2020, 10207346.

[21] LiH, ZhengX, WuJ. Efficientdet-based defect detection framework for masonry bridges under real-time constraints. J Civ Struct Health Monit, 2023, 13(4): 785-800.

[22] Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) Ssd: Single shot multibox detector. In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14, Springer, pp 21–37

[23] LutzMLearning Python: Powerful Object-Oriented Programming, 20135Sebastopol. O’Reilly Media.

[24] Mao A, Mohri M, Zhong Y (2023) Cross-entropy loss functions: Theoretical analysis and applications. arXiv preprint arXiv:2304.07288

[25] MaruccioC, BasilioI, OliveiraDV, LourençoPB, MontiG. Numerical modelling and parametric analysis of bond strength of masonry members retrofitted with frp. Constr Build Mater, 2014, 73: 713-727.

[26] MelbourneC, McKibbinsL, SawarN, GaillardCSMasonry arch bridges: condition appraisal and remedial treatment, 2006, London. Ciria.

[27] Microsoft (2022) SQL Server Studio. https://docs.microsoft.com/en-us/sql/ssms/download-sql-server-management-studio-ssms?view=sql-server-ver15. Accessed 30 Apr 2024

[28] MishraM, LourençoPB, RamanaGV. Structural health monitoring of civil engineering structures by using the internet of things: A review. J Build Eng, 2022, 48103954.

[29] Narazaki Y, Pang W, Wang G, Chai W (2023) Unsupervised domain adaptation approach for vision-based semantic understanding of bridge inspection scenes without manual annotations. J Bridg Eng 29(2). https://doi.org/10.1061/JBENF2.BEENG-6490

[30] OumaYO, HahnM. Wavelet-morphology based detection of incipient linear cracks in asphalt pavements from rgb camera imagery and classification using circular radon transform. Adv Eng Inform, 2016, 30(3): 481-499.

[31] ParkG, LeeJH, YoonH. Semantic structure from motion for railroad bridges using deep learning. Applied Sciences, 2021, 11104332.

[32] Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems (NeurIPS), Curran Associates, Inc., Red Hook, 28:91–99

[33] SaidiM, ReboulN, GaborA. Shear stress analysis in the textile-to-matrix and trc-to-masonry interfaces of textile-reinforced cement (trc) applied to masonry using distributed fibre optic sensors. J Build Eng, 2023, 78107764.

[34] Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC (2018) Mobilenetv2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 4510–4520. https://doi.org/10.1109/CVPR.2018.00474

[35] Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J Big Data 6(60). https://doi.org/10.1186/s40537-019-0197-0. Received 09 January 2019, Accepted 22 April 2019, Published 06 July 2019

[36] SpencerBFJr, SimSH, KimRE, YoonH. Advances in artificial intelligence for structural health monitoring: A comprehensive review. KSCE J Civ Eng, 2025, 293100203.

[37] Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: A simple way to prevent neural networks from overfitting. Mach Learn Res 15:1929–1958. Submitted 11/13; Published 6/14

[38] Van Rossum G, Foundation PS (2024) The Python Language Reference, Release 3.x. https://docs.python.org/3/reference/. Accessed 12 Apr 2025

[39] Wang X, Gao H, Jia Z, Li Z (2023) Bl-yolov8: An improved road defect detection model based on yolov8. Sensors 23(20):8361. https://doi.org/10.3390/s23208361. Submitted: 17 August 2023, Revised: 19 September 2023, Accepted: 5 October 2023, Published: 10 October 2023

[40] Wang Z, Xiao D-l (2009) New concept of prolonging the service life-span for asphalt pavement of expressways - theory and method of swingy lane line on flexible pavement of expressways. In: 2009 Second International Conference on Intelligent Computation Technology and Automation, Changsha, pp 223–226. https://doi.org/10.1109/ICICTA.2009.769

[41] WaqasM, HussainI, KhanA. Deep learning-based obstacle-avoiding autonomous uavs with fiducial marker-based localization for structural health monitoring. Sensors, 2023, 2331088

[42] WuY, LinK, BaiX, ZhangL, ZhouX. Sddnet: Real-time crack segmentation using spatial dynamic dilated convolution. Autom Constr, 2023, 145104626

[43] XieQ, HaoW, ZhangY, XuD. Compressive properties of traditional brick masonry with crack reinforced with gfrp bars embedded in the horizontal mortar joint. J Build Eng, 2023, 78107732.

[44] YangF, XuJ, XuY, WangQ. Efficient attention-based deep encoder and decoder for automatic crack segmentation. Eng Appl Artif Intell, 2022, 114105044

[45] YeumC, DykeSJ. Autonomous vision-based crack detection using deep learning and uavs for civil infrastructure. Struct Control Health Monit, 2017, 241e1919

[46] ZhangK, YuanY, ChenP, TangY. Hybrid pixel-level concrete crack segmentation and quantification across complex backgrounds using deep learning. Autom Constr, 2022, 139104292

[47] ZhangH, WangJ, LiuM, ChenW. Bridgedamage: a large-scale annotated dataset for visual bridge inspection using deep learning. Autom Constr, 2024, 166105601.

[48] ZhaoX, ZhangD, JiangY, LiJ. Deep learning-based automatic volumetric damage quantification using depth camera. Comput Aided Civ Infrastruct Eng, 2021, 36(3): 241-258

RIGHTS & PERMISSIONS

The Author(s)