Automated detection of bridge and road damage in orthophotos using deep learning

Abstract

When natural disasters occur, damage to roads and bridges not only disrupts transportation networks and endangers public safety but also affects the efficiency of rescue and evacuation work. Therefore, in the disaster response stage, the ability to quickly assess the situation in disaster-affected areas is extremely important. With the rapid development of technology, remote sensing provides a more efficient method for creating disaster maps compared to traditional methods. However, when dealing with large-scale data, there are still limitations in traditional image analysis methods based on remote sensing data. To solve this problem, this study proposes a binary deep learning framework to determine whether the bridges or roads in remote sensing data have been damaged, and provides an automated method for image analysis based on remote sensing data. According to the results, the EfficientNet-b1-based deep learning model excelled with an accuracy of 94.09%, which is considered sufficient for precisely conducting damage assessment after disasters. For future improvements, we suggest replacing binary classification with more precise semantic segmentation or object detection models, which can better identify and locate damaged areas, thereby enhancing the capability of damage detection.

Keywords

Deep learning / Damage detection / Orthophoto data / Remote sensing

References

[1] Alexander DE. Principles of emergency planning and management, 2002Oxford University Press

[2] Alhichri H, Alswayed AS, Bazi Y, Ammour N, Alajlan NA. Classification of remote sensing images using EfficientNet-B3 CNN model with attention. IEEE Access, 2021, 9: 14078-14094

[3] Alwis SD, Hou Z, Zhang Y, Na MH, Ologhi B, Sajjanhar A. A survey on smart farming data, applications and techniques. Computers in Industry, 2022, 138: 103624

[4] Borawar, L., & Kaur, R. (2023). ResNet: Solving vanishing gradient in deep networks. Proceedings of international conference on recent trends in computing (pp. 235–247). Springer Nature Singapore. https://doi.org/10.1007/978-981-19-8825-7_21

[5] Braik AM, Koliou M. Automated building damage assessment and large-scale mapping by integrating satellite imagery, GIS, and deep learning. Computer-Aided Civil and Infrastructure Engineering, 2024

[6] Chu H, Chun PJ. Fine-grained crack segmentation for high-resolution images via a multiscale cascaded network. Computer-Aided Civil and Infrastructure Engineering, 2024, 39: 575-594

[7] Chun PJ, Chu H, Shitara K, Yamane T, Maemura Y. Implementation of explanatory texts output for bridge damage in a bridge inspection web system. Advances in Engineering Software, 2024, 195: 103706

[8] Chun P, Kikuta T. Self-training with Bayesian neural networks and spatial priors for unsupervised domain adaptation in crack segmentation. Computer-Aided Civil and Infrastructure Engineering, 2024

[9] Cooner AJ, Shao Y, Campbell JB. Detection of urban damage using remote sensing and machine learning algorithms: Revisiting the 2010 Haiti earthquake. Remote Sensing, 2016, 8: 868

[10] Deng L, Yuan H, Long L, Chun P, Chen W, Chu H. Cascade refinement extraction network with active boundary loss for segmentation of concrete cracks from high-resolution images. Automation in Construction, 2024, 162: 105410

[11] Duarte D, Nex F, Kerle N, Vosselman G. Satellite image classification of building damages using airborne and satellite image samples in a deep learning approach. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2018, 4: 89-96

[12] Hamdi ZM, Brandmeier M, Straub C. Forest damage assessment using deep learning on high resolution remote sensing data. Remote Sensing, 2019, 11: 1976

[13] Hattori K, Niki K, Sugita A, Sugiyama T, Chun PJ. Deep learning-based corrosion inspection of long-span bridges with BIM integration. Heliyon, 2024

[14] He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–778. https://doi.org/10.1109/CVPR.2016.90

[15] Howard, A., Sandler, M., Chu, G., Chen, L.-C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., Vasudevan, V., Le, Q. V., & Adam, H. (2019). Searching for MobileNetV3. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 1314–1324

[16] Irwansyah E, Young H, Gunawan AA. Multi disaster building damage assessment with deep learning using satellite imagery data. International Journal of Intelligent Systems and Applications in Engineering, 2023, 11: 122-131

[17] Karimzadeh S, Ghasemi M, Matsuoka M, Yagi K, Zulfikar AC. A deep learning model for road damage detection after an earthquake based on synthetic aperture radar (SAR) and field datasets. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15: 5753-5765

[18] Kubo S, Yamane T, Chun P. Study on accuracy improvement of slope failure region detection using Mask R-CNN with augmentation method. Sensors, 2022, 22: 6412

[19] Kubo S, Zheng S, Chun P-J. Damage assessment of bridges using segmentation and shape analysis from aerial imagery. Proceedings of the Institution of Civil Engineers—Forensic Engineering, 2025

[20] Li X, Nishio M, Sugawara K, Iwanaga S, Chun P. Surrogate model development for slope stability analysis using machine learning. Sustainability, 2023, 15: 10793

[21] Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017). Feature pyramid networks for object detection. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 936–944. https://doi.org/10.1109/CVPR.2017.106

[22] Liu J, Wang M, Bao L, Li X. EfficientNet based recognition of maize diseases by leaf image classification. Journal of Physics: Conference Series, 2020, 1693(1 012148

[23] Maeda H, Sekimoto Y, Seto T, Kashiyama T, Omata H. Road damage detection and classification using deep neural networks with smartphone images. Computer-Aided Civil and Infrastructure Engineering, 2018, 33: 1127-1141

[24] Naddaf-Sh, S., Naddaf-Sh, M.-M., Kashani, A. R., & Zargarzadeh, H. (2020). An efficient and scalable deep learning approach for road damage detection. arXiv preprint arXiv:2011.09577. https://doi.org/10.48550/arXiv.2011.09577

[25] Prabhakaran AK, Nair JJ, Sarath S. Thermal facial expression recognition using modified ResNet152. Advances in computing and network communications, 2021Springer Singapore, 389-396

[26] Qian, S., Ning, C., & Hu, Y. (2021). MobileNetV3 for image classification. In 2021 IEEE 2nd International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE), 490–497. https://doi.org/10.1109/ICBAIE52039.2021.9389905

[27] Seydi ST, Rastiveis H. A deep learning framework for roads network damage assessment using post-earthquake LiDAR data. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2019, XLII-4/W18: 955-961

[28] Son E, Ha UR, Seong H, Park Y, Song J, Kim H. ResNet152-U-Net: Transfer learning for the semantic segmentation of forest restoration site using aerial images. Preprints, 2024

[29] Tan, M., & Le, Q. V. (2019). EfficientNet: Rethinking model scaling for convolutional neural networks. In Proceedings of the International Conference on Machine Learning (ICML) (pp. 6105–6114). PMLR

[30] Twumasi NYD, Shao Z, Orhan A. Remote sensing and GIS methods in urban disaster monitoring and management: An overview. International Journal of Trend in Scientific Research and Development, 2019, 3: 918-926

[31] Wang J, Yang L, Huo Z, He W, Luo J. Multi-label classification of fundus images with efficientnet. IEEE Access, 2020, 8: 212499-212508

[32] Yang Y, Han J. Real-time object detector based MobileNetV3 for UAV applications. Multimedia Tools and Applications, 2023, 82: 18709-18725

[33] Yu H, Miao X, Wang H. Bearing fault reconstruction diagnosis method based on ResNet-152 with multi-scale stacked receptive field. Sensors, 2022, 22: 1705

[34] Zhang, Y., & Chen, X. (2020). Lightweight semantic segmentation algorithm based on MobileNetV3 network. In 2020 International Conference on Intelligent Computing, Automation and Systems (ICICAS), 429–433. https://doi.org/10.1109/ICICAS51530.2020.00095

[35] Zhao, F., & Zhang, C. (2020). Building damage evaluation from satellite imagery using deep learning. In Proceedings of the IEEE 21st International Conference on Information Reuse and Integration for Data Science, 82–89. https://doi.org/10.1109/IRI49571.2020.00020

[36] Zhou J, Wang W, Lei B, Ge W, Huang Y, Zhang L, Yan Y, Zhou D, Ding Y, Wu J, Wang W. Automatic detection and classification of focal liver lesions based on deep convolutional neural networks: A preliminary study. Frontiers in Oncology, 2021, 10 581210

Rights & permissions

The Author(s)