Road sub-surface defect detection based on gprMax forward simulation-sample generation and Swin Transformer-YOLOX

Abstract

Training samples for deep learning networks are typically obtained through various field experiments, which require significant manpower, resource and time consumption. However, it is possible to utilize simulated data to augment the training samples. In this paper, by comparing the actual experimental model with the simulated model generated by the gprMax [1] forward simulation method, the feasibility of obtaining simulated samples through gprMax simulation is validated. Subsequently, the samples generated by gprMax forward simulation are used for training the network to detect objects in existing real samples. At the same time, aiming at the detection and intelligent recognition of road sub-surface defects, the Swin-YOLOX algorithm is introduced, and the excellence of the detection network, which is improved by augmenting the simulated samples with real samples, is further verified. By comparing the prediction performance of the object detection models, it is observed that the model trained with mixed samples achieved a recall of 94.74% and a mean average precision (mAP) of 97.71%, surpassing the model trained only on real samples by 12.95% and 15.64%, respectively. The feasibility and excellence of training the model with mixed samples are confirmed. The potential of using a fusion of simulated and existing real samples instead of repeatedly acquiring new real samples by field experiment is demonstrated by this study, thereby improving detection efficiency, saving resources, and providing a new approach to the problem of multiple interpretations in ground penetrating radar (GPR) data.

Graphical abstract

Keywords

ground penetrating radar / gprMax / forward modeling / sample generation / Swin-YOLOX / object detection

References

[1] Wang Z F, Wang J, Chen K F, Li Z P, Xu J, Li Y, Sui Q M. Unsupervised learning method for rebar signal suppression and defect signal reconstruction and detection in ground penetrating radar images. Measurement, 2023, 211: 112652

[2] Wai-Lok Lai W, Dérobert X, Annan P. A review of ground penetrating radar application in civil engineering: A 30-year journey from Locating and Testing to Imaging and Diagnosis. NDT & E International, 2018, 96: 58–78

[3] Guo H, Zhuang X, Rabczuk T. A deep collocation method for the bending analysis of Kirchhoff plate. Computers, Materials & Continua, 2019, 59(2): 433–456

[4] Dong Z H, Ye S B, Gao Y Z, Fang G Y, Zhang X J, Xue Z J, Zhang T. Rapid detection methods for asphalt pavement thicknesses and defects by a vehicle-mounted ground penetrating radar (GPR) system. Sensors, 2016, 16(12): 2067

[5] Tong Z, Gao J, Zhang H T. Innovative method for recognizing subgrade defects based on a convolutional neural network. Construction & Building Materials, 2018, 169: 69–82

[6] Samaniego E, Anitescu C, Goswami S, Nguyen-Thanh V M, Guo H, Hamdia K, Zhuang X, Rabczuk T. An energy approach to the solution of partial differential equations in computational mechanics via machine learning: Concepts, implementation and applications. Computer Methods in Applied Mechanics and Engineering, 2020, 362: 112790

[7] Zhuang X, Guo H, Alajlan N, Zhu H, Rabczuk T. Deep autoencoder based energy method for the bending, vibration, and buckling analysis of Kirchhoff plates with transfer learning. European Journal of Mechanics. A, Solids, 2021, 87: 104225

[8] Guo H, Zhuang X, Chen P, Alajlan N, Rabczuk T. Stochastic deep collocation method based on neural architecture search and transfer learning for heterogeneous porous media. Engineering with Computers, 2022, 38(6): 5173–5198

[9] Yue Y P, Liu H, Meng X, Li Y G, Du Y L. Generation of high-precision ground penetrating radar images using improved least square generative adversarial networks. Remote Sensing, 2021, 13(22): 4590

[10] Ye W J, Liu C J, Chen Y H, Liu Y J, Liu C M, Zhou H H. Multi-style transfer and fusion of image’s regions based on attention mechanism and instance segmentation. Signal Processing Image Communication, 2023, 110: 116871

[11] Guo H, Zhuang X, Chen P, Alajlan N, Rabczuk T. Analysis of three-dimensional potential problems in non-homogeneous media with physics-informed deep collocation method using material transfer learning and sensitivity analysis. Engineering with Computers, 2022, 38(6): 5423–5444

[12] Guo H, Zhuang X, Fu X, Zhu Y, Rabczuk T. Physics-informed deep learning for three-dimensional transient heat transfer analysis of functionally graded materials. Computational Mechanics, 2023, 72(3): 513–524

[13] Guo H, Zhuang X, Alajlan N, Rabczuk T. Physics-informed deep learning for melting heat transfer analysis with model-based transfer learning. Computers & Mathematics with Applications (Oxford, England), 2023, 143: 303–317

[14] Gao R X, Zhu H Q, Liao Q, Qu B L, Hu L T, Wang H R. Detection of coal fire by deep learning using ground penetrating radar. Measurement, 2022, 201: 111585

[15] Warren C, Giannopoulos A, Giannakis I. gprMax: Open source software to simulate electromagnetic wave propagation for ground penetrating radar. Computer Physics Communications, 2016, 209: 163–170

[16] Berenger J. Perfectly matched layer for the FDTD solution of wave-structure interaction problems. IEEE Transactions on Antennas and Propagation, 1996, 44(1): 110–117

[17] Zhang J, Yang X, Li W G, Zhang S B, Jia Y Y. Automatic detection of moisture damages in asphalt pavements from GPR data with deep CNN and IRS method. Automation in Construction, 2020, 113: 103119

[18] Fauchard C, Dérobert X, Cariou J, Côte P. GPR performances for thickness calibration on road test sites. NDT & E International, 2003, 36(2): 67–75

[19] Khudoyarov S, Kim N, Lee J J. Three-dimensional convolutional neural network-based underground object classification using three-dimensional ground penetrating radar data. Structural Health Monitoring, 2020, 19(6): 1884–1893

[20] Rabczuk T, Ren H, Zhuang X. A nonlocal operator method for partial differential equations with application to electromagnetic waveguide problem. Computers, Materials & Continua, 2019, 59(1): 31–55

[21] Kane Y. Numerical solution of initial boundary value problems involving maxwell’s equations in isotropic media. IEEE Transactions on Antennas and Propagation, 1966, 14(3): 302–307

[22] KunzK SLuebbers R J. The Finite Difference Time Domain Method for Electromagnetics. Leiden: CRC Press, 1993

[23] Ren H, Zhuang X, Rabczuk T. A higher order nonlocal operator method for solving partial differential equations. Computer Methods in Applied Mechanics and Engineering, 2020, 367: 113132

[24] DosovitskiyABeyerLKolesnikov AWeissenbornDZhaiXUnterthiner TDehghaniMMindererMHeigoldG GellySUszkoreit JHoulsbyN. An image is worth 16 × 16 words: Transformers for image recognition at scale. 2020, arXiv:2010.11929

[25] GeZLiuS T WangFLi Z MSunJ. YOLOX: Exceeding YOLO series in 2021. 2021, arXiv:2107.08430

[26] Ji W, Liu Q J, Huang C W, Yang R, Huang H L, Xu G H. YOLOX traffic sign detection based on Swin-Transformer. Radio Communications Technology, 2023, 49(3): 547–555

[27] Wu G X, Li Y C. Non-maximum suppression for object detection based on the chaotic whale optimization algorithm. Journal of Visual Communication and Image Representation, 2021, 74: 102985

[28] Zheng C W, Lin H. YOLOv5 helmet wearing detection method based on Swin Transformer. Computer Measurement and Control, 2023, 31(3): 15–21

[29] Iqbal A, Sharif M. BTS-ST: Swin transformer network for segmentation and classification of multimodality breast cancer images. Knowledge-Based Systems, 2023, 267: 110393

[30] Üzen H, Türkoğlu M, Yanikoglu B, Hanbay D. Swin-MFINet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications, 2022, 209: 118269

[31] Yang H N, Yang D P. CSwin-PNet: A CNN-Swin Transformer combined pyramid network for breast lesion segmentation in ultrasound images. Expert Systems with Applications, 2023, 213: 119024

[32] Jiang S, Kong R N, Li P C, lu C W, Zhang S, Li M. Intelligent detection algorithm of obstacles in front of open-pit mine cars based on Swin Transformer and CNN. Metal Mine, 2023, 5: 228–236

[33] Lu S L, Liu X Y, He Z X, Zhang X, Liu W B, Karkee M. Swin-Transformer-YOLOv5 for real-time wine grape bunch detection. Remote Sensing, 2022, 14(22): 5853

[34] Ishitsuka K, Iso S, Onishi K, Matsuoka T. Object detection in ground-penetrating radar images using a deep convolutional neural network and image set preparation by migration. International Journal of Geophysics, 2018, 2018: 1–8

RIGHTS & PERMISSIONS

Higher Education Press