Enhanced pothole detection system using YOLOX algorithm

Abstract

The road is the most commonly used means of transportation and serves as a country’s arteries, so it is extremely important to keep the roads in good condition. Potholes that happen to appear in the road must be repaired to keep the road in good condition. Spotting potholes on the road is difficult, especially in a country like India where roads stretch millions of kilometres across the country. Therefore, there is a need to automate the identification of potholes with high speed and real-time precision. YOLOX is an object detection algorithm and our main goal of this article is to train and analyse the YOLOX model for pothole detection. The YOLOX model is trained with a pothole dataset and the results obtained are analysed by calculating the accuracy, recall and size of the model which is then compared to other YOLO algorithms. The experimental results in this article show that the YOLOX-Nano model predicts potholes with higher accuracy compared to other models while having low computational costs. We were able to achieve an Average Precision (AP) value of 85.6% from training the model and the total size of the model is 7.22 MB. The pothole detection capabilities of the newly developed YOLOX algorithm have never been tested before and this paper is one of the first to detect potholes using the YOLOX object detection algorithm. The research conducted in this paper will help reduce costs and increase the speed of pothole identification and will be of great help in road maintenance.

Keywords

YOLO / YOLOX / Object detection / Pothole detection / Machine learning

References

[1] SharmaS.K., SharmaR.C.. KumarM., PandeyR.K., KumarV.. Pothole detection and warning system for Indian roads. Advances in Interdisciplinary Engineering, 2019 Singapore Springer 511-519

[2] WangH.-W., ChenC.-H., ChengD.-Y., LinC.-H., LoC.-C.. A real-time pothole detection approach for intelligent transportation system. Math. Probl. Eng., 2015, 2015

[3] SharmaS.K., PhanH., LeeJ.. An application study on road surface monitoring using dtw based image processing and ultrasonic sensors. Appl. Sci., 2020, 10(13

[4] AhmedA., AshfaqueM., UlhaqM.U., MathavanS., KamalK., RahmanM.. Pothole 3D reconstruction with a novel imaging system and structure from motion techniques. IEEE Trans. Intell. Transp. Syst., 2022, 23 5):4685-4694

[5] SheX., HongweiZ., WangZ., YanJ.. Feasibility study of asphalt pavement pothole properties measurement using 3D line laser technology. Int. J. Transp. Sci. Technol., 2021, 10(1):83-92

[6] PramestyaR.H., SulistyaningrumD.R., SetiyonoB., MukhlashI., FirdausZ.. Road defect classification using gray level co-occurrence matrix (GLCM) and radial basis function (RBF). 2018 10th International Conference on Information Technology and Electrical Engineering (ICITEE), 2018 285-289

[7] SutrisnoI., SyauqiA., HasinM., IskandarR.M., AsmaraI., SuwondoI., ArdiansyahW.D., SetiawanE.. Design of pothole detector using gray level co-occurrence matrix (GLCM) and neural network (NN). IOP Conf. Ser., Mater. Sci. Eng., 2020, 874

[8] MuslimM., SulistyaningrumD., SetiyonoB.. Detection and counting potholes using morphological method from road video. AIP Conf. Proc., 2020, 2242

[9] RyuS.-K., KimT., KimY.-R.. Image-based pothole detection system for its service and road management system. Math. Probl. Eng., 2015, 2015

[10] YousafM.H., AzharK., MurtazaF., HussainF.. Visual analysis of asphalt pavement for detection and localization of potholes. Adv. Eng. Inform., 2018, 38: 527-537

[11] WuC., WangZ., HuS., LepineJ., NaX., AinalisD., StettlerM.. An automated machine-learning approach for road pothole detection using smartphone sensor data. Sensors, 2020, 20(19

[12] HoqueS., ArafatM.Y., XuS., MaitiA., WeiY.. A comprehensive review on 3D object detection and 6D pose estimation with deep learning. IEEE Access, 2021, 9: 143746-143770

[13] HoangN.-D.. An artificial intelligence method for asphalt pavement pothole detection using least squares support vector machine and neural network with steerable filter based feature extraction. Adv. Civ. Eng., 2018, 2018

[14] LiuT., LiuY.. Moving camera-based object tracking using adaptive ground plane estimation and constrained multiple kernels. J. Adv. Transp., 2021, 2021

[15] BajammalM., StoccoA., MazinanianD., MesbahA.. A survey on the use of computer vision to improve software engineering tasks. IEEE Trans. Softw. Eng., 2022, 48 5):1722-1742

[16] MinaeeS., BoykovY.Y., PorikliF., PlazaA.J., KehtarnavazN., TerzopoulosD.. Image segmentation using deep learning: a survey. IEEE Trans. Pattern Anal. Mach. Intell., 2022, 44(7):3523-3542

[17] BibiR., SaeedY., ZebA., GhazalT., SaidR., AbbasS., AhmadM., KhanM.. Edge AI-based automated detection and classification of road anomalies in VANET using deep learning. Comput. Intell. Neurosci., 2021, 2021

[18] AnandhalliM., TanujaA., BaligarV.P., BaligarP.. Indian pothole detection based on CNN and anchor-based deep learning method. Int. J. Inf. Technol., 2022

[19] LuY., GuoY., LiangM.. CNN-enabled visibility enhancement framework for vessel detection under haze environment. J. Adv. Transp., 2021, 2021

[20] LuoD., LuJ., GuoG.. Road anomaly detection through deep learning approaches. IEEE Access, 2020, 8: 117390-117404

[21] RenS., HeK., GirshickR., SunJ.. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell., 2017, 39(6):1137-1149

[22] J. Redmon, S. Divvala, R. Girshick, A. Farhadi, You only look once: unified, real-time object detection (2015). arXiv:1506.02640

[23] KawanoM., MikamiK., YokoyamaS., YonezawaT., NakazawaJ.. Road marking blur detection with drive recorder. 2017 IEEE International Conference on Big Data (Big Data), 2017 4092-4097

[24] FelzenszwalbP., GirshickR., McallesterD., RamananD.. Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell., 2010, 32: 1627-1645

[25] J. Redmon, A. Farhadi, YOLO9000: better, faster, stronger. (2016). arXiv:1612.08242

[26] EveringhamM., Van GoolL., WilliamsC.K.I., WinnJ., ZissermanA.. The Pascal visual object classes (VOC) challenge. Int. J. Comput. Vis., 2010, 88(2):303-338

[27] LiuW., AnguelovD., ErhanD., SzegedyC., ReedS., FuC.-Y., BergA.C.. SSD: single shot MultiBox detector. Computer Vision—ECCV 2016, 2016 Cham Springer 21-37

[28] SumalathaR., RaoR.V., DeviS.M.R.. IyerB., GhoshD., BalasV.E.. Pothole detection using yolov2 object detection network and convolutional neural network. Applied Information Processing Systems, 2022 Singapore Springer 293-300

[29] J. Redmon, A. Farhadi, YOLOv3: an incremental improvement (2018). arXiv:1804.02767

[30] DharneeshkarJ., DhakshanaV., AniruthanS., KarthikaR., ParameswaranL.. Deep learning based detection of potholes in Indian roads using YOLO. 2020 International Conference on Inventive Computation Technologies (ICICT), 2020 381-385

[31] UkhwahE.N., YuniarnoE.M., SupraptoY.K.. Asphalt pavement pothole detection using deep learning method based on yolo neural network. 2019 International Seminar on Intelligent Technology and Its Applications (ISITIA), 2019 35-40

[32] ChitaleP.A., KekreK.Y., ShenaiH.R., KaraniR., GalaJ.P.. Pothole detection and dimension estimation system using deep learning (YOLO) and image processing. 2020 35th International Conference on Image and Vision Computing New Zealand (IVCNZ), 2020 1-6

[33] LinY.-C., ChenW.-H., KuoC.-H.. Implementation of pavement defect detection system on edge computing platform. Appl. Sci., 2021, 11 8

[34] A. Bochkovskiy, C.-Y. Wang, H.-Y.M. Liao, YOLOv4: optimal speed and accuracy of object detection (2020). arXiv:2004.10934

[35] OmarM., KumarP.. Detection of roads potholes using YOLOv4. 2020 International Conference on Information Science and Communications Technologies (ICISCT), 2020 1-6

[36] ParkS.-S., TranV.-T., LeeD.-E.. Application of various YOLO models for computer vision-based real-time pothole detection. Appl. Sci., 2021, 11 23

[37] G. Jocher, A. Chaurasia, A. Stoken, J. Borovec, NanoCode012, Y. Kwon, TaoXie, J. Fang, imyhxy, K. Michael, Lorna, V. Abhiram, D. Montes, J. Nadar, Laughing, tkianai, yxNONG, P. Skalski, Z. Wang, A. Hogan, C. Fati, L. Mammana, AlexWang1900, D. Patel, D. Yiwei, F. You, J. Hajek, L. Diaconu, M.T. Minh, ultralytics/yolov5: v6.1—TensorRT, TensorFlow Edge TPU and OpenVINO Export and Inference. https://doi.org/10.5281/zenodo.6222936

[38] AhmedK.R.. Smart pothole detection using deep learning based on dilated convolution. Sensors, 2021, 21 24

[39] C.-Y. Wang, I.-H. Yeh, H.-Y.M. Liao, You only learn one representation: unified network for multiple tasks (2021). arXiv:2105.04206

[40] Z. Ge, S. Liu, F. Wang, Z. Li, J. Sun, YOLOX: exceeding YOLO series in 2021 (2021). arXiv:2107.08430

[41] LiuW., AnguelovD., ErhanD., SzegedyC., ReedS., FuC.-Y., BergA.C.. LeibeB., MatasJ., SebeN., WellingM.. SSD: single shot multibox detector. Computer Vision—ECCV 2016, 2016 Cham Springer 21-37

[42] PanboonyuenT., ThongbaiS., WongweeranimitW., SantitamnontP., SuphanK., CharoenphonC.. Object detection of road assets using transformer-based yolox with feature pyramid decoder on Thai highway panorama. Information, 2022, 13 1

[43] Pothole Dataset. https://public.roboflow.com/object-detection/pothole. Accessed: 2022-02-05

[44] Carranza-GarcíaM., Torres-MateoJ., Lara-BenítezP., García-GutiérrezJ.. On the performance of one-stage and two-stage object detectors in autonomous vehicles using camera data. Remote Sens., 2021, 13 1

[45] LuX., LiQ., LiB., YanJ.. VedaldiA., BischofH., BroxT., FrahmJ.-M.. Mimicdet: bridging the gap between one-stage and two-stage object detection. Computer Vision—ECCV 2020, 2020 Cham Springer 541-557

[46] A. Lohia, K. Kadam, R. Joshi, D. Bongale, Bibliometric analysis of one-stage and two-stage object detection (2021)

[47] P. Soviany, R.T. Ionescu, Optimizing the trade-off between single-stage and two-stage object detectors using image difficulty prediction (2018). arXiv:1803.08707

[48] PoirsonP., AmmiratoP., FuC., LiuW., KoseckaJ., BergA.C.. Fast single shot detection and pose estimation. 2016 Fourth International Conference on 3D Vision (3DV), 2016 Los Alamitos IEEE Computer Society 676-684

[49] LiuS., ZhouH., LiC., WangS.. Analysis of anchor-based and anchor-free object detection methods based on deep learning. 2020 IEEE International Conference on Mechatronics and Automation (ICMA), 2020 1058-1065

[50] ZhangT., LiZ., SunZ., ZhuL.. A fully convolutional anchor-free object detector. Vis. Comput., 2022

[51] Z. Tian, C. Shen, H. Chen, T. He, FCOS: fully convolutional one-stage object detection (2019). arXiv:1904.01355

[52] HaoW., ZhiliS.. Improved mosaic: algorithms for more complex images. J. Phys. Conf. Ser., 2020, 1684 1

[53] H. Zhang, M. Cisse, Y.N. Dauphin, D. Lopez-Paz, mixup: beyond empirical risk minimization (2017). arXiv:1710.09412

[54] Z. Ge, S. Liu, Z. Li, O. Yoshie, J. Sun, OTA: optimal transport assignment for object detection (2021). arXiv:2103.14259

[55] N. Bodla, B. Singh, R. Chellappa, L.S. Davis, Soft-NMS—improving object detection with one line of code (2017). arXiv:1704.04503

[56] T.-Y. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C.L. Zitnick, P. Dollár, Microsoft COCO: common objects in context (2014). arXiv:1405.0312