Benchmarking YOLOv5 models for improved human detection in search and rescue missions

Abstract

Drone or unmanned aerial vehicle (UAV) technology has undergone significant changes. The technology allows UAV to carry out a wide range of tasks with an increasing level of sophistication, since drones can cover a large area with cameras. Meanwhile, the increasing number of computer vision applications utilizing deep learning provides a unique insight into such applications. The primary target in UAV-based detection applications is humans, yet aerial recordings are not included in the massive datasets used to train object detectors, which makes it necessary to gather the model data from such platforms. You only look once (YOLO) version 4, RetinaNet, faster region-based convolutional neural network (R-CNN), and cascade R-CNN are several well-known detectors that have been studied in the past using a variety of datasets to replicate rescue scenes. Here, we used the search and rescue (SAR) dataset to train the you only look once version 5 (YOLOv5) algorithm to validate its speed, accuracy, and low false detection rate. In comparison to YOLOv4 and R-CNN, the highest mean average accuracy of 96.9% is obtained by YOLOv5. For comparison, experimental findings utilizing the SAR and the human rescue imaging database on land (HERIDAL) datasets are presented. The results show that the YOLOv5-based approach is the most successful human detection model for SAR missions.

Key words： Unmanned aerial vehicle (UAV); Search and rescue (SAR); You look only once (YOLO) model; You only look once version 5 (YOLOv5); Unmanned aerial vehicle (UAV); Search and rescue (SAR); You look only once (YOLO) model; You only look once version 5 (YOLOv5)

References

[1]	R.J. Koester. Lost Person Behavior: A Search and Rescue. DBS Productions LLC, Charlottesville, USA (2008).
[2]	M.B. Bejiga, A. Zeggada, F. Melgani.Convolutional neural networks for near real-time object detection from UAV imagery in avalanche search and rescue operations. Proc. of IEEE Intl. Geoscience and Remote Sensing Symposium, Beijing(2016), pp. 693-696.
[3]	I. Martinez-Alpiste, G. Golcarenarenji, Q. Wang, J.M. Alcaraz-Calero. Search and rescue operation using UAVs: A case study. Expert Syst. Appl., 178 (Sept.2021). 114937:1-9.
[4]	M. Kampouraki, G.A. Wood, T.R. Brewer. Opportunities and limitations of object based image analysis for detecting urban impervious and vegetated surfaces using true-colour aerial photography. T. Blaschke, S. Lang, G.J. Hay (Eds.), Object-Based Image Analysis: Spatial Concepts for Knowledge-Driven Remote Sensing Applications, Springer, Berlin (2008), pp. 555-569.
[5]	C.A.B.Baker, S. Ramchurn, W.T. Luke Teacy, N.R. Jennings. Planning search and rescue missions for UAV teams. Proc. of the Twenty-Second European Conf. on Artificial Intelligence, Hague(2016), pp. 1777-1778.
[6]	K. Yun, L. Nguyen, T. Nguyen, et al. Small target detection for search and rescue operations using distributed deep learning and synthetic data generation. Proc. of SPIE 10995, Baltimore (2019). 1099507:1-6.
[7]	S.O. Murphy, C. Sreenan, K.N. Brown. Autonomous unmanned aerial vehicle for search and rescue using software defined radio. Proc. Of the IEEE 89th Vehicular Technology Conf., Kuala Lumpur (2019), pp. 1-6.
[8]	E. Lygouras, N. Santavas, A. Taitzoglou, K. Tarchanidis, A. Mitropoulos, A. Gasteratos. Unsupervised human detection with an embedded vision system on a fully autonomous UAV for search and rescue operations. Sensors, 19 (16) (Aug.2019). 3542:1-20.
[9]	L.-C. Jiao, F. Zhang, F. Liu, et al. A survey of deep learning-based object detection. IEEE Access, 7 (Sept.2019), pp. 128837-128868.
[10]	S.-Q. Ren, K.-M. He, R. Girshick, J. Sun. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE T. Pattern Anal., 39 (6) (Jun.2017), pp. 1137-1149.
[11]	T.-Y. Lin, P. Goyal, R. Girshick, K.-M. He, P. Dollár. Focal loss for dense object detection. Proc. of IEEE Intl. Conf. on Computer Vision, Venice(2017), pp. 2999-3007.
[12]	Z.-W. Cai, N. Vasconcelos. Cascade R-CNN: delving into high quality object detection. Proc. of IEEE/CVF Conf. on Computer Vision and Pattern Recognition, Salt Lake City (2018), pp. 6154-6162.
[13]	J. Redmon, S. Divvala, R. Girshick, A. Farhadi. You only look once: unified, real-time object detection. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, Las Vegas (2016), pp. 779-788.
[14]	W. Liu, D. Anguelov, D. Erhan, et al. SSD: single shot MultiBox detector. Proc. Of the 14th European Conf. on Computer Vision, Amsterdam (2016), pp. 21-37.
[15]	Z. Ali, Q. Memon. Time delay tracking for multiuser synchronization in CDMA networks. J. Network., 8 (9) (Oct.2013), pp. 1929-1935.
[16]	M. Tan, R. Pang, Q.V. Le. EfficientDet: Scalable and Efficient Object Detection. Proc. of IEEE/CVF Conf. on Computer Vision and Pattern Recognition, Seattle (2020), pp. 10778-10787.
[17]	M. Tan, Q.V. Le. EfficientNet: rethinking model scaling for convolutional neural networks. Proc. of 36th Intl. Conf. on Machine Learning, Long Beach, California (2019).
[18]	J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei. ImageNet: a large-scale hierarchical image database. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, Miami(2009), pp. 248-255.
[19]	T.-Y. Lin, M. Maire, S. Belongie, et al. Microsoft COCO: common objects in context. Proc. of the 13th European Conf. on Computer Vision, Zurich (2014), pp. 740-755.
[20]	M. Rostami, S. Kolouri, E. Eaton, K. Kim. Deep transfer learning for few-shot SAR image classification. Rem. Sens., 11 (11) (Jun.2019). 1374:1-19.
[21]	X.-D. Zhang, K. Zhu, G.-Z. Chen, et al. Geospatial object detection on high resolution remote sensing imagery based on double multi-scale feature pyramid network. Rem. Sens., 11 (7) (Mar.2019). 755:1-27.
[22]	S. Sambolek, M. Ivasic-Kos. Automatic person detection in search and rescue operations using deep CNN detectors. IEEE Access, 9 (2021), pp. 37905-37922.
[23]	M.K. Vasi?, V. Papi?. Improving the model for person detection in aerial image sequences using the displacement vector: A search and rescue scenario. Drones, 6 (1) (Jan.2022), pp. 1-19.
[24]	M.G. Dorrer, A.E. Tolmacheva. Comparison of the YOLOv3 and Mask R-CNN architectures’ efficiency in the smart refrigerator’s computer vision. J. Phys. Conf., 1679 (4)(Nov. 2020). 042022:1-12.
[25]	J.A. Kim, J.Y. Sung, S.H. Park.Comparison of Faster-RCNN, YOLO, and SSD for real-time vehicle type recognition. Proc.of IEEE Intl. Conf. on Consumer Electronics-Asia, Seoul(2020), pp. 1-4.
[26]	Y.-T. Mao. A pedestrian detection algorithm for low light and dense crowd Based on improved YOLO algorithm. Proc. of MATEC Web of Conf 355 (Jan.2022). 03020:1-16.
[27]	C. Wang, W. He, Y. Nie, J. Guo, C. Liu, K. Han, Y. Wang. Gold-YOLO: Efficient Object Detector via Gather-And-Distribute Mechanism. Proc. of Thirty-seventh Conf. on Neural Information Processing Systems, New Orleans (2023).
[28]	Learn OpenCV. YOLOv5: Expert guide to custom object detection training. (2022).
	[Online]. Available:. https://learnopencv.com/custom-object-detection-training-using-yolov5/. May 29, 2023.
[29]	Paperspace by DigitalOcean. How to train YOLO v5 on a custom dataset. (2021).
	[Online]. Available:. https://blog.paperspace.com/train-yolov5-custom-data/. May 29, 2023.
[30]	F. Bashir, F. Porikli. Performance evaluation of object detection and tracking systems. Proc. of the 9th IEEE Intl. Workshop On PETS, New York (2006), pp. 7-14.
[31]	N. Bachir, Q. Memon. Investigating YOLOv5 for search and rescue operations involving UAVs: investigating YOLO5. Proc. of the 5th Intl. Conf. on Control and Computer Vision, Xiamen (2022), pp. 200-204.
[32]	S. Caputo, G. Castellano, F. Greco, C. Mencar, N. Petti, G. Vessio. Human detection in drone images using YOLO for search-and-rescue operations. Proc. of the 20th Intl. Conf. of AIxIA2021 - Advances in Artificial Intelligence, Milan (2022), pp. 326-337.
[33]	M. Aljabri, M. AlAmir, M. AlGhamdi, M. Abdel-Mottaleb, F. Collado-Mesa. Towards a better understanding of annotation tools for medical imaging: a survey. Multimed. Tool. Appl., 81 (18) (Jul.2022), pp. 25877-25911.