Multi-UAV path planning for multiple emergency payloads delivery in natural disaster scenarios

Abstract

Unmanned aerial vehicles (UAVs) are widely used in situations with uncertain and risky areas lacking network coverage. In natural disasters, timely delivery of first aid supplies is crucial. Current UAVs face risks such as crashing into birds or unexpected structures. Airdrop systems with parachutes risk dispersing payloads away from target locations. The objective here is to use multiple UAVs to distribute payloads cooperatively to assigned locations. The civil defense department must balance coverage, accurate landing, and flight safety while considering battery power and capability. Deep Q-network (DQN) models are commonly used in multi-UAV path planning to effectively represent the surroundings and action spaces. Earlier strategies focused on advanced DQNs for UAV path planning in different configurations, but rarely addressed non-cooperative scenarios and disaster environments. This paper introduces a new DQN framework to tackle challenges in disaster environments. It considers unforeseen structures and birds that could cause UAV crashes and assumes urgent landing zones and winch-based airdrop systems for precise delivery and return. A new DQN model is developed, which incorporates the battery life, safe flying distance between UAVs, and remaining delivery points to encode surrounding hazards into the state space and Q-networks. Additionally, a unique reward system is created to improve UAV action sequences for better delivery coverage and safe landings. The experimental results demonstrate that multi-UAV first aid delivery in disaster environments can achieve advanced performance.

Keywords

Deep Q-network / First aid delivery / Multi-UAV path planning / Reinforcement learning / Unmanned aerial vehicle (UAV)

CRediT authorship contribution statement

Zarina Kutpanova: Conceptualization, Methodology, Writing−original draft. Mustafa Kadhim: Methodology, Validation, Writing−original draft. Xu Zheng: Data curation, Visualization. Nurkhat Zhakiyev: Supervision, Writing−review and editing.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgment

This work was supported by the Committee of Science of the Ministry of Education and Science of the Republic of Kazakhstan under Grant No. 249015/0224.

References

[1] L. Xu, X.-B. Cao, W.-B. Du, Y.-M. Li, Cooperative path planning optimization for multiple UAVs with communication constraints, Knowl.-Based Syst. 260 (2023) 110164.

[2] D. Yedilkhan, A.E. Kyzyrkanov, Z.A. Kutpanova, S. Aljawarneh, S.K. Atanov, Intelligent obstacle avoidance algorithm for safe urban monitoring with autonomous mobile drones, Journal of Electronic Science and Technology 22 (2024) 100277.

[3] R. Omirgaliyev, N. Zhakiyev, N. Aitbayeva, Y. Akhmetbekov, Application of machine learning methods for the analysis of heat energy consumption by zones with a change in outdoor temperature: case study for Nur-Sultan City, Int. J. Sustain. Dev. Plann. 17 (4) (2022) 1247-1257.

[4] K. Ellis, M. Johnson, N. Neogi, J. Homola, NASA research to expand UAS operations for disaster response, in: Proc. of the 34th Congress of the Intl. Council of the Aeronautical Sciences, Florence, USA, (2024), pp. 1-11.

[5] Q.-C. Meng, K. Chen, Q.-J. Qu, PPSwarm: multi-UAV path planning based on hybrid PSO in complex scenarios, Drones 8 (5) (2024) 192.

[6] W.-D. Wu, J.-H. Li, Y.-L. Wu, X.-G. Ren, Y.-H. Tang, Multi-UAV adaptive path planning in complex environment based on behavior tree, in: Proc. of the 16th EAI Intl. Conf. Collaborative Computing: Networking, Applications and Worksharing, Shanghai, China, (2021), pp. 494-505.

[7] S.-M. Qiu, J.-K. Dai, D.-S. Zhao, Path planning of an unmanned aerial vehicle based on a multi-strategy improved pelican optimization algorithm, Biomimetics 9 (10) (2024) 647.

[8] Y.-Y. Sheng, H.-Y. Liu, J.-B. Li, Q. Han, UAV autonomous navigation based on deep reinforcement learning in highly dynamic and high-density environments, Drones 8 (9) (2024) 516.

[9] H.X. Pham, H.M. La, D. Feil-Seifer, L. Van Nguyen, Reinforcement learning for autonomous UAV navigation using function approximation, in: Proc. of IEEE Intl. Symposium on Safety, Security, and Rescue Robotics, Philadelphia, USA, (2018), pp. 1-6.

[10] J.-J. Liang, A.-K. Qin, P.N. Suganthan, S. Baskar, Comprehensive learning particle swarm optimizer for global optimization of multimodal functions, IEEE T. Evolut. Comput. 10 (3) (2006) 281-295.

[11] Y. Zeng, X.-L. Xu, S. Jin, R. Zhang, Simultaneous navigation and radio mapping for cellular-connected UAV with deep reinforcement learning, IEEE T. Wirel. Commun. 20 (7) (2021) 4205-4220.

[12] S. Poudel, S. Moh, Priority-aware task assignment and path planning for efficient and load-balanced multi-UAV operation, Veh. Commu. 42 (2023) 100633.

[13] X. Liu, Z.-Y. Chai, Y.-L. Li, Y.-Y. Cheng, Y. Zeng, Multi-objective deep reinforcement learning for computation offloading in UAV-assisted multi-access edge computing, Inform. Sciences 642 (2023) 119154.

[14] Y. Wu, W.-L. Fan, W.-W. Yang, X.-L. Sun, X.-R. Guan, Robust trajectory and communication design for multi-UAV enabled wireless networks in the presence of jammers, IEEE Access 8 (2020) 2893-2905.

[15] G. Hu, J.-Y. Zhong, G. Wei, SaCHBA_PDN: modified honey badger algorithm with multi-strategy for UAV path planning, Expert Syst. Appl. 223 (2023) 119941.

[16] O. Walker, F. Vanegas, F. Gonzalez, S. Koenig, A deep reinforcement learning framework for UAV navigation in indoor environments, in: Proc. of IEEE Aerospace Conf., Big Sky, USA, (2019), pp. 1-14.

[17] H. Bayerlein, M. Theile, M. Caccamo, D. Gesbert, Multi-UAV path planning for wireless data harvesting with deep reinforcement learning, IEEE Open J. Comm. Soc. 2 (2021) 1171-1187.

[18] S. Dutta, P. Bansal, Behavior of parachute under varying in-flight condition—a review, Res. J. Text. Appar. 28 (4) (2024) 699-715.

[19] M. Bekhti, N. Achir, K. Boussetta, M. Abdennebi, Drone package delivery: a heuristic approach for UAVs path planning and tracking, EAI Endorsed T. Internet of Things 3 (9) (2017) e1.

[20] F. Franke, M. Schwab, U. Burger, C. Hühne, An analytical model to determine the impact force of drone strikes, CEAS Aeronautical Journal 13 (1) (2022) 69-84.

[21] Y.-W. Ho, T.-H. Lin, T. Akamatsu, L. Karczmarski, Fine-scale spatial variability of marine acoustic environment corresponds with habitat utilization of indo-pacific humpback dolphins in Hong Kong waters, Ecol. Indic. 158 (2024) 111228.

[22] C. Hütt, A. Bolten, H. Hüging, G. Bareth, UAV LiDAR metrics for monitoring crop height, biomass and nitrogen uptake: a case study on a winter wheat field trial, PFG-J. Photogramm. Rem. 91 (2) (2023) 65-76.

[23] L. Pugi, A. Mela, A. Reatti, A. Casazza, R. Fiorenzani, G. Mattei, A fixed wing UAV with VTOL capabilities: design, control and energy management, Int. J. Model. Identif. 41 (3) (2022) 206-221.

[24] A. Panta, A. Mohamed, M. Marino, S. Watkins, A. Fisher, Unconventional control solutions for small fixed wing unmanned aircraft, Prog. Aero. Sci. 102 (2018) 122-135.

[25] C. Lee, S. Kim, B. Chu, A survey: flight mechanism and mechanical structure of the UAV, Int. J. Precis. Eng. Man. 22 (4) (2021) 719-743.

[26] H. Ucgun, U. Yuzgec, C. Bayilmis, A review on applications of rotary-wing unmanned aerial vehicle charging stations, Int. J. Adv. Robot. Syst. 18 (3) (2021) https://doi.org/10.1177/17298814211015863.

[27] Z.-L. Liu, R. Sengupta, A. Kurzhanskiy, A power consumption model for multi-rotor small unmanned aircraft systems, in: Proc. of Intl. Conf. on Unmanned Aircraft Systems, Miami, USA, (2017), pp. 310-315.

[28] C.-C. Bai, P. Yan, X.-Q. Yu, J.-F. Guo, Learning-based resilience guarantee for multi-UAV collaborative QoS management, Pattern Recogn. 122 (2021) 108166.