DDQN-Based 3D Path Planning Algorithm for UAVs in Dynamic Dense Obstacle Environments

Abstract

Online three-dimensional (3D) path planning in dynamic environments is a fundamental problem for achieving autonomous navigation of unmanned aerial vehicles (UAVs). However, existing methods struggle to model traversable dynamic gaps, resulting in conservative and suboptimal trajectories. To address these challenges, this paper proposes a hierarchical reinforcement learning (RL) framework that integrates global path guidance, local trajectory generation, predictive safety evaluation, and neural network-based decision-making. Specifically, the global planner provides long-term navigation guidance, and the local module then utilizes an improved 3D dynamic window approach (DWA) to generate dynamically feasible candidate trajectories. To enhance safety in dense dynamic scenarios, the algorithm introduces a predictive axis-aligned bounding box (AABB) strategy to model the future occupancy of obstacles, combined with convex hull verification for efficient trajectory safety assessment. Furthermore, a double deep Q-network (DDQN) is employed with structured feature encoding, enabling the neural network to reliably select the optimal trajectory from the candidate set, thereby improving robustness and generalization. Comparative experiments conducted in a high-fidelity simulation environment show that the algorithm outperforms existing algorithms, reducing the average number of collisions to 0.2 while shortening the average task completion time by approximately 15%, and achieving a success rate of 97%.

Keywords

unmanned aerial vehicle (UAV) three-dimensional (3D) path planning / 3D dynamic window approach (DWA) / predictive axis-aligned bounding box (AABB) / double deep Q-network (DDQN) / autonomous navigation