Local path planning method of the self-propelled model based on reinforcement learning in complex conditions

Abstract

Conducting hydrodynamic and physical motion simulation tests using a large-scale self-propelled model under actual wave conditions is an important means for researching environmental adaptability of ships. During the navigation test of the self-propelled model, the complex environment including various port facilities, navigation facilities, and the ships nearby must be considered carefully, because in this dense environment the impact of sea waves and winds on the model is particularly significant. In order to improve the security of the self-propelled model, this paper introduces the Q learning based on reinforcement learning combined with chaotic ideas for the model’s collision avoidance, in order to improve the reliability of the local path planning. Simulation and sea test results show that this algorithm is a better solution for collision avoidance of the self navigation model under the interference of sea winds and waves with good adaptability.

Keywords

self-propelled model / local path planning / Q learning / obstacle avoidance / reinforcement learning

References

[1] Cao W, Xu L, Wu M. A double-layer decision-making model based on fuzzy Q-learning for robot soccer. CAAI Transactions on Intelligent Systems, 2008, 3(3): 234-238

[2] Chou C, Lian F. Characterizing indoor environment for robot navigation using velocity space approach with region analysis and look-ahead verification. IEEE Transactions on Instrumentation and Measurement, 2011, l(60): 442-451

[3] Karima R, Ouahiba A. BI-steerable robot navigation using a modified dynamic window approach. Proceeding of the 6th International Symposium on Mechatronics and its Applications, Sharjah, UAE, 2009, 1-6

[4] Larson J, Bruch M, Ebken J. Autonomous navigation and obstacle avoidance for unmanned surface vehicles. Proc. SPIE Unmanned Systems Technology VIII, Orlando, USA, 2006, 17-29

[5] Larson J, Bruch M, Halterman R, Rogers J, Webster R. Advances in autonomous obstacle avoidance for unmanned surface vehicles. AUVSI Unmanned Systems North America 2007, Washington, DC, USA, 2007, 6-9

[6] Manley JE. Unmanned surface vehicles, 15 years of development. Oceans, 2008, 1(4): 15-18

[7] Ogren P, Leonard NE. A convergent dynamic window approach to obstacle avoidance. IEEE Transaction on Robotics, 2005, 21(2): 188-195

[8] Pingpeng T, Rubo Z, Deli L. Research on near-field obstacle avoidance for unmanned surface vehicle based on heading window. Conference of the 24th Control and Decision Conference (CCDC), 2012, 1262-167

[9] Seder M, Petrovic I. Dynamic window based approach to mobile robot motion control in the presence of moving obstacles. IEEE International Conference on Robotics and Automation, 2007, 1986-1991

[10] Simmons R, Henriksen L, Chrisman L, Whelan G. Obstacle avoidance and safeguarding for a lunar rover. AIAA Forum on Advanced Developments in Space robotics, Madison, WI, USA, 1996, 267-270

[11] Sun S, Li J, Zhao X. Experimental research on large scale model test in real ocean wave environment. Journal of Harbin Engineering University, 2009, 30(5): 475-480

[12] Sun Y. Chaos identification based on CMAC with replacing eligibility learning. Journal of Chongqing University of Post and Telecommunications, 2009, 2: 23-26

[13] Sun Y, Gao J, Zhang C, Deng F. Chaotic genetic algorithm with feedback and its applications to constrained optimization. Journal of South China University of Technology, 2007, 35(1): 19-23

[14] Tang P, Qiao L, Zhang R. Near-field reactive obstacle-avoidance for USV. Journal of Huazhong University of Science & Technology, 2011, 39(Sup.II): 400-406

[15] Wang M, Zhang R. Research on fuzzy ND obstacle avoidance method of unmanned surface vessel. Computer Engineering, 2012, 38(21): 164-167

[16] Xu L, Chen Y, Ju H. autonomous obstacle avoidance for mobile robot based on dynamic behavior control. Computer Engineering, 2007, 33(14): 180-182