Decision making framework for autonomous vehicles driving behavior in complex scenarios via hierarchical state machine

Abstract

With the development of autonomous car, a vehicle is capable to sense its environment more precisely. That allows improved drving behavior decision strategy to be used for more safety and effectiveness in complex scenarios. In this paper, a decision making framework based on hierarchical state machine is proposed with a top-down structure of three-layer finite state machine decision system. The upper layer classifies the driving scenario based on relative position of the vehicle and its surrounding vehicles. The middle layer judges the optimal driving behavior according to the improved energy efficiency function targeted at multiple criteria including driving efficiency, safety and the grid-based lane vacancy rate. The lower layer constructs the state transition matrix combined with the calculation results of the previous layer to predict the optimal pass way in the region. The simulation results show that the proposed driving strategy can integrate multiple criteria to evaluate the energy efficiency value of vehicle behavior in real time, and realize the selection of optimal vehicle driving strategy. With popularity of automatic vehicles in future, the driving strategy can be used as a reference to provide assistance for human drive or even the real-time decision-making of autonomous driving.

Keywords

Decision making / Autonomous Car / Energy efficiency function / State machine

References

[1] OhSI, KangHB. Fast occupancy grid filtering using grid cell clusters from LIDAR and stereo vision sensor data. IEEE Sensors J., 2016, 16(19):7258-7266

[2] SivaramanS, TrivediMM. Dynamic probabilistic drivability maps for lane change and merge driver assistance. IEEE Trans. Intell. Transp. Syst., 2014, 15(5):2063-2073

[3] G. Yang, P. Wang, W. Han, et al. Automatic generation of fine-grained traffic load spectrum via fusion of weigh-in-motion and vehicle spatial–temporal information. Computer-Aided Civil and Infrastructure Engineering.

[4] W. Yuan, P. Wang, J. Yang, Y. Meng, An alternative reliability method to evaluate the regional traffic congestion from GPS data obtained from floating cars. IET Smart Cities. 3(2), 79–90 (2021)

[5] WangP, ZhangY, WangS, LiL, LiX. Forecasting travel speed in the rainfall days to develop suitable variable speed limits control strategy for less driving risk. J. Adv. Transp., 2021, 2021: Article ID 6639559 13 pages

[6] BrechtelS, GindeleT, DillmannR. Solving continuous POMDPs: Value iteration with incremental learning of an efficient space representation. Proceedings of the 30th international conference on machine learning, 2013 370-378

[7] WeiJ, DolanJM, SniderJM, LitkouhiB. A point-based MDP for robust single-lane autonomous driving behavior under uncertainties. Robotics and Automation (ICRA), 2011 IEEE international conference on, IEEE, 2011 2586-2592

[8] BrechtelS, GindeleT, DillmannR. Probabilistic decision-making under uncertainty for autonomous driving using continuous POMDPs. Intelligent Transportation Systems (ITSC), 2014 IEEE 17th international conference on, IEEE, 2014 392-399

[9] BrechtelS, GindeleT. Solving continuous POMDPs: Value iteration with incremental learning of an efficient space representation. Proceedings of the 30th International conference on machine learning, 2013 370-378

[10] BojarskiM, TestaDD, DworakowskiD. End to end learning for self -driving cars. arXiv: Computer Vision and Pattern Recognition, 2016

[11] CodevillaF, MüllerM, LópezA, KoltunV, DosovitskiyA. End-to-end driving via conditional imitation learning, 2018

[12] P. Wang, W. Hao, Y. Jin, in IEEE transactions on intelligent transportation systems. Fine-grained traffic flow prediction of various vehicle types via fusison of multisource data and deep learning approaches. https://doi.org/10.1109/TITS.2020.2997412

[13] UlbrichS, MaurerM. Probabilistic online POMDP decision making for lane changes in fully automated driving. Intelligent Transportation Systems-(ITSC), 2013 16th international IEEE conference on, IEEE, 2013 2063-2067

[14] VanholmeB, GruyerD, LusettiB, GlaserS, MammarS. Highly automated driving on highways based on legal safety. Intell. Transportation Syst. IEEE Trans., 2013, 14(1):333-347

[15] FletcherL, TellerS, OlsonE, MooreD, KuwataY, HowJ, et al.. The MIT–Cornell collision and why it happened. J. Field Robot., 2008, 25(10):775-807

[16] J. Yang, P. Wang, W. Yuan, et al., Automatic generation of optimal road trajectory for the rescue vehicle in case of emergency on mountain freeway using reinforcement learning approach. IET Intell. Transp. Syst. 15, 1142–1152 (2021)

[17] ZhaoL, IchiseR, SasakiY, ZhengL, YoshikawaT. Fast decision making using ontology-based knowledge base. Intelligent vehicles symposium, IEEE, 2016 173-178

[18] MontemerloM, BeckerJ, BhatS Jr. The Stanford entry in the urban challenge. J. Field Robot., 2008, 25(9):569-597

[19] ZieglerJ, BenderP, SchreiberM. Making bertha drive—An autonomous journey on a historic route. Intell. Transportation Syst. Mag. IEEE, 2014, 6(2):8-20

[20] GindeleT, JagszentD, PitzerB, DillmannR. Design of the planner of Team AnnieWAY’s autonomous vehicle used in the DARPA Urban Challenge 2007. Intelligent vehicles symposium, IEEE, 2008 1131-1136

[21] KurtA, OzgunerU. Hierarchical finite state machines for autonomous mobile systems. Control. Eng. Pract., 2013, 21(2):184-194

[22] MaS, GongG, HanL, SongX. Military task programming based on finite state machine (FSM) decision-making model. 2008 Asia simulation conference-7th international conference on system simulation and scientific computing, 2008 1416-1420

[23] DoQH, TehraniH, MitaS, EgawaM, MutoK, OnedaKY. Human drivers based active-passive model for automated lane change. IEEE Intell. Transp. Syst. Mag., 2017, 9(1):42-56

[24] JinL, BartV, YangS, et al.. Safety lane change model of vehicle assistant driving on highway. J. Jilin Univ. Eng. Technol., 2009, 39(3):582-586

[25] S. Wu, Research on the risk assessment algorithm for accounting information system based on analytic hierarchy process, Seventh International Conference on Measuring Technology and Mechatronics Automation (2015) pp. 934-937. https://doi.org/10.1109/ICMTMA.2015.346.

Funding

The National Key Research and Development Program of China(2020YFB1600400); Key Research and Development Program of Shaanxi Province(2020GY-020); the Fundamental Research Funds for the Central Universities, CHD(300102320305)