A novel spatio-temporal trajectory data-driven development approach for autonomous vehicles

Abstract

Nowadays, autonomous driving has been attracted widespread attention from academia and industry. As we all know, deep learning is effective and essential for the development of AI components of Autonomous Vehicles (AVs). However, it is challenging to adopt multi-source heterogenous data in deep learning. Therefore, we propose a novel data-driven approach for the delivery of high-quality Spatio-Temporal Trajectory Data (STTD) to AVs, which can be deployed to assist the development of AI components with deep learning. The novelty of our work is that the meta-model of STTD is constructed based on the domain knowledge of autonomous driving. Our approach, including collection, preprocessing, storage and modeling of STTD as well as the training of AI components, helps to process and utilize huge amount of STTD efficiently. To further demonstrate the usability of our approach, a case study of vehicle behavior prediction using Long Short-Term Memory (LSTM) networks is discussed. Experimental results show that our approach facilitates the training process of AI components with the STTD.

Graphical abstract

Keywords

spatio-temporal trajectory data / data meta-modeling / domain knowledge / LSTM / vehicle behavior prediction / AI component

References

[1] Bai X, Xu C, Ao Y, Chen B, Du D (2019). Learning-based Probabilistic Modeling and Verifying Driver Behavior using MDP. In: 2019 International Symposium on Theoretical Aspects of Software Engineering (TASE). IEEE, 152–159

[2] Czarnecki K (2018). Automated driving system (ads) task analysis—part 2: structured road maneuvers. Waterloo Intelligent Systems Engineering Lab (WISE) Report, University of Waterloo

[3] Emmert-Streib F, Yang Z, Feng H, Tripathi S, Dehmer M , (2020)). An introductory review of deep learning for prediction models with big data. Front Artificial Intell, 3: 4–88

[4] de Vries G, van Someren M (2010). Clustering vessel trajectories with alignment kernels under trajectory compression. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Berlin: Springer. 2010: 296–311

[5] Dosovitskiy A, Ros G, Codevilla F, Lopez A, Koltun V (2017). CARLA: an open urban driving simulator. In: Conference on robot learning. PMLR, 2017: 1–16

[6] Fremont D J, Kim E, Dreossi T, Ghosh S, Yue X, Sangiovanni-Vincentelli A L, Seshia S A (2019). Scenic: a language for scenario specification and scene generation. In: Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, 63–78

[7] Fung S F, Zheng A (2018). Effects of data binning techniques on results of analyzing solar wind and geomagnetic indices data. In: AGU Fall Meeting Abstracts, 2018: SM31D–3525

[8] Li J D, Cheng K W, Wang S H, Morstatter F, Trevino R P, Tang J L, Liu H (2017). Feature selection: a data perspective. ACM Comput Surv 50(6): 94

[9] Meratnia N, de By R A (2004). Spatiotemporal compression techniques for moving point objects. In: Bertino E, Christodoulakis S, Plexousakis, Chriltophides V, Koubarakis M, Böhm K, Ferrari E, eds. Advances in Database Technology-EDBT 2004. Conference Proceedings EDBT 2004. Berlin: Springer

[10] Pannekoucke O, Cébron P, Oger N, Arbogast A (2016). From the Kalman filter to the particle filter: a geometrical perspective of the curse of dimensionality. Adv Meteor, 2016: 9372786

[11] Paardekooper J P, Comi M, Grappiolo C, Snijders R, van Vught W, Beekelaar R (2021). A hybrid-AI approach for competence assessment of automated driving functions. In: CEUR Workshop Proceedings. CEUR-WS, 2808(2808)

[12] Park S H, Kim B D, Kang C M, Chung C C, Choi J W (2018). Sequence-to-sequence prediction of vehicle trajectory via LSTM encoder-decoder architecture. In: 2018 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2018: 1672–1678

[13] Seshia S A, Desai A, Dreossi T, Fremont D J, Ghosh S, Kim E, Shivakumar S, Vazquez-Chanlatte M, Yue X Y (2018). Formal specification for deep neural networks. In: International Symposium on Automated Technology for Verification and Analysis. Cham: Springer, 2018: 20–34

[14] Wang Y (2020). Modeling, simulation and verification of autonomous driving scenario based on model-driven. Dissertation for the Master’s Degree. Shanghai: East China Normal University (in Chinese)

[15] Xu K, Xiao X, Miao J, Luo Q (2020). Data driven prediction architecture for autonomous driving and its application on apollo platform. In: 2020 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2020: 175–181

[16] Zhang L, Zhang C X, Du D H, Liu B, Tian B, Yuan Q (2019). Spatial information spatial metadata construction method. Shanghai: CN110532340A, 2019–12–03

[17] Zhang M H, Du D H, Zhang M Z, Zhang L, Wang Y, Zhou W T (2021). Spatio-temporal trajectory data-driven autonomous driving scenario meta-modeling approach. J Softw, 32(4): 973–987 (in Chinese)

[18] Zhang Q, Wang Y, Zhang X, Liu L, Wu X, Shi W (2018). OpenVDAP: an open vehicular data analytics platform for CAVs. In: 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS). IEEE, 2018: 1310–1320

[19] Zhao L, Ichise R, Mita S, Sasaki Y (2015a). Core ontologies for safe autonomous driving. In: International Semantic Web Conference

[20] Zhao L, Ichise R, Yoshikawa T, Naito T, Kakinami T, Sasaki Y (2015b). Ontology-based decision making on uncontrolled intersections and narrow roads. In: 2015 IEEE intelligent vehicles symposium (IV). IEEE, 2015: 83–88

Acknowledgments

Financial supports for this work, provided by the National Key Research and Development Program (No. 2018YFE0101000), the National Natural Science Foundation of China (Grant No. 61972153), the Key projects of the Ministry of Science and Technology (No. 2020AAA0107800), are gratefully acknowledged.

RIGHTS & PERMISSIONS

2021 Higher Education Press