Approach for improved development of advanced driver assistance systems for future smart mobility concepts

Abstract

To use the benefits of Advanced Driver Assistance Systems (ADAS)-Tests in simulation and reality a new approach for using Augmented Reality (AR) in an automotive vehicle for testing ADAS is presented in this paper. Our procedure provides a link between simulation and reality and should enable a faster development process for future increasingly complex ADAS tests and future mobility solutions. Test fields for ADAS offer a small number of orientation points. Furthermore, these must be detected and processed at high vehicle speeds. That requires high computational power both for developing our method and its subsequent use in testing. Using image segmentation (IS), artificial intelligence (AI) for object recognition, and visual simultaneous localization and mapping (vSLAM), we aim to create a three-dimensional model with accurate information about the test site. It is expected that using AI and IS will significantly improve performance as computational speed and accuracy for AR applications in automobiles.

Keywords

Augmented reality / Advanced driver assistance systems / Visual simultaneous localization and mapping / European new car assessment programme

References

[1] BenglerK., DietmayerK., FarbeB., et al.. Three decades of driver assistance systems: review and future perspectives. IEEE Intell. Transp. Syst. Mag., 2014, 6: 6-22

[2] BrooksR.R., IyengarS.S.. Multi-Sensor Fusion: Fundamentals and Applications with Software, 1998 Hoboken Prentice Hall https://search.library.wisc.edu/catalog/999904353602121

[3] CamposC., ElviraR., RodríguezJ.J.G., et al.. ORB-SLAM3: an accurate open-source library for visual, visual-inertial, and multimap SLAM. IEEE Trans. Robot., 2021, 37: 1-17

[4] ChatilaR., LaumondJ.. Position referencing and consistent world modeling for mobile robots. Proceedings. 1985 IEEE International Conference on Robotics and Automation, 1985 138-145

[5] ChatilaR., LaumondJ.-P.. Position referencing and consistent world modeling for mobile robots. 1985 IEEE International Conference on Proceedings Robotics and Automation, 1985 138-145

[6] DarmsM.. Eine Basis-Systemarchitektur zur Sensordatenfusion von Umfeldsensoren fur Fahreras-sistenzsysteme, 2007 Dusseldorf TU Darmstadt de. Zugl.: Darmstadt, Techn. Univ., Diss., 2007. PhD thesis. http://tubiblio.ulb.tu-darmstadt.de/35385/

[7] M. Darms, Eine basis-systemarchitektur zur sensordatenfusion von umfeldsensoren fuer fahrerassistenzsysteme. PhD thesis, Technische Universitat (2007)

[8] DarmsM.. WinnerH., HakuliS., WiesbadenG.W.. Fusion umfelderfassender Sensoren. Handbuch Fahrerassistenzsysteme: Grundlagen, Komponenten und Systeme für aktive Sicherheit und Komfort, 2012 Wiesbaden Vieweg+Teubner 237-248

[9] EngelJ.J., SturmJ., CremersD.. Camera-based navigation of a low-cost quadrocopter. 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012 2815-2821

[10] FredrikssonR., LennéM.G., van MontfortS., et al.. European NCAP program developments to address driver distraction, drowsiness and sudden sickness. Front. Neuroergon., 2021, 2 https://www.frontiersin.org/article/10.3389/fnrgo.2021.786674

[11] GasserT.M., SeeckA., Walker SmithB., et al.. WinnerH., et al.. Rahmenbedingungen fur die Fahrerassistenzentwicklung. Handbuch Fahrerassistenzsysteme: Grundlagen, Komponenten und Systeme für aktive Sicherheit und Komfort, 2015 Wiesbaden Springer 27-54

[12] HakuliS., KrugM., et al.. WinnerH., et al.. Virtuelle integration. Handbuch Fahrerassistenzsysteme, Grundlagen, Komponenten und Systeme fur aktive Sicherheit und Komfort, 2015 Wiesbaden Springer 125-138

[13] KimB.-J., LeeS.-B.. A study on the evaluation method of autonomous emergency vehicle braking for pedestrians test using monocular cameras. Appl. Sci., 2020, 10 https://www.mdpi.com/2076-3417/10/13/4683

[14] MiquetC.. New test method for reproducible real-time tests of ADAS ECUs: “vehicle-in-the-loop” connects real-world vehicle with the virtual world. 5th International Munich Chassis Symposium 2014, 2014 Wiesbaden Springer 575-589

[15] Mur-ArtalR., MontielJ., TardosJ.. ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Trans. Robot., 2015, 31: 1147-1163

[16] NagaiM.. PfefferP.E.. Research into ADAS with autonomous driving intelligence for future innovation. 5th International Munich Chassis Symposium 2014, 2014 Wiesbaden Springer 779-793

[17] NollM., RappsP.. WinnerH., HakuliS., WiesbadenG.W.. Ultraschallsensorik. Handbuch Fahrerassistenzsysteme: Grundlagen, Komponenten und Systeme für aktive Sicherheit und Komfort, 2009 Wiesbaden Vieweg+Teubner 110-122 ISBN 978-38348-9977-4

[18] On-Road Automated Driving (ORAD) Committee. Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles. Technical repport (SAE International). https://doi.org/10.4271/J3016_202104. https://www.sae.org/content/j3016_202104 (visited on 05/30/2022)

[19] A.E.B. Pedestrian—Euro NCAP, https://www.euroncap.com:443/en/vehicle-safety/the-ratings-explained/vulnerable-road-user-vru-protection/aeb-pedestrian/ (visited on 05/30/2022)

[20] PunkeM., MenzelS., WerthessenB., et al., et al.. WinnerH., et al., et al.. Automotive camera (hardware). Handbook of Driver Assistance Systems: Basic Information, Components and Systems for Active Safety and Comfort, 2016 Cham Springer 431-460 ISBN 978-3-319-123523

[21] RajT., HashimF.H., HuddinA.B., et al.. A survey on LiDAR scanning mechanisms. Electronics, 2020, 9 https://www.mdpi.eom/2079-9292/9/5/741

[22] F. Schuldt, F. Saust, B. Lichte et al., Effiziente systematische Testgenerierung fiir Fahrerassistenzsysteme in virtuellen Umgebungen (2013). https://doi.org/10.24355/dbbs.084-201307101421-0. https://publikationsserver.tu-braunschweig.de/receive/dbbs_mods_00052570

[23] SeinigerP., WeitzelA., et al.. WinnerH., et al.. Testverfahren für Verbraucherschutz und Gesetzgebung. Handbuch Fahrerassistenzsysteme, Grundlagen, Komponenten und Systeme für aktive Sicherheit und Komfort, 2015 Wiesbaden Springer 167-182 ISBN 978-3658-05734-3

[24] SieberM., BergG., KarlI., et al.. Validation of driving behavior in the vehicle in the loop: steering responses in critical situations. 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013), 2013 1101-1106

[25] StelletJ.E., ZofkaM.R., SchumacherJ., et al.. Testing of advanced driver assistance towards automated driving: a survey and taxonomy on existing approaches and open questions. 2015 IEEE 18th International Conference on Intelligent Transportation Systems, 2015 1455-1462

[26] TaketomiT., UchiyamaH., IkedaS.. Visual SLAM algorithms: a survey from 2010 to 2016. IPSJ Trans. Comput. Vis. Appl., 2017, 9

[27] WinnerH.. WinnerH., HakuliS., WolfG.. Radarsensorik. Handbuch Fahrerassistenzsysteme: Grundlagen, Komponenten und Systeme für aktive Sicherheit und Komfort, 2012 Wiesbaden Vieweg+Teubner 123-171

[28] WinnerH., HakuliS., LotzF., et al.. Handbuch Fahrerassistenzsysteme—Grundlagen, Komponenten und Systeme für aktive Sicherheit und Komfort, 2015 Berlin Springer ISBN 9783-658-05734-3

[29] YuC., LiuZ., LiuX., et al.. DS-SLAM: a semantic visual SLAM towards dynamic environments. CoRR, 2018 arXiv:1809.08379