A Survey on XR-Based Drone Simulation: Technologies, Applications, and Future Directions

Abstract

This paper presents a comprehensive survey of Extended Reality (XR)-based drone simulation systems, encompassing their architectures, simulation engines, physics modeling, and diverse training applications. With a particular focus on manual multirotor drone operations, this study highlights how Virtual Reality (VR) and Augmented Reality (AR) are increasingly vital for pilot training and mission rehearsal. We classify these simulators based on their hardware interfaces, spatial computing capabilities, and the integration of game and physics engines. We analyze specific platforms such as Flightmare, AirSim, DroneSim, Inzpire Mixed Reality UAV Simulator, and SimFlight XR are analyzed to illustrate various design strategies, ranging from research-grade modular frameworks to commercial training tools. In this paper, we also examine the implementation of spatial mapping and weather modeling to enhance realism in AR-based simulators. Finally, we identify critical challengesthat remain to be addressed, including user immersion, regulatory alignment, and achieving high levels of physical realism, and propose future directions in which XR-integrated drone training systems can advance.

Keywords

Virtual reality / Augmented reality / Drone simulator / UAV training / HMD / Human factors / Immersive training / Simulation engine

Acknowledgments

The authors would like to thank Makoto Itoh (University of Tsukuba, ReAMo Project) for his valuable advice on references regarding the efficacy of drone simulators, Yuto Ikeda for insightful discussions on drone simulators in general, Wataru Ken Tanaka for valuable discussions on simulator development and market perspectives, and Tuong Cao and Hoan Van for providing practical implementation details and other technical contributions.

Ethics Statement

Not applicable. This article is a survey of existing literature on XR-based drone simulation and does not involve studies with human participants or animals.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Funding

This research received no external funding.

Declaration of Competing Interest

The author, Kenji Tanaka, is the CEO of Foxtrot Inc., a company active in the development and deployment of drone simulation systems. This relationship may be perceived as a potential conflict of interest.

References

[1] Somerville A, Lynar T, Joiner KF, Wild G. Use of Simulation for Pre-Training of Drone Pilots. Drones 2024, 8, 640. doi:10.3390/drones8110640.

[2] Ross G, Gilbey A. Extended Reality (XR) Flight Simulators as an Adjunct to Traditional Flight Training Methods: A Scoping Review. CEAS Aeronaut. J. 2023, 14, 799-815. doi:10.1007/s13272-023-00688-5.

[3] Albeaino G, Eiris R, Gheisari M, Issa RR. DroneSim: A VR-based Flight Training Simulator for Drone-Mediated Building Inspections. Constr. Innov. 2022, 22, 831-848. doi:10.1108/CI-03-2021-0049.

[4] Mazlan MSM, Moketar NA, Kamalrudin M, Yusop N, Abdul Aziz A, Mat Zain NH, et al. The usability evaluation of the ARDroneSim application for augmented reality-based drone flight training simulator. J. Electr. Syst. 2024, 20, 5825-5838.

[5] Nasir ANM, Arsat M, Noordin MK, Muhamad Sidek MA, Tarmidi MZ. Structured Teaching Using Drone Simulators for Students’ Confidence in Real Flight. In Methods and Applications for Modeling and Simulation of Complex Systems: Proc. AsiaSim 2023; Springer: Singapore, 2024; pp. 125-136. doi:10.1007/978-981-99-7240-1_10.

[6] Storz V.Why XR Will be the Future of Pilot Training? LinkedIn Commentary. 2025. Available online: https://www.linkedin.com/pulse/why-xr-future-pilot-training-valentin-storz-lsoyf/ (accessed on 20 September 2025).

[7] Shephard Media. I/Itsec 2023:How VR and AR are reshaping the training capabilities for the Pentagon. News Article. 2023. Available online: https://shephardmedia.com (accessed on 20 September 2025).

[8] Milgram P, Takemura H, Utsumi A, Kishino F. Augmented reality: A class of displays on the reality-virtuality continuum. In Proceedings of Telemanipulator and Telepresence Technologies; SPIE: Bellingham, WA, USA, 1994; Volume 2351, pp. 282-292. doi:10.1117/12.197321.

[9] Kim D-H, Go Y-G, Choi S-M. An aerial mixed-reality environment for first-person-view drone flying. Appl. Sci. 2020, 10, 5436. doi:10.3390/app10165436.

[10] Foxtrot Inc. Product Page. SimFlight XR—Drone Flight Training with Head-Mounted Display. Product Page. 2025. Available online: https://foxtrot.biz/xr-drone-en/ (accessed on 20 July 2025).

[11] Song Y, Naji S, Kaufmann E, Loquercio A, Scaramuzza D. Flightmare: A flexible quadrotor simulator. In Proceedings of the Conference on Robot Learning (CoRL), virtual, 16-18 November 2021; pp. 1147-1157. doi:10.5555/3495724.3495738.

[12] Shah S, Dey D, Lovett C, Kapoor A. AirSim:High-Fidelity Visual and Physical Simulation for Autonomous Vehicles. In Field and Service Robotics; Springer Proceedings in Advanced Robotics; Hutter M, Siegwart R,Eds.; Springer: Cham, Switzerland, 2018; Volume 5; pp. 621-635. doi:10.1007/978-3-319-67361-5_40.

[13] Real Drone Simulator. Real Drone Simulator—Professional Drone Flight Simulation Software. 2025. Available online: https://www.realdronesimulator.com (accessed on 21 June 2025).

[14] Inzpire Ltd. Operating Drones in Mixed Reality: How Inzpire Leverages Varjo for Next-Gen UAV Training. Varjo Case Study, n.d. Available online: https://varjo.com/case-studies/operating-drones-in-mixed-reality-how-inzpire-leverages-varjo- for-next-gen-uav-training/ (accessed on 20 September 2025).

[15] Mairaj A, Baba AI, Javaid AY. Application specific drone simulators: Recent advances and challenges. Simul. Model. Pract. Theory 2019, 94, 100-117. doi:10.1016/j.simpat.2019.01.004.

[16] DroneShield; Operator (XRG). DroneShield and Operator Team Up for Extended Reality Counter-UAS Training. C-UAS Hub Article, 26 November 2022. Available online: https://cuashub.com/en/content/droneshield-and-operator-team-up-for- extended-reality-counter-uas-training/ (accessed on 20 September 2025).

[17] Ribeiro R, Ramos J, Safadinho D, Reis A, Rabadão C, Barroso J, et al. Web ar solution for uav pilot training and usability testing. Sensors 2021, 21, 1456. doi:10.3390/s21041456.

[18] Tsang OW, Peisen X, Garcia J, Jouffrais C. Seeing Safety: How Augmented Reality Could Transform Drone Inspections Forever. NUS Computing Feature, 4 June 2025. Available online: https://www.comp.nus.edu.sg/features/augmented-reality-transform-drone-inspections/ (accessed on 20 June 2025).

[19] Xu P, Garcia J, Ooi WT, Jouffrais C. SafeSpect: Safety-first augmented reality heads-up display for drone inspections. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’25), Yokohama, Japan, 26 April-1 May 2025; pp. 1-15. doi:10.1145/3613904.3642899.

[20] Burri M, Nikolic J, Gohl P, Schneider T, Rehder J, Omari S, et al. The EuRoC micro aerial vehicle datasets. Int. J. Robot. Res. 2016, 35, 1157-1163. doi:10.1177/0278364915620033.

[21] Hassan F, Sunar N, Basri MA, Mahmud MS, Ishak MH, Ali MS. Modeling and simulation of complex systems. In Communications in Computer and Information Science; Springer Nature: Singapore, 2023; pp. 492-503. doi:10.1007/978-981-99-6995-1_38.

[22] Cardona-Reyes H, Parra-Gonzalez E, Trujillo-Espinoza C, Villalba-Condori K. Task design in virtual reality environments for drone pilot training. In New Perspectives in Software Engineering; Springer: Cham, Switzerland, 2024; pp. 261-274. doi:10.1007/978-3-031-62739-7_19.

[23] EdgyBees Ltd. Drone Prix AR—The first augmented reality game for DJI drones. DJI Developers Blog. 2017. Available online: https://developers.dji.com/newsroom/news/drone-prix-ar/ (accessed on 20 September 2025).

[24] Space Inc. Drone Simulator VR. Scheduled for release on Meta Quest Store, June 2025. Available online: https://www.meta. com/quest/store/products/drone-simulator-vr/ (accessed on 20 September 2025).

[25] Bevins A, Duncan B. Aerial Flight Paths for Communication: How Participants Perceive and Intend to Respond to Drone Movements. In Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’21), Boulder, CO, USA, 8-11 March 2021; ACM: New York, NY, USA, 2021; pp. 16-23. doi:10.1145/3434073.3444652.

[26] RC Pilot Trainer. RC Pilot Trainer on Meta Quest. 2025. Available online: https://www.meta.com/experiences/rc-pilot-trainer/8142055429181651/ (accessed on 21 June 2025).

[27] Rowland D. PicaSim—Free flight simulator for R/C planes. 2020. Available online: https://www.rowlhouse.co.uk/PicaSim/ (accessed on 21 June 2025).

[28] Aerosense Inc. Aerobo Wing: Fixed-Wing Autonomous Drone (AS-VT01). 2025. Available online: https://aerosense.co.jp/ en/products/drone/as-vt01.html (accessed on 21 June 2025).

[29] Bat Cave Games Ltd. VelociDrone FPV Simulator. Available online: https://www.velocidrone.com/ (accessed on 21 June 2025).

[30] LuGus Studios. Liftoff: FPV Drone Racing Simulator. 2023. Available online: https://www.lugus-studios.be/portfolio/liftoff- drone-racing (accessed on 21 June 2025).

[31] Ruiz-Medina JS, Maeng S, Tu N, Itoh M. Evaluating drone operator motor skills: A comparison between simulated and real environments. In Proceedings of the 23rd International Symposium on Aviation Psychology (ISAP); Corvallis, OR, USA, 27-30 May 2025.

[32] Maeng S, Tu N, Nishida H, Itoh M. Analysis of effect of geographic information on decision-making under emergency conditions during BVLOS drone operation. Tech. J. Adv. Mobil. 2024, 5, 89-108. doi:10.34590/tjam.5.1089.

[33] Kaplan AD, Cruit J, Endsley M, Beers SM, Sawyer BD, Hancock PA. The effects of virtual reality, augmented reality, and mixed reality as training enhancement methods: A meta-analysis. Hum. Factors 2021, 63, 706-726. doi:10.1177/0018720820904229.

[34] Grieves M, Vickers J. Digital Twin:Mitigating Unpredictable, Undesirable Emergent Behavior in Complex Systems. In Transdisciplinary Perspectives on Complex Systems; Kahlen F-J, Flumerfelt S, Alves A,Eds.; Springer: Cham, Switzerland, 2017; pp. 85-113. doi:10.1007/978-3-319-38756-74.

[35] Costa C, Gomes E, Rodrigues N, Gonçalves A, Ribeiro R, Costa P, et al. Augmented reality mobile digital twin for unmanned aerial vehicle wildfire prevention. Virtual Real. 2025, 29, 71. doi:10.1007/s10055-025-01145-w.

[36] Hossen MS, Gurses A, Sichitiu M, Guvenc I. Accelerating development in UAV network digital twins with a flexible simulation framework. arXiv 2025, arXiv:2503.07935.

[37] U.S. Flight Simulation Training Device (FSTD) with Extended Reality (XR) Systems—Guidance. 2023. Available online: https://ushst.org/flight-simulation-training-device-fstd-with-extended-reality-xr-systems/ (accessed on 20 September 2025).

[38] Tanaka K, Tambe P. How Will VR and AR Impact Occupations? SSRN Electron. J. 2022. doi:10.2139/ssrn.4022827.

[39] Rizvi SAQ, Rehman U, Cao S, Moncion B. Exploring technology acceptance of flight simulation training devices and augmented reality in general aviation pilot training. Sci. Rep. 2025, 15, 2302. doi:10.1038/s41598-025-85448-7.

[40] Borowik G, Kożdoń-Dębecka M, Strzelecki S. Mutable Observation Used by Television Drone Pilots: Efficiency of Aerial Filming Regarding the Quality of Completed Shots. Electronics 2022, 11, 3881. doi:10.3390/electronics11233881.

[41] Rahmani H, Weckman GR. Working under the Shadow of Drones: Investigating Occupational Safety Hazards among Commercial Drone Pilots. IISE Trans. Occup. Ergon. Hum. Factors 2024, 12, 55-67. doi:10.1080/24725838.2023.2251009.

[42] Emami Y, Li K, Almeida L, Zou S, Ni W. On the Use of Immersive Digital Technologies for Designing and Operating UAVs. arXiv 2024, arXiv:2407.16288.

[43] Brooks C. Startup DronOSS Sets Its Sights on Fewer Drone Crashes with Augmented Reality Pilot Training. Next Reality, 20 September 2019. Available online: https://next.reality.news/news/startup-dronoss-sets-its-sights-fewer-drone-crashes-with-augmented-reality-pilot-training-0206369/ (accessed on 21 June 2025).

[44] Heliguy. AirHUD: Augmented Reality Drone Software (AR + VR). Heliguy Blog, 8 August 2023. Available online: https://www.heliguy.com/blogs/posts/heliguy-offers-airhud-augmented-reality-drone-software/ (accessed on 20 September 2025).

[45] Buchner J, Buntins K, Kerres M. The impact of augmented reality on cognitive load and performance: A systematic review. J. Comput. Assist. Learn. 2022, 38, 285-303. doi:10.1111/jcal.12617.