Artificial Intelligence in Medical Metaverse: Applications, Challenges, and Future Prospects

Abstract

The medical metaverse is a combination of medicine, computer science, information technology and other cutting-edge technologies. It redefines the method of information interaction about doctor-patient communication, medical education and research through the integration of medical data, knowledge and services in a virtual environment. Artificial intelligence (AI) is a discipline that uses computer technology to study and develop human intelligence. AI has infiltrated every aspect of medical metaverse and is deeply integrated with the technologies that build medical metaverse, such as large language models (LLMs), digital twins, blockchain and extended reality (including VR/AR/XR). AI has become an integral part of the medical metaverse building process. Moreover, AI also provides richer medical metaverse functions, including diagnosis, education, and consulting. This paper aims to introduce how AI supports the development of medical metaverse, including its specific application scenarios, shortcomings and future development. Our goal is to contribute to the advancement of more sophisticated and intelligent medical methods.

References

[1] Liu PR, Lu L, Zhang JY, et al.. Application of Artificial Intelligence in Medicine: An Overview. Curr Med Sci, 2021, 41(6): 1105-1115

[2] Graham S, Depp C, Lee EE, et al.. Artificial Intelligence for Mental Health and Mental Illnesses: an Overview. Curr Psychiatry Rep, 2019, 21(11): 116

[3] Luong T, Lecuyer A, Martin N, et al.. A Survey on Affective and Cognitive VR. IEEE Trans Vis Comput Graph, 2022, 28(12): 5154-5171

[4] Xie Y, Zhang J, Wang H, et al.. Applications of Blockchain in the Medical Field: Narrative Review. J Med Internet Res, 2021, 23(10): e28613

[5] Palumbo A. Microsoft HoloLens 2 in Medical and Healthcare Context: State of the Art and Future Prospects. Sensors (Basel), 2022, 22(20): 7709

[6] Massetti M, Chiariello GA. The metaverse in medicine. Eur Heart J Suppl, 2023, 25(Suppl B): B104-B107

[7] Kaul V, Enslin S, Gross SA. History of artificial intelligence in medicine. Gastrointest Endosc, 2020, 92(4): 807-812

[8] Kufel J, Bielówka M, Rojek M, et al.. Classification of Chest X-ray Abnormalities Using Transfer Learning Techniques. J Pers Med, 2023, 13(10): 1426

[9] Kufel J, Bargiel-Lączek K, Kocot S, et al.. What Is Machine Learning, Artificial Neural Networks and Deep Learning?-Examples of Practical Applications in Medicine. Diagnostics (Basel), 2023, 13(15): 2582

[10] Hashimoto DA, Witkowski E, Gao L, et al.. Artificial Intelligence in Anesthesiology: Current Techniques, Clinical Applications, and Limitations. Anesthesiology, 2020, 132(2): 379-394

[11] Lv Z. Generative artificial intelligence in the metaverse era. Cognitive Robotics, 2023, 3: 208-217

[12] Tay Y, Dehghani M, Bahri D, et al.. Efficient Transformers: A Survey. ACM Computing Surveys, 2022, 55(6): 1-28

[13] Adesso G. Towards the ultimate brain: Exploring scientific discovery with ChatGPT AI. AI Magazine, 2023, 44(3): 328-342

[14] Leutz-Schmidt P, Grözinger M, Kauczor HU, et al.. Performance of ChatGPT on basic healthcare leadership and management questions. Health Technol, 2024, 4(6): 1161-1166

[15] Kasneci E, Sessler K, Küchemann S, et al.. ChatGPT for good? On opportunities and challenges of large language models for education. Learning Indiv Diff, 2023 103

[16] Thirunavukarasu AJ, Ting DSJ, Elangovan K, et al.. Large language models in medicine. Nat Med, 2023, 29(8): 1930-1940

[17] Waisberg E, Ong J, Masalkhi M, et al.. GPT-4: a new era of artificial intelligence in medicine. Ir J Med Sci, 2023, 192(6): 3197-3200

[18] Weisberg EM, Chu LC, Fishman EK. The first use of artificial intelligence (AI) in the ER: triage not diagnosis. Emerg Radiol, 2020, 27(4): 361-366

[19] Kaelin VC, Valizadeh M, Salgado, et al.. Artificial Intelligence in Rehabilitation Targeting the Participation of Children and Youth With Disabilities: Scoping Review. J Med Internet Res, 2021, 23(11): e25745

[20] Lee J, Yoon W, Kim S, et al.. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 2020, 36(4): 1234-1240

[21] Huang H, Zheng O, Wang D, et al.. ChatGPT for shaping the future of dentistry: the potential of multi-modal large language model. Int J Oral Sci, 2023, 15(1): 29

[22] Alberts IL, Mercolli L, Pyka T, et al.. Large language models (LLM) and ChatGPT: what will the impact on nuclear medicine be?. Eur J Nucl Med Mol Imaging, 2023, 50(6): 1549-1552

[23] Wang Y, Pan Y, Yan M, et al.. A Survey on ChatGPT: AI–Generated Contents, Challenges, and Solutions. IEEE Open J Comp Soc, 2023, 4: 280-302

[24] Tan S, Xin X, Wu D. ChatGPT in medicine: prospects and challenges: a review article. Int J Surg, 2024, 110(6): 3701-3706

[25] Mihai S, Yaqoob M, Hung DV, et al.. Digital Twins: A Survey on Enabling Technologies, Challenges, Trends and Future Prospects. IEEE Commun Surveys Tutorials, 2022, 24(4): 2255-2291

[26] Jiang Y, Yin S, Li K, et al.. Industrial applications of digital twins. Philos Trans A Math Phys Eng Sci, 2021, 379(2207): 20200360

[27] Schluse M, Priggemeyer M, Atorf L, et al.. Experimentable Digital Twins—Streamlining Simulation-Based Systems Engineering for Industry 4.0. IEEE Trans Indust Inform, 2018, 14(4): 1722-1731

[28] Huang Z, Shen Y, Li J, et al.. A Survey on AI-Driven Digital Twins in Industry 4.0: Smart Manufacturing and Advanced Robotics. Sensors (Basel), 2021, 21(19): 6340

[29] Xu X, Shen B, Ding S, et al.. Service Offloading With Deep Q-Network for Digital Twinning-Empowered Internet of Vehicles in Edge Computing. IEEE Trans Indust Inform, 2022, 18(2): 1414-1423

[30] Gazerani P. Intelligent Digital Twins for Personalized Migraine Care. J Pers Med, 2023, 13(8): 1255

[31] Cheng W, Lian W, Tian J. Building the hospital intelligent twins for all-scenario intelligence health care. Digit Health, 2022, 8: 20552076221107894

[32] Chu Y, Li S, Tang J, et al.. The potential of the Medical Digital Twin in diabetes management: a review. Front Med (Lausanne), 2023, 10: 1178912

[33] Fischer RP, Volpert A, Antonino P, et al.. Digital patient twins for personalized therapeutics and pharmaceutical manufacturing. Front Digit Health, 2023, 5: 1302338

[34] Andrews C, Southworth MK, Silva JNA, et al.. Extended Reality in Medical Practice. Curr Treat Options Cardiovasc Med, 2019, 21(4): 18

[35] Taghian A, Abo-Zahhad M, Sayed MS, et al.. Virtual and augmented reality in biomedical engineering. Biomed Eng Online, 2023, 22(1): 76

[36] Hu Z, Bulling A, Li S, et al.. FixationNet: Forecasting Eye Fixations in Task-Oriented Virtual Environments. IEEE Trans Vis Comput Graph, 2021, 27(5): 2681-2690

[37] Kataoka K, Yamamoto T, Otsuki M, et al.. A New Interactive Haptic Device for Getting Physical Contact Feeling of Virtual Objects. 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2019

[38] Wu P, Ding W, You Z, et al.. Virtual Reality Video Quality Assessment Based on 3d Convolutional Neural Networks. Paper presented at: 2019 IEEE International Conference on Image Processing (ICIP), 2019 22–25 Sep.

[39] Monrat AA, Schelen O, Andersson K. A Survey of Blockchain From the Perspectives of Applications, Challenges, and Opportunities. IEEE Access, 2019, 7: 117134-117151

[40] Price WN 2nd Cohen IG. Privacy in the age of medical big data. Nat Med, 2019, 25(1): 37-43

[41] Salah K, Rehman MHU, Nizamuddin N, et al.. Blockchain for AI: Review and Open Research Challenges. IEEE Access, 2019, 7: 10127-10149

[42] Dey S. Securing Majority-Attack in Blockchain Using Machine Learning and Algorithmic Game Theory: A Proof of Work. Paper presented at: 2018 10th Computer Science and Electronic Engineering (CEEC), 2018 19–21 Sep.

[43] Khan MA, Abbas S, Rehman A, et al.. A Machine Learning Approach for Blockchain-Based Smart Home Networks Security. IEEE Network, 2021, 35(3): 223-229

[44] Oktian YE, Lee SG. Blockchain-Based Federated Learning System: A Survey on Design Choices. Sensors (Basel), 2023, 23(12): 5658

[45] Lu Y, Huang X, Dai Y, et al.. Blockchain and Federated Learning for Privacy-Preserved Data Sharing in Industrial IoT. IEEE Trans Indust Inform, 2020, 16(6): 4177-4186

[46] Keskinbora K, Guven F. Artificial Intelligence and Ophthalmology. Turk J Ophthalmol, 2020, 50(1): 37-43

[47] Itchhaporia D. Artificial intelligence in cardiology. Trends Cardiovasc Med, 2022, 32(1): 34-41

[48] Currie G, Hawk KE, Rohren E, et al.. Machine Learning and Deep Learning in Medical Imaging: Intelligent Imaging. J Med Imaging Radiat Sci, 2019, 50(4): 477-487

[49] Vedantham S, Shazeeb MS, Chiang A, et al.. Artificial Intelligence in Breast X-Ray Imaging. Semin Ultrasound CT MR, 2023, 44(1): 2-7

[50] Pekçevik Y, Orbatu D, Güngör F, et al.. The effect of an artificial intelligence algorithm on chest X-ray interpretation of radiology residents. Br J Radiol, 2022, 95(1139): 20210688

[51] Liu P, Lu L, Chen Y, et al.. Artificial intelligence to detect the femoral intertrochanteric fracture: The arrival of the intelligent-medicine era. Front Bioeng Biotechnol, 2022, 10: 927926

[52] Trasolini R, Byrne MF. Artificial intelligence and deep learning for small bowel capsule endoscopy. Dig Endosc, 2021, 33(2): 290-297

[53] Tsai DJ, Lin C, Lin CS. Artificial Intelligence-enabled Chest X-ray Classifies Osteoporosis and Identifies Mortality Risk. J Med Syst, 2024, 48(1): 12

[54] Yasmin F, Shah SMI, Naeem A, et al.. Artificial intelligence in the diagnosis and detection of heart failure: the past, present, and future. Rev Cardiovasc Med, 2021, 22(4): 1095-1113

[55] Abramoff MD, Lavin PT, Birch M, et al.. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med, 2018, 1: 39

[56] Ometov A, Shubina V, Klus L, et al.. A Survey on Wearable Technology: History, State-of-the-Art and Current Challenges. Comput Netw, 2021, 193: 108074

[57] Nahavandi D, Alizadehsani R, Khosravi A, et al.. Application of artificial intelligence in wearable devices: Opportunities and challenges. Comput Methods Programs Biomed, 2022, 213: 106541

[58] Acharya UR, Oh SL, Hagiwara Y, et al.. A deep convolutional neural network model to classify heartbeats. Comput Biol Med, 2017, 89: 389-396

[59] Hossain MB, Posada-Quintero HF, Chon KH. A Deep Convolutional Autoencoder for Automatic Motion Artifact Removal in Electrodermal Activity Signals: A Preliminary Study. Annu Int Conf IEEE Eng Med Biol Soc, 2022, 2022: 325-328

[60] Ahlrichs C, Sama A, Lawo M, et al.. Detecting freezing of gait with a tri-axial accelerometer in Parkinson’s disease patients. Med Biol Eng Comput, 2016, 54(1): 223-233

[61] Sitaula C, Grooby E, Kwok TC, et al.. Artificial intelligence-driven wearable technologies for neonatal cardiorespiratory monitoring. Part 2: artificial intelligence. Pediatr Res, 2023, 93(2): 426-436

[62] Kevat A, Kalirajah A, Roseby R. Artificial intelligence accuracy in detecting pathological breath sounds in children using digital stethoscopes. Respir Res, 2020, 21(1): 253

[63] Bandyopadhyay A, Goldstein C. Clinical applications of artificial intelligence in sleep medicine: a sleep clinician’s perspective. Sleep Breath, 2023, 27(1): 39-55

[64] Gursoy D, Malodia S, Dhir A. The metaverse in the hospitality and tourism industry: An overview of current trends and future research directions. J Hospital Market Manag, 2022, 31(5): 527-534

[65] Gupta T, Bhutta M. Outcomes of remote versus face-to-face ear, nose and throat outpatient consultation on patient pathways. Ann R Coll Surg Engl, 2023, 105(6): 561-567

[66] Antel R, Abbasgholizadeh-Rahimi S, Guadagno E, et al.. The use of artificial intelligence and virtual reality in doctor-patient risk communication: A scoping review. Patient Educ Couns, 2022, 105(10): 3038-3050

[67] Veras M, Labbé DR, Furlano J, et al.. A framework for equitable virtual rehabilitation in the metaverse era: challenges and opportunities. Front Rehabil Sci, 2023, 4: 1241020

[68] Chen J, Or CK, Chen T. Effectiveness of using virtual reality–supported exercise therapy for upper extremity motor rehabilitation in patients with stroke: systematic review and meta-analysis of randomized controlled trials. J Med Internet Res, 2022, 24(6): e24111

[69] Hoffman HG, Boe DA, Rombokas E, et al.. Virtual reality hand therapy: A new tool for nonopioid analgesia for acute procedural pain, hand rehabilitation, and VR embodiment therapy for phantom limb pain. J Hand Ther, 2020, 33(2): 254-262

[70] Bai Y, Liu F, Zhang H. Artificial Intelligence Limb Rehabilitation System on Account of Virtual Reality Technology on LongTerm Health Management of Stroke Patients in the Context of the Internet. Comput Math Methods Med, 2022, 2022: 2688003

[71] Vourganas I, Stankovic V, Stankovic LJS. Individualised responsible artificial intelligence for home-based rehabilitation. Sensors (Basel), 2020, 21(1): 2

[72] Murakami Y, Honaga K, Kono H, et al.. New Artificial Intelligence-Integrated Electromyography-Driven Robot Hand for Upper Extremity Rehabilitation of Patients With Stroke: A Randomized, Controlled Trial. Neurorehabil Neural Repair, 2023, 37(5): 298-306

[73] Provido SMP, Abris GP, Hong S, et al.. Association of fried food intake with prehypertension and hypertension: the Filipino women’s diet and health study. Nutr Res Pract, 2020, 14(1): 76-84

[74] Wang Y, Li C, Qu L, et al.. Application and challenges of a metaverse in medicine. Front Robot AI, 2023, 10: 1291199

[75] Lopez-Martinez F, Nunez-Valdez ER, Crespo RG, et al.. An artificial neural network approach for predicting hypertension using NHANES data. Sci Rep, 2020, 10(1): 10620

[76] Tsoi K, Yiu K, Lee H, et al.. Applications of artificial intelligence for hypertension management. J Clin Hypertens (Greenwich), 2021, 23(3): 568-574

[77] Koshimizu H, Kojima R, Kario K, et al.. Prediction of blood pressure variability using deep neural networks. Int J Med Inform, 2020, 136: 104067

[78] Cambuli VM, Baroni MG. Intelligent Insulin vs. Artificial Intelligence for Type 1 Diabetes: Will the Real Winner Please Stand Up?. Int J Mol Sci, 2023, 24(17): 13139

[79] Bothe MK, Dickens L, Reichel K, et al.. The use of reinforcement learning algorithms to meet the challenges of an artificial pancreas. Expert Rev Med Devices, 2013, 10(5): 661-673

[80] Sandrone S. Medical education in the metaverse. Nat Med, 2022, 28(12): 2456-2457

[81] Zhang K, Gao Y, Lv J, et al.. Medicine MMi. Artificial intelligence-based spiral CT 3D reconstruction in transcatheter aortic valve implantation. Comput Math Methods Med, 2022, 2022: 5794681

[82] Plotzky C, Lindwedel U, Sorber M, et al.. Virtual reality simulations in nurse education: A systematic mapping review. Nurse Educ Today, 2021, 101: 104868

[83] Han ER, Yeo S, Kim MJ, et al.. Medical education trends for future physicians in the era of advanced technology and artificial intelligence: an integrative review. BMC Med Educ, 2019, 19(1): 460

[84] Duong MT, Rauschecker AM, Rudie JD, et al.. Artificial intelligence for precision education in radiology. Br J Radiol, 2019, 92(1103): 20190389

[85] Wang G, Badal A, Jia X, et al.. Development of metaverse for intelligent healthcare. Nat Mach Intell, 2022, 4(11): 922-929

[86] Elbadawi M, McCoubrey LE, Gavins FKH, et al.. Harnessing artificial intelligence for the next generation of 3D printed medicines. Adv Drug Deliv Rev, 2021, 175: 113805

[87] Zhao Y, Li W, Shi Z, et al.. Prediction of Dissolution Profiles From Process Parameters, Formulation, and Spectroscopic Measurements. J Pharm Sci, 2019, 108(6): 2119-2127

[88] Bakkar N, Kovalik T, Lorenzini I, et al.. Artificial intelligence in neurodegenerative disease research: use of IBM Watson to identify additional RNA-binding proteins altered in amyotrophic lateral sclerosis. Acta Neuropathol, 2018, 135(2): 227-247

[89] Bain EE, Shafner L, Walling DP, et al.. Use of a Novel Artificial Intelligence Platform on Mobile Devices to Assess Dosing Compliance in a Phase 2 Clinical Trial in Subjects With Schizophrenia. JMIR Mhealth Uhealth, 2017, 5(2): e18

[90] Mak KK, Pichika MR. Artificial intelligence in drug development: present status and future prospects. Drug Discov Today, 2019, 24(3): 773-780

[91] Kakisaka Y, Jin K, Fujikawa M, et al.. Teleconference-based education of epileptic seizure semiology. Epilepsy Res, 2018, 145: 73-76

[92] Ting DSW, Lin H, Ruamviboonsuk P, et al.. Artificial intelligence, the internet of things, and virtual clinics: ophthalmology at the digital translation forefront. Lancet Digit Health, 2020, 2(1): e8-e9

[93] Li X, Hao X. English Machine Translation Model Based on Artificial Intelligence. J Physics: Conference Series, 2021, 1982(1): 012098

[94] Kufel J, Paszkiewicz I, Bielówka M, et al.. Will ChatGPT pass the Polish specialty exam in radiology and diagnostic imaging? Insights into strengths and limitations. Pol J Radiol, 2023, 88: e430-e434

[95] Trocin C, Mikalef P, Papamitsiou Z, et al.. Responsible AI for Digital Health: a Synthesis and a Research Agenda. Inf Syst Front, 2023, 25(6): 2139-2157