Application and Prospect of Mixed Reality Technology in Medical Field

Abstract

Mixed reality (MR) technology is a new digital holographic image technology, which appears in the field of graphics after virtual reality (VR) and augmented reality (AR) technology, a new interdisciplinary frontier. As a new generation of technology, MR has attracted great attention of clinicians in recent years. The emergence of MR will bring about revolutionary changes in medical education training, medical research, medical communication, and clinical treatment. At present, MR technology has become the popular frontline information technology for medical applications. With the popularization of digital technology in the medical field, the development prospects of MR are inestimable. The purpose of this review article is to introduce the application of MR technology in the medical field and prospect its trend in the future.

Keywords

mixed reality / augmented reality / virtual reality / education / surgery

References

[1] Xiao2R, XiongG. Computer-assisted Surgery for Scaphoid Fracture. Curr Med Sci, 2018, 38(6): 941-948

[2] TepperOM, RudyHL, LefkowitzA, et al.. Mixed Reality with HoloLens: Where Virtual Reality Meets Augmented Reality in the Operating Room. Plast Reconstr Surg, 2017, 140(5): 1066-1070

[3] de RibaupierreS, EaglesonR. Editorial: Challenges for the usability of AR and VR for clinical neurosurgical procedures. Healthc Technol Lett, 2017, 4(5): 151

[4] FertlemanC, Aubugeau-WilliamsP, SherC, et al.. A Discussion of Virtual Reality As a New Tool for Training Healthcare Professionals. Front Public Health, 2018, 6: 44

[5] HamacherA, KimSJ, ChoST, et al.. Application of Virtual, Augmented, and Mixed Reality to Urology. Int Neurourol J, 2016, 20(3): 172-181

[6] BarsomEZ, GraaflandM, SchijvenMP. Systematic review on the effectiveness of augmented reality applications in medical training. Surg Endosc, 2016, 30(10): 4174-4183

[7] NicolaS, Stoicu-TivadarL. Mixed Reality Supporting Modern Medical Education. Stud Health Technol Inform, 2018255 242-255 246

[8] EckU, WinklerA. Display technologies for augmented reality in medical applications. Unfallchirurg, 2018, 121(4): 278-285

[9] BovaFJ, RajonDA, FriedmanWA, et al.. Mixed-reality simulation for neurosurgical procedures. Neurosurgery, 2013, 73(Suppl1): 138-145

[10] SappenfieldJW, SmithWB, CooperLA, et al.. Visualization Improves Supraclavicular Access to the Subclavian Vein in a Mixed Reality Simulator. Anesth Analg, 2018, 127(1): 83-89

[11] HalicT, KockaraS, BayrakC, et al.. Mixed reality simulation of rasping procedure in artificial cervical disc replacement (ACDR) surgery. BMC Bioinformatics, 2010, 11(Suppl6): S11

[12] HootenKG, ListerJR, LombardG, et al.. Mixed reality ventriculostomy simulation: experience in neurosurgical residency. Neurosurgery, 2014, 10(Suppl4): 576-581

[13] RobinsonA 3, GravensteinN, CooperLA, et al.. A mixed-reality part-task trainer for subclavian venous access. Simul Healthc, 2014, 9(1): 56-64

[14] KotranzaA, LindDS, LokB. Real-time evaluation and visualization of learner performance in a mixed-reality environment for clinical breast examination. IEEE Trans Vis Comput Graph, 2012, 18(7): 1101-1114

[15] ShenY, HananelD, ZhaoZ, et al.. A New Design for Airway Management Training with Mixed Reality and High Fidelity Modeling. Stud Health Technol Inform, 2016, 220: 359-362

[16] StefanP, PfandlerM, WuchererP, et al.. Team training and assessment in mixed reality-based simulated operating room: Current state of research in the field of simulation in spine surgery exemplified by the ATMEOS project. Unfallchirurg, 2018, 121(4): 271-277

[17] LuK, MarinoNE, RussellD, et al.. Use of Short Message Service and Smartphone Applications in the Management of Surgical Patients: A Systematic Review. Telemed J E Health, 2018, 24(6): 406-414

[18] RheeT, PetikamL, AllenB, et al.. MR360: Mixed Reality Rendering for 360 degrees Panoramic Videos. IEEE Trans Vis Comput Graph, 2017, 23(4): 1379-1388

[19] WuX, LiuR, YuJ, et al.. Mixed Reality Technology Launches in Orthopedic Surgery for Comprehensive Preoperative Management of Complicated Cervical Fractures. Surg Innov, 2018, 25(4): 421-422

[20] HeutsS, Sardari NiaP, MaessenJG. Preoperative planning of thoracic surgery with use of threedimensional reconstruction, rapid prototyping, simulation and virtual navigation. J Vis Surg, 2016, 2: 77

[21] WuX, LiuR, YuJ, et al.. Mixed Reality Technology-Assisted Orthopedics Surgery Navigation. Surg Innov, 2018, 25(3): 304-305

[22] SauerIM, QueisnerM, TangP, et al.. Mixed Reality in Visceral Surgery: Development of a Suitable Workflow and Evaluation of Intraoperative Use-cases. Ann Surg, 2017, 266(5): 706-712

[23] MertA, KieselB, WohrerA, et al.. Introduction of a standardized multimodality image protocol for navigation-guided surgery of suspected low-grade gliomas. Neurosurg Focus, 2015, 38(1): E4

[24] LinteCA, DavenportKP, ClearyK, et al.. On mixed reality environments for minimally invasive therapy guidance: systems architecture, successes and challenges in their implementation from laboratory to clinic. Comput Med Imaging Graph, 2013, 37(2): 83-97

[25] LeeSC, FuerstB, TatenoK, et al.. Multi-modal imaging, model-based tracking, and mixed reality visualisation for orthopaedic surgery. Healthc Technol Lett, 2017, 4(5): 168-173

[26] FischerM, FuerstB, LeeSC, et al.. Preclinical usability study of multiple augmented reality concepts for K-wire placement. Int J Comput Assist Radiol Surg, 2016, 11(6): 1007-1014

[27] IncekaraF, SmitsM, DirvenC, et al.. Clinical Feasibility of a Wearable Mixed-Reality Device in Neurosurgery. World Neurosurg, 2018, 118: e422-e427

[28] WangS, ParsonsM, Stone-McLeanJ, et al.. Augmented Reality as a Telemedicine Platform for Remote Procedural Training. Sensors (Basel), 2017, 17(10): E2294

[29] YetisenAK, Martinez-HurtadoJL, UnalB, et al.. Wearables in Medicine. Adv Mater, 2018e1706910

[30] AppelbaumL, MahgereftehSY, SosnaJ, et al.. Imageguided fusion and navigation: applications in tumor ablation. Tech Vasc Interv Radiol, 2013, 16(4): 287-295

[31] YoshinoM, SaitoT, KinT, et al.. A Microscopic Optically Tracking Navigation System That Uses Highresolution 3D Computer Graphics. Neurol Med Chir (Tokyo), 2015, 55(8): 674-679

[32] BealMD, DelagramaticasD, FitzD. Improving outcomes in total knee arthroplasty-do navigation or customized implants have a role. J Orthop Surg Res, 2016, 11(1): 60

[33] ZhangW, TakigawaT, WuY, et al.. Accuracy of pedicle screw insertion in posterior scoliosis surgery: a comparison between intraoperative navigation and preoperative navigation techniques. Eur Spine J, 2017, 26(6): 1756-1764

[34] LiL, YangJ, ChuY, et al.. A Novel Augmented Reality Navigation System for Endoscopic Sinus and Skull Base Surgery: A Feasibility Study. PLoS One, 2016, 11(1): e0146996

[35] BrighamTJ. Reality Check: Basics of Augmented, Virtual, and Mixed Reality. Med Ref Serv Q, 2017, 36(2): 171-178

[36] HannaMG, AhmedI, NineJ, et al.. Augmented Reality Technology Using Microsoft HoloLens in Anatomic Pathology. Arch Pathol Lab Med, 2018, 142(5): 638-644

[37] ChenAD, LinSJ. Discussion: Mixed Reality with HoloLens: Where Virtual Reality Meets Augmented Reality in the Operating Room. Plast Reconstr Surg, 2017, 140(5): 1071-1072

[38] Valdes OlmosRA, Vidal-SicartS, GiammarileF, et al.. The GOSTT concept and hybrid mixed/virtual/ augmented reality environment radioguided surgery. Q J Nucl Med Mol Imaging, 2014, 58(2): 207-215

[39] LeeC, RinconGA, MeyerG, et al.. The effects of visual realism on search tasks in mixed reality simulation. IEEE Trans Vis Comput Graph, 2013, 19(4): 547-556

[40] FerrariV, MegaliG, TroiaE, et al.. A 3-D mixed-reality system for stereoscopic visualization of medical dataset. IEEE Trans Biomed Eng, 2009, 56(11): 2627-2633

[41] MartelliN, SerranoC, van den BrinkH, et al.. Advantages and disadvantages of 3-dimensional printing in surgery: A systematic review. Surgery, 2016, 159(6): 1485-1500