Leveraging the smarts in your phone: An artificial intelligence-driven iOS application for neurosurgical navigation of external ventricular drains

Abstract

External ventricular drain (EVD) placement is a critical neurosurgical procedure traditionally performed freehand, with inherent risks of malposition, infection, and hemorrhage. Recent advances in artificial intelligence (AI), particularly in medical imaging and real-time computer vision, have enabled the development of portable navigation tools that may enhance accuracy, safety, and bedside accessibility. This study evaluated whether iOS devices equipped with a TrueDepth camera could perform real-time object and facial recognition, tracking, and semantic segmentation of computed tomography (CT) scans for non-immobilized heads to guide EVD placement via a custom AI-driven application. A custom iOS application was developed to provide a complete, real-time surgical navigation experience on an iPhone or iPad Pro. Three AI models were trained, tuned, and validated: a semantic segmentation model for brain anatomy, a semantic segmentation model for facial features, and an object detection model for a custom EVD stylet attachment. GPU programming accelerated on-device real-time, continuous registration while optimizing power consumption. A UNet convolutional neural network trained on eight 1 mm head CTs achieved 98.3% testing and 98.2% validation accuracy using a 50/50 test-validation split, segmenting a thin-cut CT in 3 s on an iPhone 12 Pro. Point cloud merging of patient anatomy took 4 seconds with an initial depth scan of 30,000 points, updating in real time with a cumulative error of 1 × 10^-8 cm. Transfer learning-powered EVD tracking, trained for 1,000 epochs, achieved an intersection over union of 1.0 and 0.98 for the detection model, with inference times of 800 μs on Apple’s Neural Engine. This feasibility study demonstrates that iOS devices with TrueDepth cameras can enable real-time, continuous surgical navigation for EVD stylets.

Keywords

External ventricular drain / Surgical navigation / Artificial intelligence / Machine learning

Funding

This work was graciously funded by UNC Health’s Innovation Pilot Grant (Grant no.: 29201).

Conflict of interest

The authors declare they have no competing interests.

References

[1] Grunert P, Darabi K, Espinosa J, Filippi R. Computer-aided navigation in neurosurgery. Neurosurg Rev. 2003; 26:73-99; discussion100-1. doi: 10.1007/s10143-003-0262-0

[2] Mezger U, Jendrewski C, Bartels M. Navigation in surgery. Langenbecks Arch Surg. 2013; 398:501-514. doi: 10.1007/s00423-013-1059-4

[3] Tagaytayan R, Kelemen A, Sik-Lanyi C. Augmented reality in neurosurgery. Arch Med Sci. 2018; 14(3):572-578. doi: 10.5114/aoms.2016.58690

[4] Rahmathulla G, Nottmeier EW, Pirris SM, Deen HG, Pichelmann MA. Intraoperative image-guided spinal navigation: Technical pitfalls and their avoidance. Neurosurg Focus. 2014; 36(3):E3. doi: 10.3171/2014.1.FOCUS13516

[5] Moiraghi A, Pallud J. Intraoperative ultrasound techniques for cerebral gliomas resection: Usefulness and pitfalls. Ann Transl Med. 2020; 8(8):523. doi: 10.21037/atm.2020.03.178

[6] Khoshnevisan A, Allahabadi NS. Neuronavigation: Principles, clinical applications and potential pitfalls. Iran J Psychiatry. 2012; 7(2):97-103.

[7] Harwick E, Singhal I, Conway B, Mueller W, Treffy R, Krucoff MO. Pinless electromagnetic neuronavigation during awake craniotomies: Technical pearls, pitfalls, and nuances. World Neurosurg. 2023; 175:e159-e166. doi: 10.1016/j.wneu.2023.03.045

[8] Anwar SM, Majid M, Qayyum A, Awais M, Alnowami M, Khan MK. Medical image analysis using convolutional neural networks: A review. J Med Syst. 2018; 42:226. doi: 10.1007/s10916-018-1088-1

[9] Ronneberger O, Fischer P, Brox T. U-net: Convolutional Networks for Biomedical Image Segmentation. Berlin: Springer; 2015. p. 234-241.

[10] Rawat W, Wang Z. Deep convolutional neural networks for image classification: A comprehensive review. Neural Computat. 2017; 29(9):2352-2449.

[11] Gu J, Wang Z, Kuen J, et al. Recent advances in convolutional neural networks. Pattern Recogn. 2018; 77:354-377.

[12] Albawi S, Mohammed TA, Al-Zawi S. Understanding of a Convolutional Neural Network. New York: IEEE; 2017. p. 1-6.

[13] Yamashita R, Nishio M, Do RKG, Togashi K. Convolutional neural networks: An overview and application in radiology. Insights Imaging. 2018; 9:611-629. doi: 10.1007/s13244-018-0639-9

[14] Huang J, Shen H, Wu J, et al. Spine Explorer: A deep learning based fully automated program for efficient and reliable quantifications of the vertebrae and discs on sagittal lumbar spine MR images. Spine J. 2020; 20(4):590-599. doi: 10.1016/j.spinee.2019.11.010

[15] Lehnen NC, Haase R, Faber J, et al. Detection of degenerative changes on MR images of the lumbar spine with a convolutional neural network: A feasibility study. Diagnostics (Basel). 2021; 11(5):902. doi: 10.3390/diagnostics11050902

[16] Cheng YK, Lin CL, Huang YC, et al. Automatic segmentation of specific intervertebral discs through a two-stage multiresunet model. J Clin Med. 2021; 10(20):4760. doi: 10.3390/jcm10204760

[17] Lessmann N, Van Ginneken B, De Jong PA, Išgum I. Iterative fully convolutional neural networks for automatic vertebra segmentation and identification. Med Image Anal. 2019; 53:142-155. doi: 10.1016/j.media.2019.02.005

[18] Janssens R, Zeng G, Zheng G. Fully Automatic Segmentation of Lumbar Vertebrae from CT Images using Cascaded 3D Fully Convolutional Networks. New York: IEEE; 2018. p. 893-897.

[19] Caesarendra W, Rahmaniar W, Mathew J, Thien A. Automated Cobb angle measurement for adolescent idiopathic scoliosis using convolutional neural network. Diagnostics (Basel). 2022; 12(2):396. doi: 10.3390/diagnostics12020396

[20] Sun Y, Xing Y, Zhao Z, Meng X, Xu G, Hai Y. Comparison of manual versus automated measurement of Cobb angle in idiopathic scoliosis based on a deep learning keypoint detection technology. Eur Spine J. 2022; 31:1969-1978. doi: 10.1007/s00586-021-07025-6

[21] Small J, Osler P, Paul A, Kunst M. CT cervical spine fracture detection using a convolutional neural network. AJNR Am J Neuroradiol. 2021; 42(7):1341-1347. doi: 10.3174/ajnr.A7094

[22] Seetha J, Raja SS. Brain tumor classification using convolutional neural networks. Biomed Pharmacol J. 2018; 11(3):1457.

[23] Pereira S, Pinto A, Alves V, Silva CA. Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans Med Imaging. 2016; 35(5):1240-1251.

[24] Abumoussa A, Gopalakrishnan V, Succop B, et al. Machine learning for automated and real-time two-dimensional to three-dimensional registration of the spine using a single radiograph. Neurosurg Focus. 2023; 54(6):E16. doi: 10.3171/2023.3.FOCUS2345

[25] Eckert M, Volmerg JS, Friedrich CM. Augmented reality in medicine: Systematic and bibliographic review. JMIR Mhealth Uhealth. 2019; 7(4):e10967. doi: 10.2196/10967

[26] Brookes MJ, Chan CD, Baljer B, et al. Surgical advances in osteosarcoma. Cancers (Basel). 2021; 13(3):388. doi: 10.3390/cancers13030388

[27] Kraeima J, Glas HH, Merema BBJ, Vissink A, Spijkervet FK, Witjes MJ. Three-dimensional virtual surgical planning in the oncologic treatment of the mandible. Oral Dis. 2021; 27(1):14-20. doi: 10.1111/odi.13631

[28] Bobak MJ, Weber MW, Doellman MA, et al. Modafinil activates phasic dopamine signaling in dorsal and ventral striata. J Pharmacol Exp Ther. 2016; 359(3):460-470. doi: 10.1124/jpet.116.236000

[29] Lavé A, Meling TR, Schaller K, Corniola MV. Augmented reality in intracranial meningioma surgery: Report of a case and systematic review. J Neurosurg Sci. 2020; 64(4):369-376. doi: 10.23736/S0390-5616.20.04945-0

[30] Lee C, Wong GKC. Virtual reality and augmented reality in the management of intracranial tumors: A review. J Clin Neurosci. 2019; 62:14-20. doi: 10.1016/j.jocn.2018.12.036

[31] Gerard IJ, Kersten-Oertel M, Petrecca K, Sirhan D, Hall JA, Collins DL. Brain shift in neuronavigation of brain tumors: A review. Med Image Anal. 2017; 35:403-420. doi: 10.1016/j.media.2016.08.007

[32] Inoue D, Cho B, Mori M, et al. Preliminary study on the clinical application of augmented reality neuronavigation. J Neurol Surg A Cent Eur Neurosurg. 2013; 74(2):71-76. doi: 10.1055/s-0032-1333415

[33] Tabrizi LB, Mahvash M. Augmented reality-guided neurosurgery: Accuracy and intraoperative application of an image projection technique. J Neurosurg. 2015; 123(1):206-211. doi: 10.3171/2014.9.JNS141001

[34] Cabrilo I, Sarrafzadeh A, Bijlenga P, Landis B, Schaller K. Augmented reality-assisted skull base surgery. Neurochirurgie. 2014; 60(6):304-306. doi: 10.1016/j.neuchi.2014.07.001

[35] Molina CA, Phillips FM, Poelstra KA, Colman M, Khoo LT. 151. A cadaveric precision and accuracy analysis of augmented reality mediated percutaneous pedicle implant insertion. Spine J. 2020; 20(9):S74.

[36] Burström G, Persson O, Edström E, Elmi-Terander A. Augmented reality navigation in spine surgery: A systematic review. Acta Neurochir (Wien). 2021;163:843-852. doi: 10.1007/s00701-021-04708-3

[37] Yuk FJ, Maragkos GA, Sato K, Steinberger J. Current innovation in virtual and augmented reality in spine surgery. Ann Transl Med. 2021; 9(1):94. doi: 10.21037/atm-20-1132

[38] Vadalà G, De Salvatore S, Ambrosio L, Russo F, Papalia R, Denaro V. Robotic spine surgery and augmented reality systems: A state of the art. Neurospine. 2020; 17(1):88-100. doi: 10.14245/ns.2040060.030

[39] Parsons D, MacCallum K. Current perspectives on augmented reality in medical education: Applications, affordances and limitations. Adv Med Educ Pract. 2021; 12:77-91. doi: 10.2147/AMEP.S249891

[40] Williams MA, McVeigh J, Handa AI, Lee R. Augmented reality in surgical training: A systematic review. Postgrad Med J. 2020; 96(1139):537-542. doi: 10.1136/postgradmedj-2020-137600

[41] Muralidharan R. External ventricular drains: Management and complications. Surg Neurol Int. 2015; 6(Suppl 6):S271-S274. doi: 10.4103/2152-7806.157620

[42] Chau CYC, Craven CL, Rubiano AM, et al. The evolution of the role of external ventricular drainage in traumatic brain injury. J Clin Med. 2019; 8(9):1422. doi: 10.3390/jcm8091422

[43] Huyette DR, Turnbow BJ, Kaufman C, Vaslow DF, Whiting BB, Oh MY. Accuracy of the freehand pass technique for ventriculostomy catheter placement: Retrospective assessment using computed tomography scans. J Neurosurg. 2008; 108(1):88-91. doi: 10.3171/jns/2008/108/01/0088

[44] Brattain LJ, Pierce TT, Gjesteby LA, et al. AI-enabled, ultrasound-guided handheld robotic device for femoral vascular access. Biosensors (Basel). 2021; 11(12):522. doi: 10.3390/bios11120522

[45] Chilamkurthy S, Ghosh R, Tanamala S, et al. Deep learning algorithms for detection of critical findings in head CT scans: A retrospective study. Lancet. 2018; 392(10162):2388-2396. doi: 10.1016/S0140-6736(18)31645-3

[46] Bevan N. Classifying and Selecting UX and Usability Measures. Toulouse, France: Institute of Research in Informatics of Toulouse (IRIT); 2008. p. 13-18.

[47] Support A. About Face ID Advanced Technology.Available from: https://support.apple.com/en-us/102381#:-:text=Face%20ID%20works%20best%20when,camera%20can%20see%20your%20eyes [Last accessed on 2025 Aug 31].

[48] Depth estimation Technology in Iphones. Available from: https://www.opencv.ai/blog/depth-estimation#:-:text=Depth%20in%20iPhone,TrueDepth%20camera%2C%20and%20Scene%20Geometry [Last accessed on 2025 Aug 31].

[49] Sang J, Wu Z, Guo P, et al. An improved YOLOv2 for vehicle detection. Sensors. 2018; 18(12):4272. doi: 10.3390/s18124272

[50] Kakarla UK, Kim LJ, Chang SW, Theodore N, Spetzler RF. Safety and accuracy of bedside external ventricular drain placement. Neurosurgery. 2008; 63(1 Suppl 1):ONS162- ONS166; discussion ONS166-ONS167. doi: 10.1227/01.neu.0000335031.23521.d0

[51] Patel EA, Aydin A, Cearns M, Dasgupta P, Ahmed K. A systematic review of simulation-based training in neurosurgery, part 1: Cranial neurosurgery. World Neurosurg. 2020; 133:e850-e873. doi: 10.1016/j.wneu.2019.08.262

[52] Sakai D, Joyce K, Sugimoto M, et al. Augmented, virtual and mixed reality in spinal surgery: A real-world experience. J Orthop Surg. 2020; 28(3):2309499020952698. doi: 10.1177/2309499020952698