Construction of a Multimodal 3D Atlas for a Micrometer-Scale Brain-Computer Interface Based on Mixed Reality

Abstract

Objective

To develop a multimodal imaging atlas of a rat brain-computer interface (BCI) that incorporates brain, arterial, bone tissue and a BCI device using mixed reality (MR) for three-dimensional (3D) visualization.

Methods

An invasive BCI was implanted in the left visual cortex of 4-week-old Sprague-Dawley rats. Multimodal imaging techniques, including micro-CT and 9.0 T MRI, were used to acquire images of the rat cranial bone structure, vascular distribution, brain tissue functional zones, and BCI device before and after implantation. Using 3D-slicer software, the images were fused through spatial transformations, followed by image segmentation and 3D model reconstruction. The HoloLens platform was employed for MR visualization.

Results

This study constructed a multimodal imaging atlas for rats that included the skull, brain tissue, arterial tissue, and BCI device coupled with MR technology to create an interactive 3D anatomical model.

Conclusions

This multimodal 3D atlas provides an objective and stable reference for exploring complex relationships between brain tissue structure and function, enhancing the understanding of the operational principles of BCIs. This is the first multimodal 3D imaging atlas related to a BCI created using Sprague-Dawley rats.

Keywords

Brain-computer interface / Mixed reality / Three-dimensional atlas / Multimodal imaging

© The Author(s), under exclusive licence to Huazhong University of Science and Technology 2025

Author Contributions Hong Zhou, Zi-neng Yan, Wei-hang Gao and Zhe-wei Ye contributed to conception and design of the study. Xiang- xin Lv, Rui Luo, Jia-ming Yang, Ming-di Xue, Ying Fang, Yu-yu Duan and Rui-yuan Li contributed to acquisition and processing data. Xu- dong Wang, Rui-lin Wang and Jia-yao Zhang performed the image making. Hong-lin Wang, Yi Xie and Xiao-liang Chen wrote sections of the manuscript. Jason Shih Hoellwarth, Lei He, Li Huang, Mao Xie, Peng-ran Liu and Zhe-wei Ye contributed to manuscript revision and supervision. All authors contributed to the article and approved the submitted version.

Funding This work was supported by the National Natural Science Foundation of China (No. 82172524 and No. 81974355); National Innovation Platform Development Program (No. 2020021105012440), China; Major Program of Hubei Province (No. 2021BEA161), China; Major Key Project of Hubei Province (No. JD2023BAA005), China; Wuhan Union Hospital Free Innovation Preliminary Research Fund (No. 2024XHYN047), China; Joint Funds for the Innovation of Science and Technology (No. 2024Y9062), Fujian Province, China.

Consent for Publication Not applicable.

Ethics Approval and Consent to Participate All the animal experiments were carried out in accordance with the guidelines of the Declaration of Helsinki, Hubei Province Laboratory Animal Management Regulations and Institutional Animal Care and Use Committee of Huazhong University of Science and Technology ([2024] IACUC Number: 4271) and with the approval of the Animal Ethics Committee of Huazhong University of Science and Technology.

References

[1] Chang EF. Brain-Computer Interfaces for Restoring Communication. N Engl J Med. 2024; 391(7):654-657.

[2] Willett FR, Avansino DT, Hochberg LR, et al. High-performance brain-to-text communication via handwriting. Nature. 2021; 593(7858):249-254.

[3] Liu Y, Liu R, Ge J, et al. Advancements in brain-machine interfaces for application in the metaverse. Front Neurosci. 2024; 18:1383319.

[4] Abbasi J. Advanced brain-computer interface for people with paralysis. JAMA. 2019; 321(6):537.

[5] Orban M, Elsamanty M, Guo K, et al. A Review of Brain Activity and EEG-Based Brain-Computer Interfaces for Rehabilitation Application. Bioengineering. 2022; 9(12):768.

[6] Wu X, Metcalfe B, He S, et al. A Review of Motor Brain-Computer Interfaces Using Intracranial Electroencephalography Based on Surface Electrodes and Depth Electrodes. IEEE Trans Neural Syst Rehabil Eng. 2024; 32:2408-2431.

[7] Rouzitalab A, Boulay CB, Park J, et al. Intracortical brain-computer interfaces in primates: a review and outlook. Biomed Eng Lett. 2023; 13(3):375-390.

[8] Ma Y, Gong A, Nan W, et al. Personalized Brain-Computer Interface and Its Applications. J Pers Med. 2022; 13(1):46.

[9] Klein F. Optimizing spatial specificity and signal quality in fNIRS: an overview of potential challenges and possible options for improving the reliability of real-time applications. Front Neuroergonomics. 2024; 5:1286586.

[10] Yu L, Liu Q, Bai X, et al. A digital rat atlas of sectional anatomy. In: Xu K, Luo Q, Xing D et al. editors. SPIE Proceedings. Tianjin, China; 2006. p. 60470Y.

[11] Liao Y, Yu L, Bai X, et al. 3D model construction for arterial vessels system of digital rat. Sheng Wu Yi Xue Gong Cheng Xue Za Zhi (Chinese). 2008; 25(3):552-555.

[12] Perens J, Salinas CG, Roostalu U, et al. Multimodal 3D Mouse Brain Atlas Framework with the Skull-Derived Coordinate System. Neuroinform. 2023; 21(2):269-286.

[13] Huang S, Lu Z, Huang W, et al.High-resolution in vivo Wistar rodent brain atlas based on T 1 weighted image. In Proceedings of SPIE Photonics Asia 2016; Chengdu, China; 2016 Sep 26-30. Berlingham, WA: SPIE; 2016. p. 978825.

[14] Bai X, Yu L, Liu Q, et al. A high‐resolution anatomical rat atlas. J Anat. 2006; 209(5):707-708.

[15] Nie B, Chen K, Zhao S, et al. A rat brain MRI template with digital stereotaxic atlas of fine anatomical delineations in paxinos space and its automated application in voxel‐wise analysis. Human Brain Mapping. 2013; 34(6):1306-1318.

[16] Sadeghi M, Ramos-Prats A, Neto P, et al. Localization and Registration of 2D Histological Mouse Brain Images in 3D Atlas Space. Neuroinform. 2023; 21(3):615-630.

[17] Naito S, Kajiwara M, Nakashima R, et al. Application of Extended Reality (Virtual Reality and Mixed Reality) Technology in Laparoscopic Liver Resections. Cureus. 2023; 15(9):e44520.

[18] Chytas D, Nikolaou VS. Mixed reality for visualization of orthopedic surgical anatomy. World J Orthop. 2021; 12(10):727-731.

[19] Qi Z, Jin H, Xu X, et al. Head model dataset for mixed reality navigation in neurosurgical interventions for intracranial lesions. Sci Data. 2024; 11(1):538.

[20] Jeising S, Liu S, Blaszczyk T, et al. Combined use of 3D printing and mixed reality technology for neurosurgical training: getting ready for brain surgery. Neurosurg Focus. 2024; 56(1):E12.

[21] Bartella AK, Kamal M, Kuhnt T, et al. Mixed reality in oral and maxillofacial surgery: a symbiosis of virtual and augmented reality or a pointless technological gadget? Int J Comput Dent. 2021; 24(1):65-76.

[22] Jain S, Timofeev I, Kirollos RW, et al. Use of Mixed Reality in Neurosurgery Training: A Single Centre Experience. World Neurosurg. 2023; 176:e68-e76.

[23] Andrews A. Integration of augmented reality and brain-computer interface technologies for health care applications: exploratory and prototyping study. JMIR Form Res. 2022; 6(4):e18222.

[24] Schoch CL, Ciufo S, Domrachev M, et al. NCBI Taxonomy: a comprehensive update on curation, resources and tools. Database (Oxford). 2020;2020:baaa062.

[25] Zhu Y, Xu K, Xu C, et al. PET Mapping for Brain-Computer Interface Stimulation of the Ventroposterior Medial Nucleus of the Thalamus in Rats with Implanted Electrodes. J Nucl Med. 2016; 57(7):1141-1145.

[26] Charvet G, Billoint O, Gharbi S, et al. A modular 256-channel Micro Electrode Array platform for in vitro and in vivo neural stimulation and recording: BioMEA. Annu Int Conf IEEE Eng Med Biol Soc. 2010; 2010:1804-1807.

[27] Fedorov A, Beichel R, Kalpathy-Cramer J, et al. 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magnetic Resonance Imaging. 2012; 30(9):1323-1341.

[28] Kleven H, Bjerke IE, Clascá F, et al. Waxholm Space atlas of the rat brain: a 3D atlas supporting data analysis and integration. Nat Methods. 2023; 20(11):1822-1829.

[29] Papp EA, Leergaard TB, Calabrese E, et al. Waxholm Space atlas of the Sprague Dawley rat brain. NeuroImage. 2014; 97:374-386.

[30] Liu S, Xie M, Zhang Z, et al. A 3D Hologram With Mixed Reality Techniques to Improve Understanding of Pulmonary Lesions Caused by COVID-19: Randomized Controlled Trial. J Med Internet Res. 2021; 23(9):e24081.

[31] Lu L, Wang H, Liu P, et al. Applications of Mixed Reality Technology in Orthopedics Surgery: A Pilot Study. Front Bioeng Biotechnol. 2022; 10:740507.

[32] Wang Q, Ding SL, Li Y, et al. The Allen Mouse Brain Common Coordinate Framework: A 3D Reference Atlas. Cell. 2020; 181(4):936-953.e20.

[33] Chen J, Xia Y, Zhou X, et al. fNIRS-EEG BCIs for Motor Rehabilitation: A Review. Bioengineering. 2023; 10(12):1393.

[34] Chai X, Cao T, He Q, et al. Brain-computer interface digital prescription for neurological disorders. CNS Neurosci Ther. 2024; 30(2):e14615.

[35] De Miguel-Rubio A, Gallego-Aguayo I, De Miguel-Rubio MD, et al. Effectiveness of the Combined Use of a Brain-Machine Interface System and Virtual Reality as a Therapeutic Approach in Patients with Spinal Cord Injury: A Systematic Review. Health-care. 2023; 11(24):3189.

[36] Lin S, Jiang J, Huang K, et al. Advanced Electrode Technologies for Noninvasive Brain-Computer Interfaces. ACS Nano. 2023; 17(24):24487-24513.

[37] Hawrylycz M, Baldock RA, Burger A, et al. Digital Atlasing and Standardization in the Mouse Brain. PLoS Comput Biol. 2011; 7(2):e1001065.

[38] Shcherbatyy V, Carson J, Yaylaoglu M, et al. A Digital Atlas of Ion Channel Expression Patterns in the Two-Week-Old Rat Brain. Neuroinform. 2015; 13(1):111-125.

[39] Zhang H, Jiao L, Yang S, et al. Brain-computer interfaces: the innovative key to unlocking neurological conditions. Int J Surg. 2024; 110(9):5745-5762.

[40] Pelanis E, Kumar RP, Aghayan DL, et al. Use of mixed reality for improved spatial understanding of liver anatomy. Minim Invasive Ther Allied Technol. 2020; 29(3):154-160.

[41] Pasquali M, Fusini L, Italiano G, et al. Feasibility study of a mixed reality tool for real 3D visualization and planning of left atrial appendage occlusion. J Cardiovasc Comput Tomog. 2022; 16(5):460-462.