Research Progress on Motor Imagery Decoding in Brain-Computer Interface Based on EEG-fNIRS Multimodal Fusion

Abstract

Motor imagery-based brain-computer interface (MI-BCI) decodes subjective motor intentions to achieve proactive output control of external devices, representing a core research direction in BCI with important applications in post-stroke motor rehabilitation. Neural signals for motor imagery can be recorded using electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). Combining the high temporal resolution of EEG with the high spatial resolution of fNIRS through multimodal fusion enhances the decoding accuracy of motor imagery commands and provides comprehensive insights into brain dynamics. This paper systematically reviews EEG-fNIRS multimodal fusion strategies and representative algorithms from traditional machine learning (ML) and deep learning (DL) for decoding fused data. We also discuss current challenges and potential solutions to facilitate MI-BCI research and support future BCI-related industries.

Keywords

Brain-computer interface / Electroencephalography / Functional near-infrared spectroscopy / Motor imagery / Multimodal fusion

References

[1] Rodríguez-Obregón DE, Moreno HA, Carrera IG, Lopez-Lopez L . A Decade-Long Systematic Review of EEG-Driven Gait-Intention Interfaces for Lower-Limb Rehabilitation and Prosthetics. Appl. Mech. Mater. 2025, 929, 107-118. DOI: 10.4028/p-rpE8vv

[2] Palanichamy C, Thirumoorthi SP, Lakshminarayanan K, Madathil D, Rahman MH . Multimodal brain-computer interface for robotic control: Integration of real-time gaze tracking and EEG-based motor imagery. Med. Biol. Eng. Comput. 2026, 64, 781-792. DOI: 10.1007/s11517-025-03489-6

[3] Zhang M, Yang Y, Zhao Y, Sui C, Sui Y, Jiang Y, et al. The application of integrating electroencephalograph-based emotion recognition technology into brain-computer interface systems for the treatment of depression: A narrative review. Adv. Technol. Neurosci. 2024, 1, 188-200. DOI: 10.4103/ATN.ATN-D-24-00018

[4] Zhang Y, Liu D, Li T, Zhang P, Li Z, Gao F. CGAN-rIRN: A data-augmented deep learning approach to accurate classification of mental tasks for a fNIRS-based brain-computer interface. Biomed. Opt. Express 2023, 14, 2934-2954. DOI: 10.1364/BOE.489179

[5] Guo JC. Frequent policies boost the accelerated breakthrough of the brain-computer interface industry chain. Securities Daily, 6 February 2025. DOI: 10.28096/n.cnki.ncjrb.2025.000509

[6] Qin X. Beijing and Shanghai introduce action plans, brain-computer interface industry may explode. China Business News, 10 February 2025. DOI: 10.38300/n.cnki.nzgjy.2025.000199

[7] Chai YY. Research on Feature Extraction and Classification Methods for Motor Imagery EEG Signals. Master’s Thesis, Beijing University of Chemical Technology, Beijing, China, 2024.

[8] Xu GY.Research on Classification and Recognition of Motor Imagery EEG Signals in Brain-Computer Interface. Master’s Thesis, Xi’an Technological University, Xi’an, China, 2024.

[9] Murtaza H, Ahmed M, Murtaza G, Zafar SN, Nazir T . A novelty towards neural signatures-Unveiling the inter-subject distance metric for EEG-based motor imagery. Biomed. Signal Process. Control 2025, 105, 107552. DOI: 10.1016/j.bspc.2025.107552

[10] AL-Quraishi MS, Ahmed G, AbouOmar MS, Ouerdane F, Eltayeb A, Alyazidi NM, et al. A Brain-Machine Interface Framework for Motor Imagery Classification Using Functional Connectivity and Ensemble Learning: Toward Real-Time Applications in Healthcare and Industry. Arab. J. Sci. Eng. 2026, 51, 15973-15987. DOI: 10.1007/s13369-025-10616-w

[11] Du PF.Research on Motion Planning of Upper Limb Rehabilitation Robot Based on Motor Imagery EEG Signals. Master’s Thesis, Anhui University of Science and Technology, Huainan, China, 2025.

[12] Cariello S, Sanalitro D, Micali A, Buscarino A, Bucolo M. Brain-Computer-Interface-Based Smart-Home Interface by Leveraging Motor Imagery Signals. Inventions 2023, 8, 91. DOI: 10.3390/inventions8040091

[13] Al-qaysi ZT, Zaidan BB, Zaidan AA, Suzani MS . A review of disability EEG based wheelchair control system: Coherent taxonomy, open challenges and recommendations. Comput. Methods Programs Biomed. 2018, 164, 221-237. DOI: 10.1016/j.cmpb.2018.06.012

[14] Melissant C, Ypma A, Frietman EEE, Stam CJ . A method for detection of Alzheimer's disease using ICA-enhanced EEG measurements. Artif. Intell. Med. 2005, 33, 209-222. DOI: 10.1016/j.artmed.2004.07.003

[15] Yan H, Zhou Z, He X, Li J, Yuan Z,Wang H. Dynamic graph-temporal convolutional neural network for EEG-fNIRS multimodal motor imagery/execution decoding. Control Theory Appl. 2024, 36, 1-8. Available online: https://link.cnki.net/urlid/44.1240.TP.20241130.0849.002 (accessed on 27 July 2026).

[16] Blanco R, Koba C, Crimi A . Investigating the interaction between EEG and fNIRS: A multimodal network analysis of brain connectivity. J. Comput. Sci. 2024, 82, 102416. DOI: 10.1016/j.jocs.2024.102416

[17] Zhang J, Zhang ZY, Wang YL, Zhou B, Xia CY, Su M, et al. Unveiling the upper-limb functional recovery mechanisms in stroke patients using brain-machine interfaces: A near-infrared functional imaging-based study. Sci. Rep. 2025, 15, 39704. DOI: 10.1038/s41598-025-23267-6

[18] Balendra, Negi PC, Sharma N, Sharma S . Scalogram sets based motor imagery EEG classification using modified vision transformer: A comparative study on scalogram sets. Biomed. Signal Process. Control 2025, 104, 107640. DOI: 10.1016/j.bspc.2025.107640

[19] Codina T, Blankertz B, Lühmann AV . Multimodal fNIRS-EEG sensor fusion: Review of data-driven methods and perspective for naturalistic brain imaging. Imaging Neurosci. 2025, 3, IMAG-a. DOI: 10.1162/IMAG.a.974

[20] Deligani RJ, Borgheai SB, McLinden J, Shahriari Y. Multimodal fusion of EEG-fNIRS: A mutual information-based hybrid classification framework. Biomed. Opt. Express 2021, 12, 1635. DOI: 10.1364/BOE.413666

[21] Liu M, Yang B, Meng L, Zhang Y, Gao S, Zan P, et al. STA-Net: Spatial-temporal alignment network for hybrid EEG-fNIRS decoding. Inf. Fusion 2025, 119, 103023. DOI: 10.1016/j.inffus.2025.103023

[22] Zhou H, Zhao Y, Liu Y, Lu S, An X, Liu Q. Multi-Sensor Data Fusion and CNN-LSTM Model for Human Activity Recognition System. Sensors 2023, 23, 4750. DOI: 10.3390/s23104750

[23] Faisal M, Sahoo S, Hazarika J. STeCANet: Spatio-temporal cross attention network for brain computer interface systems using EEG-fNIRS signals. J. Neural Eng. 2025, 22, 066035. DOI: 10.1088/1741-2552/ae2954

[24] Chen J, Wang D, Hu B, Yi W, Xu M, Chen D, et al. MCFHNet: Multi-Channel Fusion Hybrid Network for Efficient EEG-fNIRS Multi-modal Motor Imagery Decoding. In Proceedings of the 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Glasgow, UK, 11-15 July 2022; pp. 4821-4825. DOI: 10.1109/EMBC48229.2022.9871385

[25] Li LZ, Meng M, Gao YY, Ma YL. A data fusion method for hybird EEG-fNIRS BCI base on wavelet transform. Chin. J. Sens. Actuators. 2023, 36, 1064-1072. Available online: https://chinatransducers.seu.edu.cn/ch/reader/view_abstract.aspx?file_no=cg220191&flag=1 (accessed on 27 July 2026).

[26] Wojewoda KK, Culmer PR, Jackson AE, Levesley MC. Position tracking of a passive rehabilitation robot. In Proceedings of the 2016 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), Chengdu, China, 19-22 June 2016; pp. 1-6. DOI: 10.1109/CYBER.2016.7574785

[27] Li H. DeepSyncNet: Deep synchronized fusion network for EEG-fNIRS multimodal brain-computer interfaces. Biomed. Signal Process. Control 2026, 113, 109100. DOI: 10.1016/j.bspc.2025.109100

[28] Muller CO, Prampart T, Bannier E, Corouge I, Maurel P. Evaluating the effects of multimodal EEG-fNIRS neurofeedback for motor imagery: An experimental platform and study protocol. PLoS ONE 2025, 20, e0331177. DOI: 10.1371/journal.pone.0331177

[29] Amin SU, Alsulaiman M, Muhammad G, Mekhtiche MA, Shamim Hossain M. Deep Learning for EEG motor imagery classification based on multi-layer CNNs feature fusion. Future Gener. Comput. Syst. 2019, 101, 542-554. DOI: 10.1016/j.future.2019.06.027

[30] Bunterngchit C, Wang J, Su J, Wang Y, Liu S, Hou ZG. AMFN: Autoencoder-led Multimodal Fusion Network for EEG-fNIRS Classification. Procedia Comput. Sci. 2024, 250, 8-14. DOI: 10.1016/j.procs.2024.11.002

[31] Yang S, Yin Z, Wang Y, Zhang W, Wang Y, Zhang J. Assessing cognitive mental workload via EEG signals and an ensemble deep learning classifier based on denoising autoencoders. Comput. Biol. Med. 2019, 109, 159-170. DOI: 10.1016/j.compbiomed.2019.04.034

[32] Ali MU, Zafar A, Kallu KD, Masood H, Mannan MM, Ibrahim MM, et al. Correlation-filter-based channel and feature selection framework for hybrid EEG-fNIRS BCI applications. IEEE J. Biomed. Health Inform. 2023, 28, 3361-3370. DOI: 10.1109/JBHI.2023.3294586

[33] Guo M, Feng L, Chen X, Li M, Xu G. A novel strategy for differentiating motor imagination brain-computer interface tasks by fusing EEG and functional near-infrared spectroscopy signals. Biomed. Signal Process. Control 2024, 95, 106448. DOI: 10.1016/j.bspc.2024.106448

[34] Lin J, Lu J, Shu Z, Han J, Yu N. Subject-Specific Modeling of EEG-fNIRS Neurovascular Coupling by Task-Related Tensor Decomposition. IEEE Trans. Neural Syst. Rehabil. Eng. 2024, 32, 452-461. DOI: 10.1109/TNSRE.2024.3355121

[35] Zhu Z, Han J, Zhang Z, Wannawas N, Faisal AA.Identifying the Nature of Grip Force Signals in EEG & fNIRS with Multi-Modal Graph Fusion Network. In Proceedings of the 2025 47th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Copenhagen, Denmark, 14-18 July 2025; pp. 1-7. DOI: 10.1109/EMBC58623.2025.11254624

[36] Gao T, Chen C, Liang G, Ran Y, Huang Q, Liao Z, et al. Feature fusion analysis approach based on synchronous EEG-fNIRS signals: Application in etomidate use disorder individuals. Biomed. Opt. Express 2025, 16, 382. DOI: 10.1364/BOE.542078

[37] Nia AF, Tang V, Maso Talou GD, Billinghurst M. Decoding emotions through personalized multi-modal fNIRS-EEG Systems: Exploring deterministic fusion techniques. Biomed. Signal Process. Control 2025, 105, 107632. DOI: 10.1016/j.bspc.2025.107632

[38] Shi J, Chen D, Zhao X, Zhao Z, Li S, Xu Y, et al. HEFMI-ICH: A hybrid EEG-fNIRS motor imagery dataset for brain-computer interface in intracerebral hemorrhage. Sci Data 2025, 12, 1816. DOI: 10.1038/s41597-025-06100-7

[39] Fu C, Qian F, Su K, Su Y, Wang Z, Shi J, et al. HiMul-LGG: A hierarchical decision fusion-based local-global graph neural network for multimodal emotion recognition in conversation. Neural Netw. 2025, 181, 106764. DOI: 10.1016/j.neunet.2024.106764

[40] Zhang YT, Wang XY, Guo JQ, Liu ZX. Multimodal sentiment analysis of book reviews based on decision-level fusion. J. Libr. Inf. Sci. 2025, 10, 72-79. Available online: https://d.wanfangdata.com.cn/periodical/kjqbkfyjj202505011 (accessed on 27 July 2026).

[41] Cai H, Qu Z, Li Z, Zhang Y, Hu X,Hu B. Feature-level fusion approaches based on multimodal EEG data for depression recognition. Inf. Fusion 2020, 59, 127-138. DOI: 10.1016/j.inffus.2020.01.008

[42] Rabbani MHR, Islam SMR. Deep learning networks based decision fusion model of EEG and fNIRS for classification of cognitive tasks. Cogn. Neurodyn. 2024, 18, 1489-1506. DOI: 10.1007/s11571-023-09986-4

[43] Kang RL, Li YR, Shi WX, Li JX. Decision-level fusion method for EEG-fNIRS motor imagery classification based on Dempster-Shafer evidential reasoning. Acta Electron Sin. 2025, 53, 941-950. Available online: https://link.cnki.net/urlid/11.2087.TN.20250508.1349.010 (accessed on 27 July 2026).

[44] Li Y, Zhu L, Huang A, Zhang J, Yuan P. Multimodal MBC-ATT: Cross-modality attentional fusion of EEG-fNIRS for cognitive state decoding. Front. Hum. Neurosci. 2025, 19, 1660532. DOI: 10.3389/fnhum.2025.1660532

[45] Wang H, Huang X, Xu L, Guo S, Gong C, Mengcheng C, et al. Analysis of the analgesic mechanism of TENS-WAA in colonoscopy using the EEG-fNIRS system: A study protocol for a randomised controlled trial. BMJ Open 2025, 15, e103681. DOI: 10.1136/bmjopen-2025-103681

[46] Berger A, Horst F, Müller S, Steinberg F, Doppelmayr M . Current State and Future Prospects of EEG and fNIRS in Robot-Assisted Gait Rehabilitation: A Brief Review. Front. Hum. Neurosci. 2019, 13, 172. DOI: 10.3389/fnhum.2019.00172

[47] Wu F, Liu Y, Jiang H, Li L, Feng C, Sun J, et al. Integrated EEG-fNIRS for Characterizing Cortical Responses and Neurovascular Coupling in Automated and Discrete Gait Tasks. IEEE Trans. Neural Syst. Rehabil. Eng. 2025, 33, 3805-3814. DOI: 10.1109/TNSRE.2025.3610690

[48] Moreno-Castelblanco SR, Vélez-Guerrero MA, Callejas-Cuervo M . Lower-Limb Motor Imagery Recognition Prototype Based on EEG Acquisition, Filtering, and Machine Learning-Based Pattern Detection. Sensors 2025, 25, 6387. DOI: 10.3390/s25206387

[49] Atilla F, Postma M, Alimardani M . Gamification of motor imagery brain-computer interface training protocols: A systematic review. Comput. Hum. Behav. Rep. 2024, 16, 100508. DOI: 10.1016/j.chbr.2024.100508

[50] Gong J, Liu H, Zhao L, Maeda T, Cao J. EEG-Powered UAV Control via Attention Mechanisms. Appl. Sci. 2025, 15, 10714. DOI: 10.3390/app151910714

[51] Du M, Chen L, Li Y, Xia L, Liu Y, Guo M, et al. Effect of exercise-based interventions on stroke rehabilitation: An umbrella review of systematic reviews and meta-analyses. J. NeuroEng. Rehabil. 2025, 22, 240. DOI: 10.1186/s12984-025-01781-y

[52] Slama K, Yahyaouy A, Riffi J, Mahraz MA, Tairi H . Comprehensive review of machine learning and deep learning techniques for epileptic seizure detection and prediction based on neuroimaging modalities. Vis. Comput. Ind. Biomed. Art 2025, 8, 27. DOI: 10.1186/s42492-025-00208-8

[53] Zhang T, Li CL, Meng L, Wang H, Su YC. Review of multimodal data fusion. Henan Sci. Technol. 2025, 52, 37-42. DOI: 10.19968/j.cnki.hnkj.1003-5168.2025.20.008

[54] Hasan MAH, Khan MU, Mishra D . A computationally efficient method for hybrid EEG-fNIRS BCI based on the Pearson correlation. BioMed Res. Int. 2020, 2020, 1838140. DOI: 10.1155/2020/1838140

[55] Meng M, Dai LY, Ma YL, Gao YY. Multimodal EEG-fNIRS channel selection method based on PF coefficient. Chin. J. Sens. Actuators. 2022, 35, 84-91. Available online: https://chinatransducers.seu.edu.cn/ch/reader/view_abstract.aspx?flag=1&file_no=cg210208&journal_id=cgjsxb (accessed on 27 July 2026).

[56] Qiu L, Zhong Y, He Z, Pan J. Improved classification performance of EEG-fNIRS multimodal brain-computer interface based on multi-domain features and multi-level progressive learning. Front. Hum. Neurosci. 2022, 16, 973959. DOI: 10.3389/fnhum.2022.973959

[57] Tang X, Yang C, Sun X, Zou M, Wang H. Motor Imagery EEG Decoding Based on Multi-Scale Hybrid Networks and Feature Enhancement. IEEE Trans. Neural Syst. Rehabil. Eng. 2023, 31, 1208-1218. DOI: 10.1109/TNSRE.2023.3242280

[58] Liu HD, Xu BJ, Li MQ. EEG-fNIRS feature fusion based on PCA. Mod. Electron. Technol. 2023, 46, 29-33. DOI: 10.16652/j.issn.1004-373x.2023.19.006

[59] Yang JC.Research on Decoding of Upper Limb Motor Parameters and Neurofeedback Based on VR Cues. Master’s Thesis, Kunming University of Science and Technology, Kunming, China, 2023.

[60] Mateen N, Naeem M, Khan MJ, Yousaf T, Ali A, Altabey WA, et al. Subject based feature selection for hybrid brain computer interface using genetic algorithm and support vector machine. Results Eng. 2025, 27, 105649. DOI: 10.1016/j.rineng.2025.105649

[61] Dipietro L, Eden U, Teixeira P, Tirasawasdichai N, Warinpramote J, Pundik S, et al. A multimodal machine learning approach to predict Fugl-Meyer scores and motor recovery potential in stroke rehabilitation: Toward precision-based therapies. Inf. Sci. 2025, 721, 122564. DOI: 10.1016/j.ins.2025.122564

[62] Huang X, Meng H, Li Z. Deep learning for epileptic seizure prediction from EEG signals: A review. Biomed. Signal Process. Control 2026, 117, 109518. DOI: 10.1016/j.bspc.2026.109518

[63] Zheng G, Wang H, Chai X, Xin X, Li F, Li H,et al.Interpretable deep learning for multicenter gastric cancer T staging from CT images. NPJ Digit. Med. 2025, 9, 2. DOI: 10.1038/s41746-025-02002-5

[64] Ancilin J, Lyernisha SR . PV2ES-BiRTM: Variational auto encoder sampling-based deep learning framework for emotion recognition from speech signals. Knowl. Based Syst. 2026, 332, 114885. DOI: 10.1016/j.knosys.2025.114885

[65] Baig MAA, Ratyal NI, Amin A, Jamil U, Khalid HM, Zia MF . A multi-modal deep learning framework for power quality disturbance classification: An integration of 1D time-series signals and 2D scalograms. Comput. Electr. Eng. 2025, 128, 110716. DOI: 10.1016/j.compeleceng.2025.110716

[66] Cooney C, Folli R, Coyle D. A Bimodal Deep Learning Architecture for EEG-fNIRS Decoding of Overt and Imagined Speech. IEEE Trans. Biomed. Eng. 2022, 69, 1983-1994. DOI: 10.1109/TBME.2021.3132861

[67] Mughal NE, Khan MJ, Khalil K, Javed K, Sajid H, Naseer N, et al. EEG-fNIRS-based hybrid image construction and classification using CNN-LSTM. Front. Neurorobot. 2022, 16, 873239. DOI: 10.3389/fnbot.2022.873239

[68] Liu JR, Song T, Shu ZL, Han JD, Yu NB. An EEG-fNIRS time-frequency feature fusion and collaborative classification method for motor decoding. Chin. J. Sci. Instrum. 2022, 43, 165-173. DOI: 10.19650/j.cnki.cjsi.J2209389

[69] Nour M, Öztürk Ş, Polat K . A novel classification framework using multiple bandwidth method with optimized CNN for brain-computer interfaces with EEG-fNIRS signals. Neural Comput. Applic. 2021, 33, 15815-15829. DOI: 10.1007/s00521-021-06202-4

[70] Arif A, Wang Y, Yin R, Zhang X, Helmy A . EF-Net: Mental State Recognition by Analyzing Multimodal EEG-fNIRS via CNN. Sensors 2024, 24, 1889. DOI: 10.3390/s24061889

[71] Bunterngchit C, Wang J, Su J, Wang Y, Liu S, Hou ZG. Temporal attention fusion network with custom loss function for EEG-fNIRS classification. J. Neural Eng. 2024, 21, 066016. DOI: 10.1088/1741-2552/ad8e86

[72] Jung E, An J. EFRM: A Multimodal EEG-fNIRS Representation-learning Model for few-shot brain-signal classification. Comput. Biol. Med. 2025, 199, 111292. DOI: 10.1016/j.compbiomed.2025.111292

[73] Yuan Z, Li Y, Zhang H, Liu X, Li S, Zhu Y, et al. Decoding Hybrid EEG-fNIRS Upper Limb Motor Execution with Capsule Dynamic Graph Convolutional Neural Network. In Proceedings of the 2025 47th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Copenhagen, Denmark, 14-18 July 2025; pp. 1-7. DOI: 10.1109/EMBC58623.2025.11251587

[74] Bunterngchit C, Baniata LH, Baniata MH, ALDabbas A, Khair MA, Chearanai T, et al. GACL-Net: Hybrid Deep Learning Framework for Accurate Motor Imagery Classification in Stroke Rehabilitation. Comput. Mater. Contin. 2025, 83, 517-536. DOI: 10.32604/cmc.2025.060368

[75] Barón Díaz D, Drommershausen A, Grünberger A, Holtmann D . Transfer Learning Approaches in Bioprocess Engineering: Opportunities and Challenges. Biotechnol. Bioeng. 2026, 123, 1417-1431. DOI: 10.1002/bit.70186

[76] Yu B, Zhang L. A Multi-Constrained Transfer Learning for Cross-Subject Decoding of Motor Imagery-Based BCI. Mathematics 2026, 14, 1314. DOI: 10.3390/math14081314

[77] Liu FD. Research on Classification of Motor Imagery EEG Signals Based on Deep Transfer Learning. Master’s Thesis, Changchun University of Technology, Changchun, China, 2025.

[78] Balsabti SM, Al-Gburi RM, gaib R, Mustafa A, Ahmed SK, Issa AM, et al. Advances in deep learning for multimodal brain imaging: A comprehensive survey. Neurosci. Inform. 2026, 6, 100252. DOI: 10.1016/j.neuri.2025.100252

[79] Sun X, Dai C, Wu X, Han T, Li Q, Lu Y, et al. Current implications of EEG and fNIRS as functional neuroimaging techniques for motor recovery after stroke. Med. Rev. 2024, 4, 492-509. DOI: 10.1515/mr-2024-0010

[80] Pan YD, Kang RL, Li YR. Cross-subject classification method based on EEG-fNIRS fusion. J. Nanjing Univ. Inf. Sci. Technol. 2025, 17, 1-13. DOI: 10.13878/j.cnki.jnuist.20250510001