Electroencephalography decoding model based on fusion deep graph convolutional neural network for spinal cord injury

Abstract

Background:Electroencephalography (EEG) signals can be used to measure neuronal activity in different regions of the brain through electrodes.
Objective:To enhance the decoding of motor imagery (MI) EEG signals in spinal cord injury (SCI) patients, this study proposes a feature fusion graph convolutional neural network (F-GCN) model that integrates wavelet-based time-frequency features and functional topological relationships among EEG electrodes, aiming to improve classification accuracy and provide guidance for rehabilitation.
Study design:This study included 10 patients with spinal cord injuries as the experimental group, and 10 healthy individuals as the control group. After the experiment began, the subjects underwent 2-min recordings of their EEG signals in resting states with eyes open or closed, with records for each state repeated twice. The participants were then asked to imagine the movements of their left hand, and right hand. The entire process of MI consists of four task stages, with each stage containing three tasks. Each task randomly appears 10 times.
Methods:Time-frequency features of MI-EEG signals were extracted using a continuous wavelet transform to enhance the effectiveness of decoding raw EEG signals. Functional and statistical analyses of brain regions during MI were conducted based on the extracted time-frequency features. Based on this, the motor intentions of patients with SCI were decoded using a GCN that integrates the functional topological relationships of the electrodes.
Results:The proposed network achieved a classification accuracy of 92.44 % for MI task recognition. Furthermore, the fusion of wavelet features demonstrated superior performance in classification and recognition.
Conclusions:The results of this study confirm the efficacy of wavelet fusion in advancing MI feature decoding, enhancing the understanding of neurological conditions, such as SCI, and offering promising prospects for improving rehabilitation methods.

Keywords

Spinal cord injury / Electroencephalogram / Motor imagery / Fusion wavelet / Combined graph convolution network

CRediT authorship contribution statement

Yang Zhang: Writing -review & editing, Supervision, Project administration, Funding acquisition, Conceptualization. Fangzhou Xu: Writing -review & editing, Supervision, Project administration, Funding acquisition, Conceptualization. Tzyy-Ping Jung: Writing -review & editing, Supervision, Methodology. Tianwei Lou: Writing -original draft, Validation, Methodology, Investigation, Formal analysis. Xinting Zhang: Writing -original draft, Validation, Methodology, Investigation, Formal analysis. Lei Jiang: Validation, Investigation, Data curation. Lei Chen: Validation, Investigation, Data curation. Licai Gao: Software, Data curation. Zhixiao Lun: Software, Data curation. Jincheng Li: Formal analysis, Data curation. All authors have read and approved the final version of this manuscript.

Ethics approval

This research was approved by the Ethics Committee of Qilu Hospital, Shandong University, under protocol number KYLL-2020(KS)−475. Written informed consent was obtained from all the participants.

Funding

The authors declare financial support was received for the research, authorship, and/or publication of this article. The project is supported in part by the Fundamental Research Funds for the Central Universities [grant number 2022JC013], the Natural Science Foundation of Shandong Province of China [grant number ZR2021MH023], the Program for Youth Innovative Research Team in the University of Shandong Province in China [grant number 2019KJN010], the Introduce Innovative Teams of 2021 “New High School 20 Items” Project [grant number 2021GXRC071], the Clinical Research Cross-Project of Shandong University [grant number 2020SDUCRCB004], the National Natural Science Foundation of China [grant number 82172535], the Key Program of the National Natural Science Foundation of China [grant number 82330064], the Natural Science Foundation of Shandong Province [grant number ZR2022MF289], the National Natural Science Foundation of China [grant number 62271293], the Graduate Education and Teaching Reform Project of Qilu University of Technology

Data availability

The data underlying this article will be shared upon reasonable request from the corresponding authors.

Declaration of Competing Interest

The authors declare that they have no competing interests.

Acknowledgments

The authors give thanks to all the spinal cord injury patients for their participation in the experiment.

References

[1] Fouad K, Popovich PG, Kopp MA, Schwab JM. The neuroanatomical-functional paradox in spinal cord injury. Nat Rev Neurol. 2020; 17(1):53-62. https://doi.org/10.1038/s41582-020-00436-x

[2] Calderone A, Cardile D, De Luca R, Quartarone A, Corallo F, Calabrò RS. Brain plasticity in patients with spinal cord injuries: a systematic review. Int J Mol Sci. 2024; 25(4):2224. https://doi.org/10.3390/ijms25042224

[3] Di X, Biswal BB. Dynamic brain functional connectivity modulated by resting-state networks. Brain Struct Funct. 2015; 220(1):37-46. https://doi.org/10.1007/s00429-013-0634-3

[4] Kirshblum S, Snider B, Eren F, Guest J. Characterizing natural recovery after traumatic spinal cord injury. J Neurotrauma. 2021; 38(9):1267-1284. https://doi.org/10.1089/neu.2020.7473

[5] Inanici F, Brighton LN, Samejima S, Hofstetter CP, Moritz CT. Transcutaneous spinal cord stimulation restores hand and arm function after spinal cord injury. IEEE Trans Neural Syst Rehabil Eng. 2021; 29:310-319. https://doi.org/10.1109/tnsre.2021.3049133

[6] Moro V, Corbella M, Ionta S, Ferrari S, Scandola M. Cognitive Training Improves Disconnected Limbs’ Mental Representation and Peripersonal Space after Spinal Cord Injury. Int J Environ Res Public Health. 2021; 18(18):9589. https://doi.org/10.3390/ijerph18189589

[7] Herbert D, Tran Y, Craig A, Boord P, Middleton J, Siddall P. Altered brain wave activity in persons with chronic spinal cord injury. Int J Neurosci. 2007; 117(12):1731-1746. https://doi.org/10.1080/00207450701242826

[8] Hermosilla DM, Codorniú RT, Baracaldo RL, et al. Shallow convolutional network excel for classifying motor imagery EEG in BCI applications. IEEE Access. 2021; 9:98275-98286. https://doi.org/10.1109/ACCESS.2021.3091399

[9] Janapati R, Dalal V, Govardhan N, Sengupta R. Signal Processing Algorithms Based on Evolutionary Optimization Techniques in the BCI:A Review. In: SmysS, eds. Computational Vision and Bio-Inspired Computing. Springer, Singapore; Tavares J.M.R. S., Bestak R., Shi F., 2021:165-174. https://doi.org/10.1007/978-981-33-6862-0_14

[10] Omidvar M, Zahedi A, Bakhshi H. EEG signal processing for epilepsy seizure detection using 5-level Db4 discrete wavelet transform, GA-based feature selection and ANN/SVM classifiers. J Ambient Intell Humaniz Comput. 2021; 12(11):10395-10403. https://doi.org/10.1007/s12652-020-02837-8

[11] Yong X, Menon C. EEG classification of different imaginary movements within the same limb. PLoS One. 2015; 10(4):e0121896. https://doi.org/10.1371/journal.pone.0121896

[12] Mohseni Salehi SS, Moghadamfalahi M, Quivira F, Piers A, Nezamfar H, Erdogmus D.Decoding complex imagery hand gestures. In:Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine Biology Society (EMBC); 2017:2968-2971. https://doi.org/10.1109/embc.2017.8037480

[13] Zhang X, Yong X, Menon C. Evaluating the versatility of EEG models generated from motor imagery tasks: An exploratory investigation on upper-limb elbow-centered motor imagery tasks. PLoS One. 2017; 12(11):e0188293. https://doi.org/10.1371/journal.pone.0188293

[14] Suwannarat A, Pan-Ngum S, Israsena P. Comparison of EEG measurement of upper limb movement in motor imagery training system. Biomed Eng Online. 2018; 17(1):103. https://doi.org/10.1186/s12938-018-0534-0

[15] Chen Z, Wang Z, Wang K, Yi W, Qi H. Recognizing Motor Imagery Between Hand and Forearm in the Same Limb in a Hybrid Brain Computer Interface Paradigm: An Online Study. IEEE Access. 2019; 7:59631-59639. https://doi.org/10.1109/ACCESS.2019.2915614

[16] Ma X, Qiu S, He H. Multi-channel EEG recording during motor imagery of different joints from the same limb. Sci Data. 2020; 7(1):191. https://doi.org/10.1038/s41597-020-0535-2

[17] Willett FR, Avansino DT, Hochberg LR, Henderson JM, Shenoy KV.High-performance brain-to-text communication via handwriting. Nature. 2021; 593(7858):249-254. https://doi.org/10.1038/s41586-021-03506-2

[18] Sreeja SR, Samanta D. Classification of multiclass motor imagery EEG signal using sparsity approach. Neurocomputing (Amst). 2019; 368:133-145. https://doi.org/10.1016/j.neucom.2019.08.037

[19] Lu Z, Lu X, Yang R, Chang S. Research on Motor Imagery EEG Signal Classification on Multi-Features Fusion. SmartTech Innovations. 2019(3):3-7. https://doi.org/10.3969/j.issn.1007-1423.2019.03.001

[20] Ha KW, Jeong JW. Motor imagery EEG classification using capsule networks. Sensors (Basel). 2019; 19(13):2854. https://doi.org/10.3390/s19132854

[21] Zou X, Zhang Y, Sun Y. A method for extraction of motor imagery EEG features based on local mean decomposition and multiscale entropy. Chinese High Technology Letters. 2018; 28(1):22-28. https://doi.org/10.3772/j.issn.1002-0470.2018.01.004

[22] Schirrmeister RT, Springenberg JT, Fiederer LDJ, et al. Deep learning with convolutional neural networks for EEG decoding and visualization. Hum Brain Mapp. 2017; 38(11):5391-5420. https://doi.org/10.1002/hbm.23730

[23] Lawhern VJ, Solon AJ, Waytowich NR, Gordon SM, Hung CP, Lance BJ. EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces. J Neural Eng. 2018; 15(5):056013. https://doi.org/10.1088/1741-2552/aace8c

[24] Zhang D, Yao L, Zhang X, et al.Cascade and Parallel Convolutional Recurrent Neural Networks on EEG-based Intention Recognition for Brain Computer Interface. In:Proceedings of the AAAI Conference on Artificial Intelligence. 2018; 32(1). https://doi.org/10.1609/aaai.v32i1.11496.

[25] Sakhavi S, Guan C, Yan S. Learning temporal information for brain-computer interface using convolutional neural networks. IEEE Trans Neural Netw Learn Syst. 2018; 29(11):5619-5629. https://doi.org/10.1109/tnnls.2018.2789927

[26] Ma X, Qiu S, Wei W, Wang S, He H. Deep channel-correlation network for motor imagery decoding from the same limb. IEEE Trans Neural Syst Rehabil Eng. 2020; 28(1):297-306. https://doi.org/10.1109/tnsre.2019.2953121

[27] Zhang Y, Qiu S, Wei W, Ma X, He H. Filter Bank Adversarial Domain Adaptation For Motor Imagery Brain Computer Interface. In: 2021 International Joint Conference on Neural Networks (IJCNN); 2021:1-7. https://doi.org/10.1109/IJCNN52387.2021.9534286

[28] Zhang K, Robinson N, Lee SW, Guan C. Adaptive transfer learning for EEG motor imagery classification with deep convolutional neural network. Neural Netw. 2021; 136:1-10. https://doi.org/10.1016/j.neunet.2020.12.013

[29] Di X, Biswal BB. Dynamic brain functional connectivity modulated by resting-state networks. Brain Struct Funct. 2015; 220(1):37-46. https://doi.org/10.1007/s00429-013-0634-3

[30] Pfurtscheller G. EEG event-related desynchronization (ERD) and event related synchronization (ERS). In: NiedermeyerE., eds. Electroencephalography:basic principles, clinical applications and related fields. 4th ed. Baltimore, MD: Williams and Wilkins; 1999: 958-967.

[31] Tariq M, Trivailo PM, Simic M. Mu-Beta event-related (de)synchronization and EEG classification of left-right foot dorsiflexion kinaesthetic motor imagery for BCI. PLoS One. 2020; 15(3):e0230184. https://doi.org/10.1371/journal.pone.0230184

[32] Wirawan IMA, Wardoyo R, Lelono D. The challenges of emotion recognition methods based on electroencephalogram signals: a literature review. Int J Electr Comput Eng. 2022; 12(2):1508-1519. https://doi.org/10.11591/ijece.v12i2.pp1508-1519

[33] Masum M, Shahriar H, Haddad HM, Song W.A statistical summary analysis of window-based extracted features for EEG signal classification. In:Proceedings of the 2021 IEEE International Conference on Digital Health (ICDH); 2021:293-298. https://doi.org/10.1109/ICDH52753.2021.00053

[34] Li B, Cheng T, Guo Z.A review of EEG acquisition, processing and application. J Phys Conf Ser. 2021;1907:012045. https://doi.org/10.1088/1742-6596/1907/1/012045

[35] Chiarion G, Sparacino L, Antonacci Y, Faes L, Mesin L. Connectivity analysis in EEG data: a tutorial review of the state of the art and emerging trends. Bioengineering (Basel). 2023; 10(3):372. https://doi.org/10.3390/bioengineering10030372

[36] Khosla A, Khandnor P, Chand T. A comparative analysis of signal processing and classification methods for different applications based on EEG signals. Biocybern Biomed Eng. 2020; 40(2):649-690. https://doi.org/10.1016/j.bbe.2020.02.002

[37] Mohamed EA, Yusoff MZ, Selman NK, Malik AS. Enhancing EEG signals in brain computer interface using wavelet transform. Int J Inf Electron Eng. 2014; 4(3):234-238. https://doi.org/10.7763/ijiee.2014.v4.440

[38] Subasi A, Tuncer T, Dogan S, Tanko D, Sakoglu U. EEG-based emotion recognition using tunable Q wavelet transform and rotation forest ensemble classifier. Biomed Signal Process Control. 2021;68:102648. https://doi.org/10.1016/j.bspc.2021.102648

[39] Gupta V, Pachori RB. Classification of focal EEG signals using FBSE based flexible time-frequency coverage wavelet transform. Biomed Signal Process Control. 2020;62:102124. https://doi.org/10.1016/j.bspc.2020.102124

[40] Xu F, Rong F, Miao Y, et al. Representation learning for motor imagery recognition with deep neural network. Electronics (Basel). 2021; 10(2):112. https://doi.org/10.3390/electronics10020112

[41] Katthi JR, Ganapathy S.Deep Multiway Canonical Correlation Analysis for Multi-Subject EEG Normalization. In: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 2021:1245-1249. https://doi.org/10.1109/ICASSP39728.2021.9414274

[42] Anila Glory H, Vigneswaran C, Shankar Sriram VS. Identification of Suitable Basis Wavelet Function for Epileptic Seizure Detection Using EEG Signals. In: First International Conference on Sustainable Technologies for Computational Intelligence; 2020:607-621. https://doi.org/10.1007/978-981-15-0029-9_48

[43] Cheng L, Li D, Li X, Yu S. The optimal wavelet basis function selection in feature extraction of motor imagery electroencephalogram based on wavelet packet transformation. IEEE Access. 2019; 7:174465-174481. https://doi.org/10.1109/ACCESS.2019.2953972

[44] Chiang W-L, Liu X, Si S, Li Y, Bengio S, Hsieh C-J. Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks. In:Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining; 2019:257-266. https://doi.org/10.1145/3292500.3330925

[45] Wang D, Lei C, Zhang X, et al. Identification of Depression with a Semi-supervised GCN based on EEG Data. In: 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM); 2021:2338-2345. https://doi.org/10.1109/BIBM52615.2021.9669572

[46] Zhang S, Chen D, Tang Y, Zhang L. Children ASD evaluation through joint analysis of EEG and eye-tracking recordings with graph convolution network. Front Hum Neurosci. 2021;15:651349. https://doi.org/10.3389/fnhum.2021.651349

[47] Lee J, Choi JW, Jo S. Subject-Independent Motor Imagery EEG Classification Based on Graph Convolutional Network. In: WallravenC, LiuQ, NagaharaH, eds. Pattern Recognition. ACPR 2021. Lecture Notes in Computer Science, vol 13189. Cham, Switzerland: Springer; 2022:268-281 https://doi.org/10.1007/978-3-031-02444-3_20

[48] Clifton L, Clifton DA. The correlation between baseline score and post-intervention score, and its implications for statistical analysis. Trials. 2019; 20(1):43. https://doi.org/10.1186/s13063-018-3108-3

[49] Xu F, Miao Y, Sun Y, et al. A transfer learning framework based on motor imagery rehabilitation for stroke. Sci Rep. 2021; 11(1):19783. https://doi.org/10.1038/s41598-021-99114-1

[50] Xu F, Rong F, Leng J, et al. Classification of left-versus right-hand motor imagery in stroke patients using supplementary data generated by cycleGAN. IEEE Trans Neural Syst Rehabil Eng. 2021; 29:2417-2424. https://doi.org/10.1109/tnsre.2021.3123969

[51] Keelawat P, Thammasan N, Kijsirikul B, Numao M. Subject-independent emotion recognition during music listening based on EEG using deep convolutional neural networks. In: 2019 IEEE 15th International Colloquium on Signal Processing & Its Applications (CSPA); 2019:21-26. https://doi.org/10.1109/CSPA.2019.8696054

[52] Molla MKI, Shiam AA, Islam MR, Tanaka T. Discriminative feature selection-based motor imagery classification using EEG signal. IEEE Access. 2020; 8:98255-98265. https://doi.org/10.1109/ACCESS.2020.2996685