Multiband decomposition and spectral discriminative analysis for motor imagery BCI via deep neural network

Abstract

Human limb movement imagery, which can be used in limb neural disorders rehabilitation and brain-controlled external devices, has become a significant control paradigm in the domain of brain-computer interface (BCI). Although numerous pioneering studies have been devoted to motor imagery classification based on electroencephalography (EEG) signal, their performance is somewhat limited due to insufficient analysis of key effective frequency bands of EEG signals. In this paper, we propose a model of multiband decomposition and spectral discriminative analysis for motor imagery classification, which is called variational sample-long short term memory (VS-LSTM) network. Specifically, we first use a channel fusion operator to reduce the signal channels of the raw EEG signal. Then, we use the variational mode decomposition (VMD) model to decompose the EEG signal into six band-limited intrinsic mode functions (BIMFs) for further signal noise reduction. In order to select discriminative frequency bands, we calculate the sample entropy (SampEn) value of each frequency band and select the maximum value. Finally, to predict the classification of motor imagery, a LSTM model is used to predict the class of frequency band with the largest SampEn value. An open-access public data is used to evaluated the effectiveness of the proposed model. In the data, 15 subjects performed motor imagery tasks with elbow flexion / extension, forearm supination / pronation and hand open/close of right upper limb. The experiment results show that the average classification result of seven kinds of motor imagery was 76.2%, the average accuracy of motor imagery binary classification is 96.6% (imagery vs. rest), respectively, which outperforms the state-of-the-art deep learning-based models. This framework significantly improves the accuracy of motor imagery by selecting effective frequency bands. This research is very meaningful for BCIs, and it is inspiring for end-to-end learning research.

Graphical abstract

Keywords

brain computer interface / EEG / long short-term memory / VMD / sample entropy / motor imagery

References

[1] Li Y , Pan J , Wang F , Yu Z . A hybrid BCI system combining P300 and SSVEP and its application to wheelchair control. IEEE Transactions on Biomedical Engineering, 2013, 60( 11): 3156– 3166

[2] Nicolas L F , Gomez J . Brain computer interfaces, a review. Sensors, 2012, 12 : 1211– 1279

[3] Wolpaw J R , Bedlack R S , Reda D J , Ringer R J , Banks P G , Vaughan T M , Heckman S M , McCane L M , Carmack C S , Winden S , McFarland D J , Sellers E W , Shi H R , Paine T , Higgins D S , Lo A C , Patwa H S , Hill K J , Huang G D , Ruff R L . Independent home use of a brain-computer interface by people with amyotrophic lateral sclerosis. Neurology, 2018, 91( 3): 258– 267

[4] Delisle D , Cardoso V , Gurve D , Loterio F , Romero-Laiseca M A , Krishnan S , Bastos T . System based on subject-specific bands to recognize pedaling motor imagery: towards a BCI for lower-limb rehabilitation. Journal of Neural Engineering, 2019, 16( 5): 056005–

[5] Wilmskoetter J , Gaizo J D , Phillip L , Behroozmand R , Gleichgerrcht E , Fridriksson J , Riley E , Bonilha L . Predicting naming responses based on pre-articulatory electrical activity in individuals with aphasia. Clinical Neurophysiology, 2019, 130( 11): 2153– 2163

[6] Yang C , Zhang H , Zhang S , Han X , Gao S , Gao X . The spatio-temporal equalization for evoked or event-related potential detection in multichannel EEG data. IEEE Transaction on Biomedical Engineering, 2020, 67 : 2397– 2414

[7] Xu M , Xiao X , Wang Y , Qi H , Jung T P , Ming D . A brain-computer interface based on miniature-event-related potentials induced by very small lateral visual stimuli. IEEE Transaction on Biomedical Engineering, 2018, 65 : 1166– 1175

[8] Ramele R , Villar A J , Santos J M . EEG waveform analysis of p300 ERP with applications to brain computer interfaces. Brain Sciences, 2018, 8( 11): 199–

[9] Nakanishi M , Wang Y , Chen X , Wang Y T , Gao X , Jung T P . Enhancing detection of SSVEPs for a high-speed brain speller using task-related component analysis. IEEE Transaction on Biomedical Engineering, 2018, 65 : 104– 112

[10] Nakanishi M , Wang Y , Chen X , Wang Y T , Gao X , Jung T P . Enhancing detection of SSVEPs for a high-speed brain speller using task-related component analysis. IEEE Transaction on Biomedical Engineering, 2018, 65( 1): 104– 112

[11] Han X , Lin K , Gao S , Gao X . A novel system of SSVEP-based human-robot coordination. Journal of Neural Engineering, 2019, 16( 1): 016006–

[12] Azab A M , Mihaylova L , Ang K K , Arvaneh M . Weighted transfer learning for improving motor imagery-based brain-computer interface. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27 : 1352– 1359

[13] Jiao Y , Zhang Y , Chen X , Yin E , Jin J , Wang X , Cichocki A . Sparse group representation model for motor imagery EEG classification. IEEE Journal of Biomedical and Health Informatics, 2019, 23 : 631– 641

[14] Zhang Y , Wang Y , Jin J , Wang X . Sparse Bayesian learning for obtaining sparsity of EEG frequency bands based feature vectors in motor imagery classification. International Journal of Neural Systems, 2017, 27( 2): 1650032–

[15] Tabar Y R , Halici U . A novel deep learning approach for classification of EEG motor imagery signal. Journal of Neural Engineering, 2017, 14( 1): 016003–

[16] Edelman B J , Baxter B , He B . EEG source imaging enhances the decoding of complex right-hand motor imagery tasks. IEEE Transaction on Biomedical Engineering, 2016, 63 : 4– 14

[17] Quiles E , Ferran S , Candela G . Low-cost robotic guide based on a motor imagery brain-computer interface for arm assisted rehabilitation. International Journal of Environmental Research and Public Health, 2020, 17( 3): 699–

[18] Mane R , Chew E , Phua K S , Kai K A , Guan C . Prognostic and monitory EEG-biomarkers for BCI upper-limb stroke rehabilitation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27 : 1654– 1664

[19] Nourmohammadi A , Jafari M , Zander T . A survey on unmanned aerial vehicle remote control using brain-computer Interface. IEEE Transaction on Human-Machine Systems, 2018, 48( 4): 337– 348

[20] Jafarifarmand A , Badamchizadeh M. A . EEG artifacts handling in a real practical brain-computer interface controlled vehicle. IEEE Transaciton on Neural Systems and Rehabilitation Engineering, 2019, 27( 6): 1200– 1208

[21] Edelman B J , Meng J , Suma D , Suma C , Zurn E . Noninvasive neuroimaging enhances continuous neural tracking for robotic device control. Science Robotics, 2019, 4(31): eaaw6844 :

[22] Su Y , Li W , Bi N , Lv Z . Adolescents environmental emotion perception by integrating EEG and eye movements. Frontiers in Neurorobotics, 2019, 13 : 46–

[23] Klimesch W . Alpha-band oscillations, attention, and controlled access to stored information. Trends in Cognitive Sciences, 2012, 16 : 606– 617

[24] Bartur G , Pratt H , Soroker N . Changes in mu and beta amplitude of the EEG during upper limb movement correlate with motor impairment and structural damage in subacute stroke. Clinical Neurophysiology, 2019, 130 : 1644– 1651

[25] Uji M , Wilson R , Francis S T , Mullinger K J , Mayhew S D . Exploring the advantages of multiband fMRI with simultaneous EEG to investigate coupling between gamma frequency neural activity and the BOLD response in humans. Human Brain Mapping, 2018, 39 : 1673– 1687

[26] Milz P , Pascual-Marqui R D , Achermann P , Kochi K , Faber P L . The EEG microstate topography is predominantly determined by intracortical sources in the alpha band. Neuroimage, 2017, 162 : 353– 361

[27] Zhao Y , Zhao Y , Pholpat D , Chen L , Sarrigiannis P G . Imaging of nonlinear and dynamic functional brain connectivity based on EEG recordings with the application on the diagnosis of alzheimer's disease. IEEE Transaction on Medical Imaging, 2020, 39 : 1571– 1581

[28] Li Q , Şentürk D , Sugar C A , Jeste S , DiStefano C , Frohlich J , Telesca D . Inferring brain signals synchronicity from a sample of EEG readings. Journal of the American Statistical Association, 2019, 114 : 991– 1001

[29] Jiang J , Yan Z , Shen T , Xu G , Guan Q , Yu Z . Use of deep belief network model to discriminate mild cognitive impairment and normal controls based on EEG, eye movement signals and neuropsychological tests. Journal of Medical Imaging and Health Informatics, 2019, 9 : 1978– 1985

[30] Chen X , Chen Q , Zhang Y , Wang Z J . A novel EEMD-CCA approach to removing muscle artifacts for pervasive EEG. IEEE Sensors Journal, 2019, 19 : 8420– 8431

[31] Zhang D , Yao L , Chen K , Wang S , Chang X , Liu Y . Making sense of Spatio-temporal preserving representations for EEG-based human intention recognition. IEEE Transaction on Cybernetics, 2019, 50 : 3033– 3044

[32] Aline D , Delorme R , Delanoë C , Amsellem F , Beggiato A , Germanaud D , Bourgeron T , Toro R , Dumas G . Alpha waves as a neuromarker of autism spectrum disorder: the challenge of reproducibility and heterogeneity. Frontiers in Neuroscience, 2018, 12 : 662–

[33] Su Y , Chen P , Liu X , Li W , Lv Z . A spatial filtering approach to environmental emotion perception based on electroencephalography. Medical Engineering & Physics, 2018, 60 : 77– 85

[34] Ghaemi A , Rashedi E , Pourrahimi A M , Kamandar M , Rahdari F . Automatic channel selection in EEG signals for classification of left or right hand movement in Brain Computer Interfaces using improved binary gravitation search algorithm. Biomedical Signal Processing and Control, 2017, 33 : 109– 118

[35] Lee H K , Choi Y S . Application of continuous wavelet transform and convolutional neural network in decoding motor imagery brain-computer interface. Entropy, 2019, 21( 12): 1199–

[36] Zeng W , Li M , Yuan C , Wang Q , Liu F , Wang Y . Identification of epileptic seizures in EEG signals using time-scale decomposition (ITD), discrete wavelet transform (DWT), phase space reconstruction (PSR) and neural networks. Artificial Intelligence Review, 2020, 53 : 3059– 3088

[37] Oh S H . Improving the subject independent classification of implicit intention by generating additional training data with PCA and ICA. International Journal of Contents, 2018, 14 : 24– 29

[38] Jin J , Xiao R , Daly I , Miao Y , Cichocki A . Internal feature selection method of CSP based on L1-norm and dempster-shafer theory. IEEE Transactions on Neural Networks and Learning Systems, 2020,

[39] Sachin B T , Sharma V , Siuly D , Sengur S . Features based on analytic IMF for classifying motor imagery EEG signals in BCI applications. Measurement, 2018, 116 : 68– 76

[40] Balaiah P . Comparative evaluation of adaptive filter and neuro-fuzzy filter in artifacts removal from electroencephalogram signal. American Journal of Applied Sciences, 2012, 9( 10): 1583– 1593

[41] Ameri R , Pouyan A , Abolghasemi V . Projective dictionary pair learning for EEG signal classification in brain computer interface applications. Neurocomputing, 2016, 218 : 382– 389

[42] Ma J , Zhang D . EEG signals feature extraction based on DWT and EMD combined with approximate entropy. Brain Sciences, 2019, 9( 8): 201–

[43] Jin J , Miao Y , Daly I , Zuo C , Cichocki A . Correlation-based channel selection and regularized feature optimization for MI-based BCI. Neural Networks, 2019, 118 : 262– 270

[44] Zhu X , Li P , Li C . Separated channel convolutional neural network to realize the training free motor imagery BCI systems. Biomedical Signal Processing and Control, 2019, 49 : 396– 403

[45] Alchalabi B , Faubert J . A comparison between BCI simulation and neurofeedback for forward/backward navigation in virtual reality. Computational Intelligence and Neuroscience, 2019, 4 : 1– 12

[46] Aljal M , Ridha D , Sutrisno I . Robot navigation using a brain computer interface based on motor imagery. Journal of Medical and Biological Engineering, 2019, 39( 4): 508– 522

[47] Huang L , Zhao Y , Zeng Y , Lin Z . BHCR: RSVP target retrieval BCI framework coupling with CNN by a Bayesian method. Neurocomputing, 2017, 238 : 255– 268

[48] Zhang Y , Zhou T , Wu W , Xie H , Cichocki A . Improving EEG decoding via clustering-based multi-task feature learning. IEEE Transactions on Neural Networks and Learning Systems, 2021,

[49] Jiao Y , Zhou T , Yao L , Zhou G , Zhang Y . Multi-view multi-scale optimization of feature representation for EEG classification improvement. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28( 12): 2589– 2597

[50] Zhang X, Yao L, Wang X, Monaghan J, Mcalpine D, Zhang Y. A survey on deep learning-based non-invasive brain signals: recent advances and new frontiers. Journal of Neural Engineering, 2021, 18(3): 1−44

[51] Craik A , He Y , Jose L . Contreras-vidal deep learning for electroencephalogram (EEG) classification tasks: a review. Journal of Neural Engineering, 2019, 16( 3): 031001–

[52] Wang M , Lian C , Yao D. Shen D . Spatial-temporal dependency modeling and network hub detection for functional MRI analysis via convolutional-recurrent network. IEEE Transactions on Biomedical Engineering, 2020, 67( 8): 2241– 2252

[53] Acharya U , Oh S L , Hagiwara Y , Tan J H , Adeli H . Deep convolutional neural network for the automated detection and diagnosis of seizure using EEG signals. Computers in Biology and Medicine, 2018, 100 : 270– 278

[54] Hou Y , Zhou L , Jia S , Lun X . A novel approach of decoding EEG four-class motor imagery tasks via scout ESI and CNN. Journal of Neural Engineering, 2019, 17( 1): 016048–

[55] Chen J , Yu Z , Gu Z , Li Y . Deep temporal-spatial feature learning for motor imagery-based brain-computer interfaces. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28( 11): 2356– 2366

[56] Li Y , Zhang X R , Zhang B , Lei M Y , Cui W G , Guo Y Z . A channel-projection mixed-scale convolutional neural network for motor imagery EEG decoding. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27( 6): 1170– 1180

[57] Ma X , Qiu S , Wei W , Wang S , He H . Deep channel-correlation network for motor imagery decoding from the same limb. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28( 1): 297– 306

[58] Luo T J , Zhou C L , Luo F . Exploring spatial-frequency-sequential relationships for motor imagery classification with recurrent neural network. BMC Bioinformatics, 2018, 19(1): 1−18 :

[59] Wang P , Jiang A , Liu X , Shang J , Zhang L . LSTM-based EEG classification in motor imagery tasks. IEEE Transactions Neural Systems and Rehabilitation Engineering, 2018, 26( 11): 2086– 2095

[60] Tayeb Z , Fedjaev J , Ghaboosi N , Richter C , Everding L , Qu X , Wu Y , Cheng G , Conradt J . Validating deep neural networks for online decoding of motor imagery movements from EEG signals. Sensors, 2019, 19( 1): 210–

[61] Ofner P , Schwarz A , Pereira J , Müller-Putz G R . Upper limb movements can be decoded from the time-domain of low-frequency EEG. PLoS ONE, 2017, 12( 8): e0182578–

[62] Tangermann M , Müller K R , Aertsen A , Birbaumer N , Blankertz B . Review of the BCI competition IV. Frontiers on Neuroscience, 2012, 6 : 55–

[63] Rehman N U , Aftab H . Multivariate variational mode decomposition. IEEE Transaction on Signal Processing, 2019, 67 : 6039– 6052

[64] Cheng Q , Yang W , Liu K , Zhao W , Yang Y . Increased sample entropy in EEGs during the functional rehabilitation of an injured brain. Entropy, 2019, 21( 7): 698–

[65] Martinez C V , Santamaria V E , Hornero R . Asynchronous control of P300-based brain-computer interfaces using sample entropy. Entropy, 2019, 21( 3): 230–

[66] Wang M , Zhang D , Huang J , Yap P T , Liu M . Identifying autism spectrum disorder with multi-site fMRI via low-rank domain adaptation. IEEE Transactions on Medical Imaging, 2020, 39( 3): 644– 655

[67] Fan J H , Sun C L , Chen C , Chen J , Xinyu Liu X , Zhao X , Meng L , Dai C , Chen W . EEG data augmentation: towards class imbalance problem in sleep staging tasks. Journal of Neural Engineering, 2020, 17( 5): 056017–

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (Grant Nos. 61876082, 61861130366, 61732006) and National Key R&D Program of China (2018YFC2001600, 2018YFC2001602).

RIGHTS & PERMISSIONS

2022 Higher Education Press