Research on recognition of O-MI based on CNN combined with SST and LSTM

Abstract

Recognition algorithms have been widely used in brain computer interface (BCI) for neural paradigms classification. To improve the classification and recognition effect of motor imagery with motor observation (O-MI) in BCI rehabilitation technology, this paper explores the function of convolutional neural network (CNN) combined with synchrosqueezed wavelet transform (SST) and long short-term memory (LSTM) in the recognition and classification of neural activities in the brain motor area. Combining the advantages of SST in signal feature extraction in the pretreatment stage and the ability of LSTM network in time series information modeling, the purpose is to make up for CNN’s shortcomings in both aspects. This paper verifies the algorithm on the self-collected O-MI experimental datasets and the public datasets (BCI competition IV datasets 2a). The results show that the composite CNN algorithm incorporating SST and LSTM achieves higher classification accuracy than classic algorithms and the similar new method which is CNN combined with discrete wavelet transform (DWT) and power spectral density (PSD), so it is convenient for practical application in O-MI BCI system.

References

[1] ManeR, ChouhanT, GuanC. BCI for stroke rehabilitation: motor and beyond[J]. Journal of neural engineering, 2020, 17(4):041001

[2] KanekoN, SasakiA, YokoyamaH, et al.. Effects of action observation and motor imagery of walking on the corticospinal and spinal motoneuron excitability and motor imagery ability in healthy partici-pants[J]. Plos one, 2022, 17(4): e0266000

[3] StolbkovY K, GerasimenkoY P. Cognitive motor rehabilitation: imagination and observation of motor actions[J]. Human physiology, 2021, 47(1): 104-112

[4] StolbkovY K, GerasimenkoY P. Observation of motor actions as a tool for motor rehabilitation[J]. Neuroscience and behavioral physiology, 2021, 51(7):1018-1026

[5] TorrisiM, MaggioM G, ColaM, et al.. Beyond motor recovery after stroke: the role of hand robotic rehabilitation plus virtual reality in improving cognitive function[J]. Journal of clinical neuroscience, 2021, 92(9859):11-16

[6] DaubechiesI, LuJ, WuH T. Synchrosqueezed wavelet transforms: an empirical mode decomposition-like tool[J]. Applied and computational harmonic analysis, 2011, 30(2):243-261

[7] MandhoujB, CherniM A, SayadiM. An automated classification of EEG signals based on spectrogram and CNN for epilepsy diagnosis[J]. Analog integrated circuits and signal processing, 2021, 108(1):101-110

[8] OnoY, WadaK, KurataM, et al.. Enhancement of motor-imagery ability via combined action observation and motor-imagery training with proprioceptive neuro-feedback[J]. Neuropsychologia, 2018, 114(1):134-142

[9] XieJ, PengM, LuJ, et al.. Enhancement of event-related desynchronization in motor imagery based on transcranial electrical stimulation[J]. Frontiers in human neuroscience, 2021, 15(1):141-147

[10] NagaiH, TanakaT. Action observation of own hand movement enhances event-related desynchroniza-tion[J]. IEEE transactions on neural systems and rehabilitation engineering, 2019, 27(7):1407-1415

[11] KinoshitaT, FujiwaraK, KanoM, et al.. Sleep spindle detection using RUSBoost and synchrosqueezed wavelet transform[J]. IEEE transactions on neural systems and rehabilitation engineering, 2020, 28(2):390-398

[12] SheykhivandS, MousaviZ, RezaiiT Y, et al.. Recognizing emotions evoked by music using CNN-LSTM networks on EEG signals[J]. IEEE access, 2020, 8(1):139332-139345

[13] ShoeibiA, SadeghiD, MoridianP, et al.. Automatic diagnosis of schizophrenia in EEG signals using CNN-LSTM models[J]. Frontiers in neuroinformatics, 2021, 15(1):1-7

[14] XuG, RenT, ChenY, et al.. A one-dimensional CNN-LSTM model for epileptic seizure recognition using EEG signal analysis[J]. Frontiers in neuroscience, 2020, 14(1):1253-1259

[15] SimarC, PetieauM, CebollaA, et al.. EEG-based brain-computer interface for alpha speed control of a small robot using the MUSE head-band[C], 2020, New York, IEEE: 1-4

[16] DaiG, ZhouJ, HuangJ, et al.. HS-CNN: a CNN with hybrid convolution scale for EEG motor imagery classification[J]. Journal of neural engineering, 2020, 17(1):016025.1-016025.11

[17] HwaidiJ F, ChenT M. Classification of motor imagery EEG signals based on deep autoencoder and con-volutional neural network approach[J]. IEEE access, 2022, 10(1):48071-48081

[18] KhademiZ, EbrahimiF, KordyH M. A transfer learning-based CNN and LSTM hybrid deep learning model to classify motor imagery EEG signals[J]. Computers in biology and medicine, 2022, 143(1):105288

[19] AminS U, AlsulaimanM, MuhammadG, et al.. Deep learning for EEG motor imagery classification based on multi-layer CNNs feature fusion[J]. Future generation computer systems, 2019, 101(1): 542-554

[20] SinghalV, MathewJ, BeheraR K. Detection of alcoholism using EEG signals and a CNN-LSTM-ATTN network[J]. Computers in biology and medicine, 2021, 138(1):104940

[21] LiH, DingM, ZhangR, et al.. Motor imagery EEG classification algorithm based on CNN-LSTM feature fusion network[J]. Biomedical signal processing and control, 2022, 72(1): 103342

[22] JeongJ H, ShimK H, KimD J, et al.. Brain-controlled robotic arm system based on multi-directional CNN-BiLSTM network using EEG signals[J]. IEEE transactions on neural systems and rehabilitation engineering, 2020, 28(5): 1226-1238

[23] KaiK A, ZhangY C, ZhangH, et al.. Filter bank common spatial pattern (FBCSP) in brain-computer in-terface[C], 2008, New York, IEEE: 2390-2397

[24] MaX, WangD, LiuD, et al.. DWT and CNN based multi-class motor imagery electroencephalographic signal recognition[J]. Journal of neural engineering, 2020, 17(1): 016073