Research on EEG emotion recognition based on CNN+BiLSTM+self-attention model

Abstract

To address the problems of insufficient dimensionality of electroencephalogram (EEG) feature extraction, the tendency to ignore the importance of different sequential data segments, and the poor generalization ability of the model in EEG based emotion recognition, the model of convolutional neural network and bi-directional long short-term memory and self-attention (CNN+BiLSTM+self-attention) is proposed. This model uses convolutional neural network (CNN) to extract more distinctive features from both spatial and temporal dimensions. The bi-directional long short-term memory (BiLSTM) is used to further preserve the long-term dependencies between the temporal phases of sequential data. The self-attention mechanism can change the weights of different channels to extract and highlight important information and address the often-ignored importance of different channels and samples when extracting EEG features. The subject-dependent experiment and subject-independent experiment are performed on the database for emotion analysis using physiological signals (DEAP) and collected datasets to verify the recognition performance. The experimental results show that the model proposed in this paper has excellent recognition performance and generalization ability.

References

[1] IbrahimA, ElijahO, OmobolajiF O. Performance evaluation of different support vector machine kernels for face emotion recognition[C], 2015, New York, IEEE: 15670349

[2] WeiC, ChenL L, SongZ Z, et al.. EEG-based emotion recognition using simple recurrent units network and ensemble learning[J]. Biomedical signal processing and control, 2020, 58(3):101756

[3] SoraiaM A, ManueJ F. Emotions recognition using EEG signals: a survey[J]. IEEE transactions on affective computing, 2019, 10(3): 374-393

[4] GaoY Y, SunX Y, MengM, et al.. EEG emotion recognition based on enhanced SPD matrix and manifold dimensionality reduction[J]. Computers in biology and medicine, 2022, 146(3):105606

[5] RahulS, RamB P, PradipS. Automated emotion recognition based on higher order statistics and deep learning algorithm[J]. Biomedical signal processing and control, 2020, 58: 101867

[6] ChengW X, GaoR B, SuganthanP N, et al.. EEG-based emotion recognition using random convolutional neural networks[J]. Engineering applications of artificial intelligence, 2022, 116(5–6): 105349

[7] CuiF, WangR, DingW, et al.. A novel DE-CNN-BiLSTM multi-fusion model for EEG emotion recognition[J]. Mathematics, 2022, 10(4): 582

[8] ChenP Y, GaoZ K, YinM M, et al.. Multiattention adaptation network for motor imagery recognition[J]. IEEE transactions on systems, man, and cybernetics: system, 2022, 52(8): 5127-5139

[9] LiC, WangB, ZhangS L, et al.. Emotion recognition from EEG based on multi-task learning with capsule network and attention mechanism[J]. Computers in biology and medicine, 2022, 143(22):105303

[10] WeiY, LiuY, LiC, et al.. TC-Net: a transformer capsule network for EEG-based emotion recognition[J]. Computers in biology and medicine, 2023, 152: 106463

[11] ZhongM Y, YangQ Y, LiuY, et al.. EEG emotion recognition based on TQWT-features and hybrid convolutional recurrent neural network[J]. Biomedical signal processing and control, 2023, 79(10): 104211

[12] LiuS Q, WangX, ZhaoL, et al.. Subject-independent emotion recognition of EEG signals based on dynamic empirical convolutional neural network[J]. IEEE/ACM transactions on computational biology and bioinformatics, 2021, 18(5): 1710-1721

[13] OrhanA, AbdulkadirŞ. Attention guided 3D CNN-LSTM model for accurate speech based emotion recognition[J]. Applied acoustics, 2021, 182(1):108260

[14] KongW C, DongZ Y, JiaY W, et al.. Short-term residential load forecasting based on LSTM recurrent neural network[J]. IEEE transactions on smart grid, 2019, 10(1): 841-851

[15] ELESSAWY R H, ELDAWLATLY S, ABBAS H M. A long short-term memory auto encoder approach for EEG motor imagery classification[C]//International Conference on Computation, Automation and Knowledge Management (ICCAKM), January 9–10, 2020, Dubai, United Arab Emirates. ICCAKM, 2020: 9051489.

[16] DadebayevD, GohW W, TanE X. EEG-based emotion recognition: Review of commercial EEG devices and machine learning techniques[J]. Journal of King Saud University-computer and information sciences, 2022, 34(7):4385-4401

[17] BhattA, AnantR B. EEG based emotion recognition using SVM and LibSVM[J]. International journal of computer applications, 2019, 178(45):1-3

[18] PanC, ShiC, MuH, et al.. EEG-based emotion recognition using logistic regression with Gaussian kernel and Laplacian prior and investigation of critical frequency bands[J]. Applied sciences, 2020, 10(5):1619

[19] HuyN P, MayD T, HuongN T, et al.. Combination of wavelet and MLP neural network for emotion recognition system[J]. International journal on future revolution in computer science & communication engineering, 2018, 4(11): 105-109

[20] NaliniP, AnuragS, ShrivishalT. Learning DenseNet features from EEG based spectrograms for subject independent emotion recognition[J]. Biomedical signal processing and control, 2022, 74(2):103485

[21] HuZ, ChenL, LuoY, et al.. EEG-based emotion recognition using convolutional recurrent neural network with multi-head self-attention[J]. Applied sciences, 2022, 12(21): 11255

[22] LiuW, QiuJ L, ZhengW L, et al.. Comparing recognition performance and robustness of multimodal deep learning models for multimodal emotion recogni-tion[J]. IEEE transactions on cognitive and developmental systems, 2022, 14(2): 715-729

[23] LiY, FuB X, LiF, et al.. A novel transferability attention neural network model for EEG emotion recognition[J]. Neurocomputing, 2021, 447(2):92-101