Attention emotion recognition via ECG signals

Abstract

Background: Physiological signal-based research has been a hot topic in affective computing. Previous works mainly focus on some strong, short-lived emotions (e.g., joy, anger), while the attention, which is a weak and long-lasting emotion, receives less attraction. In this paper, we present a study of attention recognition based on electrocardiogram (ECG) signals, which contain a wealth of information related to emotions.

Methods: The ECG dataset is derived from 10 subjects and specialized for attention detection. To relieve the impact of noise of baseline wondering and power-line interference, we apply wavelet threshold denoising as preprocessing and extract rich features by pan-tompkins and wavelet decomposition algorithms. To improve the generalized ability, we tested the performance of a variety of combinations of different feature selection algorithms and classifiers.

Results: Experiments show that the combination of generic algorithm and random forest achieve the highest correct classification rate (CCR) of 86.3%.

Conclusion: This study indicates the feasibility and bright future of ECG-based attention research.

Graphical abstract

Keywords

affective computing / attention recognition / ECG signals

References

[1] Picard R.. ( 2000) Affective Computing. Cambridge: MIT press

[2] Ekman, P. Friesen, W. ( 1971). Constants across cultures in the face and emotion. J. Pers. Soc. Psychol., 17 : 124– 129

[3] Fox, C. J. Barton, J. ( 2007). What is adapted in face adaptation? The neural representations of expression in the human visual system.. Brain Res., 1127 : 80– 89

[4] Batty, M. Taylor, M. ( 2003). Early processing of the six basic facial emotional expressions. Brain Res. Cogn. Brain Res., 17 : 613– 620

[5] Perikos, I. ( 2016). Recognizing emotions in text using ensemble of classifiers. Eng. Appl. Artif. Intell., 51 : 191– 201

[6] Plutchik, R. ( 2001). The nature of emotions. Am. Sci., 89 : 344– 350

[7] Lang, P. ( 1995). The emotion probe. Studies of motivation and attention. Am. Psychol., 50 : 372– 385

[8] Kuo, Y. Chu, H. Tsai, M. ( 2017). Effects of an integrated physiological signal-based attention-promoting and English listening system on students’ learning performance and behavioral patterns. Comput. Human Behav., 75 : 218– 227

[9] Song, T., Zheng, W., Song, P. ( 2020). EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans. Affect. Comput., 11 : 532– 541

[10] Hsu, Y., Wang, J., Chiang, W. ( 2020). Automatic ECG-based emotion recognition in music listening. IEEE Trans. Affect. Comput., 11 : 85– 99

[11] Liu, Y., Yu, M., Zhao, G., Song, J., Ge, Y. ( 2018). Real-time movie-induced discrete emotion recognition from EEG signals. IEEE Trans. Affect. Comput., 9 : 550– 562

[12] Ding, Y., Hu, X., Xia, Z., Liu, Y. ( 2021). Inter-brain EEG feature extraction and analysis for continuous implicit emotion tagging during video watching. IEEE Trans. Affect. Comput., 12 : 92– 102

[13] Du, X., Ma, C., Zhang, G., Li, J., Lai, Y. Zhao, G., Deng, X., Liu, Y. ( 2020). An efficient LSTM network for emotion recognition from multichannel EEG signals. IEEE Trans. Affect. Comput., 3013711

[14] Zhang, G., Yu, M., Liu, Y. Zhao, G., Zhang, D. ( 2021). SparseDGCNN: Recognizing emotion from multichannel EEG signals. IEEE Trans. Affect. Comput., 3051332

[15] Pourtois, G., Schettino, A. ( 2013). Brain mechanisms for emotional influences on perception and attention: what is magic and what is not. Biol. Psychol., 92 : 492– 512

[16] Taylor, J. G. Fragopanagos, N. ( 2005). The interaction of attention and emotion. Neural Netw., 18 : 353– 369

[17] Aliakbaryhosseinabadi, S., Kamavuako, E. N., Jiang, N., Farina, D. ( 2017). Classification of EEG signals to identify variations in attention during motor task execution. J. Neurosci. Methods, 284 : 27– 34

[18] Liu, N. H., Chiang, C. Y. Chu, H. ( 2013). Recognizing the degree of human attention using EEG signals from mobile sensors. Sensors (Basel), 13 : 10273– 10286

[19] Hamadicharef B., Zhang H., Guan C., Wang C., Phua K. S., Tee K. P. Ang K.. ( 2009) Learning EEG-based spectral-spatial patterns for attention level measurement. In: IEEE Inter. Symp. Circ. Syst., pp. 1465– 1468

[20] Eddin Alchalabi A., Elsharnouby M., Shirmohammadi S.. ( 2017) Feasibility of detecting ADHD patients’attention levels by classifying their EEG signals. In: 2017 IEEE Inter. Symp. Medic. Measur. Applic. (MeMeA), pp. 314– 319

[21] Ghanadian, H., Ghodratigohar, M. ( 2018). A machine learning method to improve non-contact heart rate monitoring using an RGB camera. IEEE Access, 6 : 57085– 57094

[22] Egger, M., Ley, M. ( 2019). Emotion recognition from physiological signal analysis: A review. Electron. Notes Theor. Comput. Sci., 343 : 35– 55

[23] Emanet N.. ( 2009) ECG beat classification by using discrete wavelet transform and Random Forest algorithm. In: 2009 Fifth Inter. Confer. Soft Comput., Comput. Words Percept. Syst. Anal., Decis. Contr., Famag., 5379457

[24] Zhang, Y. D., Yang, Z. J., Lu, H. M., Zhou, X. X., Phillips, P., Liu, Q. M. Wang, S. ( 2016). Facial emotion recognition based on biorthogonal wavelet entropy, fuzzy support vector machine, and stratified cross validation. IEEE Access, 4 : 8375– 8385

[25] Desimone, R. ( 1995). Neural mechanisms of selective visual attention. Annu. Rev. Neurosci., 18 : 193– 222

[26] Agante P. M. Marques de Sa J.. ( 1999) ECG noise filtering using wavelets with soft-thresholding methods. In: Proc. Comput. Cardiology 1999, pp. 535– 538

[27] Lu, G., Brittain, J. S., Holland, P., Yianni, J., Green, A. L., Stein, J. F., Aziz, T. Z. ( 2009). Removing ECG noise from surface EMG signals using adaptive filtering. Neurosci. Lett., 462 : 14– 19

[28] Donoho D. L. Johnstone I.. ( 1995) Adapting to unknown smoothness via wavelet shrinkage. J. Am. Stat. Assoc., 90, 1200– 1224.

[29] Lenis, G., Pilia, N., Loewe, A., Schulze, W. H. ( 2017). Comparison of baseline wander removal techniques considering the preservation of ST changes in the ischemic ECG: A simulation Study. Comput. Math. Methods. Med., 2017 : 9295029

[30] Pan, J. Tompkins, W. ( 1985). A real-time QRS detection algorithm. IEEE Trans. Biomed. Eng., 32 : 230– 236

[31] Hamilton, P. S. Tompkins, W. ( 1986). Quantitative investigation of QRS detection rules using the MIT/BIH arrhythmia database. IEEE Trans. Biomed. Eng., 33 : 1157– 1165

[32] Liu C., Rani P.. ( 2005) An empirical study of machine learning techniques for affect recognition in human-robot interaction. In: 2005 IEEE/RSJ Inter. Confer. Intellig. Robots Syst., pp. 2662– 2667

[33] Website: Accessed: January 5, 2021

[34] Bradley, M. M. Lang, P. ( 1994). Measuring emotion: The self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry, 25 : 49– 59

RIGHTS & PERMISSIONS

The Author(s). Published by Higher Education Press.