Uncovering Emotion Correlates to Transitions in EEG Energy Landscapes

Abstract

Wearable brain-computer interfaces (BCIs) have made it feasible to monitor brain activity for emotion recognition in real-world settings. While deep learning models achieve high classification accuracy on electroencephalography (EEG) data, they often lack interpretability, limiting their neuroscientific relevance. In this study, we present an interpretable framework for EEG-based emotion analysis rooted in energy landscape analysis. EEG signals from the DEAP dataset were standardized and binarized prior to quantification of neural state transitions. We found significant subject-specific correlations between the number of state transitions and emotional ratings of valence and arousal. Further analysis revealed that certain binary brain states, particularly complementary pairs, were among the most frequently observed and showed emotion-dependent frequency differences. Transitions between these state pairs varied across subjects, suggesting their role as local minima in the brain’s dynamic landscape. Our findings demonstrate that energy landscape analysis provides an interpretable alternative to black-box models, offering insights into how brain dynamics relate to emotional experiences. This approach contributes toward building explainable affective computing systems and supports the use of neural state modeling in emotion-aware BCIs.

Keywords

EEG / energy landscape analysis / emotion recognition

Author Contributions

A.: conceptualization, methodology, software, formal analysis, investigation, visualization, writing—original draft preparation; K.F.: supervision, validation, resources, funding acquisition, writing—review and editing. All authors have read and agreed to the published version of the manuscript.

Funding

This work is supported by JSPS KAKENHI Grant Numbers JP22K18419, JP24K15161, JP25H00451, JST Moonshot RD Grant No. JPMJMS2021.

Institutional Review Board Statement

Ethical review and approval were waived for this study, as it involved the analysis of a publicly available dataset with no personally identifiable information, and no new data were collected from human participants. The original data collection procedures and ethical approvals are described in the DEAP dataset documentation.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Use of AI and AI-Assisted Technologies

During the preparation of this work, the authors used ChatGPT (OpenAI) to assist with language polishing and improving the clarity and flow of the text. After using this tool, the authors reviewed and edited all content as needed and take full responsibility for the content of the published article.

References

[1] Wu, Y.; Qiao, S.; Zhong, J.; et al. FNIRS—based energy landscape analysis to signify brain activity dynamics of individuals with depression. CNS Neurosci. Ther. 2024, 30 , e70139.

[2] Xing, L.; Guo, Z.; Long, Z. Energy landscape analysis of brain network dynamics in Alzheimer’s disease. Front. Aging Neurosci. 2024, 16 , 1375091.

[3] Liu, J.; Wu, G.; Luo, Y.; et al. EEG—Based Emotion Classification Using a Deep Neural Network and Sparse Autoencoder. Front. Syst. Neurosci. 2020, 14 , 43.

[4] Anubhav.; Nath, D.; Singh, M.; et al. An Efficient Approach to EEG—Based Emotion Recognition using LSTM Network. In Proceedings of the 16th IEEE International Colloquium on Signal Processing & Its Applications (CSPA), Langkawi, Malaysia, 28—29 February 2020; pp. 88-92.

[5] Anubhav; Fujiwara, K. Reservoir Splitting Method for EEG—based Emotion Recognition. In Proceedings of the 11th International Winter Conference on Brain—Computer Interface (BCI), Gangwon, Republic of Korea, 20—22 February 2023; pp. 1-5.

[6] Anubhav.; Fujiwara, K. Across Trials vs. Subjects vs. Contexts: A Multi—Reservoir Computing Approach for EEG Variations in Emotion Recognition. In Proceedings of the 26th International Conference on Multimodal Interaction, San Jose, Costa Rica, 4—8 November 2024; pp. 518-525.

[7] Yin, Y.; Zheng, X.; Hu, B.; et al. EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM. Appl. Soft Comput. 2021, 100 , 106954.

[8] Koelstra, S.; Muhl, C.; Soleymani, M.; et al. DEAP: A Database for Emotion Analysis ;Using Physiological Signals. IEEE Trans. Affect. Comput. 2012, 3, 18-31.

[9] Ma, J.; Tang, H.; Zheng, W.L.; et al. Emotion Recognition Using Multimodal Residual LSTM Network. In Proceedings of the 27th ACM International Conference on Multimedia (MM’19), Nice, France, 21—25 October 2019; pp. 1776-1782.

[10] Hasan, M.; Anzum, R.N.; Yasmin, S.; et al. Fine—Grained Emotion Recognition from EEG Signal Using Fast Fourier Transformation and CNN. In Proceedings of the Joint 10th International Conference on Informatics, Electronics & Vision (ICIEV) and 2021 5th International Conference on Imaging, Vision & Pattern Recognition (icIVPR), Kitakyushu, Japan, 16—20 August 2021; pp. 1-9.

[11] Nath, D.; Anubhav.; Singh, M.; Sethia, D.; Kalra, D.; Indu, S. A Comparative Study of Subject—Dependent and Subject—Independent Strategies for EEG—Based Emotion Recognition Using LSTM Network. In Proceedings of the 4th International Conference on Compute and Data Analysis, Silicon Valley, CA, USA, 9—12 March 2020; pp. 142-147.

[12] Alhalaseh, R.; Alasasfeh, S. Machine—Learning—Based Emotion Recognition System Using EEG Signals. Computers 2020, 9, 95.

[13] Kim, Y.; Choi, A. EEG—Based Emotion Classification Using Long Short—Term Memory Network with Attention Mechanism. Sensors 2020, 20 , 6727.

[14] Karthiga, M.; Suganya, E.; Sountharrajan, S.; et al. EEG based smart emotion recognition using meta heuristic optimization and hybrid deep learning techniques. Sci. Rep. 2024, 14 , 30251.

[15] Watanabe, T.; Hirose, S.; Wada, H.; et al. A Pairwise Maximum Entropy Model Accurately Describes Resting—State Human Brain Networks. Nat. Commun. 2013, 4, 1370.

[16] Watanabe, T.; Masuda, N.; Megumi, F.; et al. Energy Landscape and Dynamics of Brain Activity during Human Bistable Perception. Nat. Commun. 2014, 5, 4765.

[17] Barzon, G.; Ambrosini, E.; Vallesi, A.; et al. EEG microstate transition cost correlates with task demands. PLoS Comput. Biol. 2024, 20 , e1012521.

[18] Kang, J.; Pae, C.; Park, H.J. Graph—theoretical analysis for energy landscape reveals the organization of state transitions in the resting—state human cerebral cortex. PLoS ONE 2019, 14 , e0222161.