Predicting rock–paper–scissors choices based on single-trial EEG signals

Abstract

Decision prediction based on neurophysiological signals is of great application value in many real-life situations, especially in human–AI collaboration or counteraction. Single-trial analysis of electroencephalogram (EEG) signals is a very valuable step in the development of an online decision-prediction system. However, previous EEG-based decision-prediction methods focused mainly on averaged EEG signals of all decision-making trials to predict an individual's general decision tendency (e.g., risk seeking or aversion) over a period rather than on a specific decision response in a single trial. In the present study, we used a rock–paper–scissors game, which is a common multichoice decision-making task, to explore how to predict participants' single-trial choice with EEG signals. Forty participants, comprising 20 females and 20 males, played the game with a computer player for 330 trials. Considering that the decision-making process of this game involves multiple brain regions and neural networks, we proposed a new algorithm named common spatial pattern-attractor metagene (CSP-AM) to extract CSP features from different frequency bands of EEG signals that occurred during decision making. The results showed that a multilayer perceptron classifier achieved an accuracy significantly exceeding the chance level among 88.57% (31 of 35) of participants, verifying the classification ability of CSP features in multichoice decision-making prediction. We believe that the CSP-AM algorithm could be used in the development of proactive AI systems.

Keywords

attractor metagene / common spatial pattern / decision making / electroencephalogram (EEG) / single-trial prediction

References

[1] Ang, K. K., Chin, Z. Y., Wang, C., Guan, C., & Zhang, H. (2012). Filter Bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b. Frontiers in Neuroscience, 6, 39.

[2] Ang, K. K., Chin, Z. Y., Zhang, H., & Guan, C. (2008). Filter Bank common spatial pattern (FBCSP) in brain-computer Interface. IEEE International Joint Conference on Neural Networks, Hong Kong, China, 2008, 2390–2397.

[3] Anumanchipalli, G. K., Chartier, J., & Chang, E. F. (2019). Speech synthesis from neural decoding of spoken sentences. Nature, 568(7753), 493–498.

[4] Awais, M., & Henrich, D. (2010). Human-robot collaboration by intention recognition using probabilistic state machines. In 19th International Workshop on Robotics in Alpe-Adria-Danube Region (RAAD 2010) (pp. 75–80). Budapest, Hangary, 2010.

[5] Bartra, O., McGuire, J. T., & Kable, J. W. (2013). The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. NeuroImage, 76, 412–427.

[6] Bhushan, V., Saha, G., Lindsen, J., Shimojo, S., & Bhattacharya, J. (2012). How we choose one over another: Predicting trial-by-trial preference decision. PLoS One, 7(8), e43351.

[7] Blankertz, B., Tomioka, R., Lemm, S., Kawanabe, M., & Muller, K. (2008). Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Processing Magazine, 25(1), 41–56.

[8] Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.

[9] Busemeyer, J. R., Gluth, S., Rieskamp, J., & Turner, B. M. (2019). Cognitive and neural bases of multi-attribute, multi-alternative, Value-based Decisions. Trends in Cognitive Sciences, 23(3), 251–263.

[10] Chen, Z., Hu, X., Chen, Q., & Feng, T. (2019). Altered structural and functional brain network overall organization predict human intertemporal decision-making. Human Brain Mapping, 40(1), 306–328.

[11] Chong, A. Y.-L., Chan, F. T. S., & Ooi, K.-B. (2012). Predicting consumer decisions to adopt mobile commerce: Cross country empirical examination between China and Malaysia. Decision Support Systems, 53(1), 34–43.

[12] Combrisson, E., & Jerbi, K. (2015). Exceeding chance level by chance: The caveat of theoretical chance levels in brain signal classification and statistical assessment of decoding accuracy. Journal of Neuroscience Methods, 250, 126–136.

[13] Critchley, H. D., Elliott, R., Mathias, C. J., & Dolan, R. J. (2000). Neural activity relating to generation and representation of galvanic skin conductance responses: A functional magnetic resonance imaging study. The Journal of Neuroscience, 20(8), 3033–3040.

[14] Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems, 2(4), 303–314.

[15] Davis, C. E., Hauf, J. D., Wu, D. Q., & Everhart, D. E. (2011). Brain function with complex decision making using electroencephalography. International Journal of Psychophysiology, 79(2), 175–183.

[16] Delorme, A., & Makeig, S. (2004). EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134(1), 9–21.

[17] Ditterich, J. (2010). A comparison between mechanisms of multi-alternative perceptual decision making: Ability to explain human behavior, predictions for neurophysiology, and relationship with decision theory. Frontiers in Neuroscience, 4, 184.

[18] Dodds, C. M., Morein-Zamir, S., & Robbins, T. W. (2011). Dissociating inhibition, attention, and response control in the frontoparietal network using functional magnetic resonance imaging. Cerebral Cortex, 21(5), 1155–1165.

[19] Engel, A. K., & Fries, P. (2010). Beta-band oscillations—Signalling the status quo? Current Opinion in Neurobiology, 20(2), 156–165.

[20] Foxe, J. J., & Snyder, A. C. (2011). The role of alpha-band brain oscillations as a sensory suppression mechanism during selective attention. Frontiers in Psychology, 2, 154.

[21] Galar, M., Fernández, A., Barrenechea, E., Bustince, H., & Herrera, F. (2011). An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and one-vs-all schemes. Pattern Recognition, 44(8), 1761–1776.

[22] Güth, W., Schmittberger, R., & Schwarze, B. (1982). An experimental analysis of ultimatum bargaining. Journal of Economic Behavior & Organization, 3(4), 367–388.

[23] Hosseini, M.-P., Hosseini, A., & Ahi, K. (2021). A review on machine learning for EEG signal processing in bioengineering. IEEE Reviews in Biomedical Engineering, 14, 204–218.

[24] Hsu, W.-Y. (2013). Embedded prediction in feature extraction: Application to single-trial EEG discrimination. Clinical EEG and Neuroscience, 44(1), 31–38.

[25] Klimesch, W. (2012). Alpha-band oscillations, attention, and controlled access to stored information. Trends in Cognitive Sciences, 16(12), 606–617.

[26] Knerr, S., Personnaz, L., & Dreyfus, G. (1990). Single-layer learning revisited: A stepwise procedure for building and training a neural network. In F. F. Soulié & J. Hérault (Eds.), Neurocomputing (pp. 41–50). Springer.

[27] Krain, A. L., Wilson, A. M., Arbuckle, R., Castellanos, F. X., & Milham, M. P. (2006). Distinct neural mechanisms of risk and ambiguity: A meta-analysis of decision-making. NeuroImage, 32(1), 477–484.

[28] Lawhern, V. J., Solon, A. J., Waytowich, N. R., Gordon, S. M., Hung, C. P., & Lance, B. J. (2018). EEGNet: A compact convolutional neural network for EEG-based brain–computer interfaces. Journal of Neural Engineering, 15(5), 056013.

[29] Liu, H., & Yu, L. (2005). Toward integrating feature selection algorithms for classification and clustering. IEEE Transactions on Knowledge and Data Engineering, 17(4), 491–502.

[30] Lorena, A. C., de Carvalho, A. C. P. L. F., & Gama, J. M. P. (2008). A review on the combination of binary classifiers in multiclass problems. Artificial Intelligence Review, 30(1–4), 19–37.

[31] Lotte, F., Bougrain, L., Cichocki, A., Clerc, M., Congedo, M., Rakotomamonjy, A., & Yger, F. (2018). A review of classification algorithms for EEG-based brain–computer interfaces: A 10 year update. Journal of Neural Engineering, 15(3), 031005.

[32] Lu, H., Eng, H.-L., Guan, C., Plataniotis, K. N., & Venetsanopoulos, A. N. (2010). Regularized common spatial pattern with aggregation for EEG classification in small-sample setting. IEEE Transactions on Biomedical Engineering, 57(12), 2936–2946.

[33] Lv, C., Wang, Q., Chen, C., Qiu, J., Xue, G., & He, Q. (2019). The regional homogeneity patterns of the dorsal medial prefrontal cortex predict individual differences in decision impulsivity. NeuroImage, 200, 556–561.

[34] MuÈller-Gerking, J., Pfurtscheller, G., & Flyvbjerg, H. (1999). Designing optimal spatial filters for single-trial EEG classification in a movement task. Clinical Neurophysiology, 110(5), 787–798.

[35] Nguyen, T., Hettiarachchi, I., Khatami, A., Gordon-Brown, L., Lim, C. P., & Nahavandi, S. (2018). Classification of multi-class BCI data by common spatial pattern and fuzzy system. IEEE Access, 6, 27873–27884.

[36] Ou Yang, T.-H., Cheng, W.-Y., Zheng, T., Maurer, M. A., & Anastassiou, D. (2014). Breast cancer prognostic biomarker using attractor metagenes and the FGD3–SUSD3 metagene. Cancer Epidemiology Biomarkers & Prevention, 23(12), 2850–2856.

[37] Page, K. (2012). The four principles: Can they be measured and do they predict ethical decision making? BMC Medical Ethics, 13(1), 10.

[38] Park, Y., & Chung, W. (2019). Selective feature generation method based on time domain parameters and correlation coefficients for filter-Bank-CSP BCI systems. Sensors, 19(17), 3769.

[39] Pei, Y., Luo, Z., Zhao, H., Xu, D., Li, W., Yan, Y., Yan, H., Xie, L., Xu, M., & Yin, E. (2022). A tensor-based frequency features combination method for brain–computer interfaces. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 30, 465–475.

[40] Ratcliff, R., Philiastides, M. G., & Sajda, P. (2009). Quality of evidence for perceptual decision making is indexed by trial-to-trial variability of the EEG. Proceedings of the National Academy of Sciences, 106(16), 6539–6544.

[41] Rokach, L., & Maimon, O. (2013). Data mining with decision trees (Vol. 81). World Scientific.

[42] Sathya, R., & Abraham, A. (2013). Comparison of supervised and unsupervised learning algorithms for pattern classification. International Journal of Advanced Research in Artificial Intelligence, 2(2), 34–38.

[43] Selim, S., Tantawi, M. M., Shedeed, H. A., & Badr, A. (2018). A CSP\AM-BA-SVM approach for motor imagery BCI system. IEEE Access, 6, 49192–49208.

[44] Si, Y., Jiang, L., Tao, Q., Chen, C., Li, F., Jiang, Y., Zhang, T., Cao, X., Wan, F., Yao, D., & Xu, P. (2019). Predicting individual decision-making responses based on the functional connectivity of resting-state EEG. Journal of Neural Engineering, 16(6), 066025.

[45] Si, Y., Li, F., Duan, K., Tao, Q., Li, C., Cao, Z., Zhang, Y., Biswal, B., Li, P., Yao, D., & Xu, P. (2020). Predicting individual decision-making responses based on single-trial EEG. NeuroImage, 206, 116333.

[46] Sun, Q., Zhang, H., Zhang, J., & Zhang, X. (2018). Why Can't we accurately predict Others' decisions? Prediction discrepancy in risky decision-making. Frontiers in Psychology, 9, 2190.

[47] Suri, G., Sheppes, G., & Gross, J. J. (2013). Predicting affective choice. Journal of Experimental Psychology: General, 142(3), 627–632.

[48] Suzuki, S., Harasawa, N., Ueno, K., Gardner, J. L., Ichinohe, N., Haruno, M., Cheng, K., & Nakahara, H. (2012). Learning to simulate Others' decisions. Neuron, 74(6), 1125–1137.

[49] Szolnoki, A., Mobilia, M., Jiang, L.-L., Szczesny, B., Rucklidge, A. M., & Perc, M. (2014). Cyclic dominance in evolutionary games: A review. Journal of the Royal Society Interface, 11(100), 20140735.

[50] Tabar, Y. R., & Halici, U. (2017). A novel deep learning approach for classification of EEG motor imagery signals. Journal of Neural Engineering, 14(1), 016003.

[51] Tanabe, J., Thompson, L., Claus, E., Dalwani, M., Hutchison, K., & Banich, M. T. (2007). Prefrontal cortex activity is reduced in gambling and nongambling substance users during decision-making. Human Brain Mapping, 28(12), 1276–1286.

[52] Wang, L., Huang, W., Li, Y., Evans, J., & He, S. (2020). Multi-AI competing and winning against humans in iterated rock-paper-scissors game. Scientific Reports, 10(1), 13873.

[53] Wang, Z., Zhang, J., He, Y., & Zhang, J. (2022). EEG emotion recognition using multichannel weighted multiscale permutation entropy. Applied Intelligence, 52(10), 12064–12076.

[54] Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., & Vaughan, T. M. (2002). Brain–computer interfaces for communication and control. Clinical Neurophysiology, 113(6), 767–791.

[55] Wyart, V., de Gardelle, V., Scholl, J., & Summerfield, C. (2012). Rhythmic fluctuations in evidence accumulation during decision making in the human brain. Neuron, 76(4), 847–858.

[56] Xu, D., & Li, M. (2022). A dual alignment-based multi-source domain adaptation framework for motor imagery EEG classification. Applied Intelligence, 53, 10766–10788.

[57] Yin, X., Meng, M., She, Q., Gao, Y., & Luo, Z. (2021). Optimal channel-based sparse time-frequency blocks common spatial pattern feature extraction method for motor imagery classification. Mathematical Biosciences and Engineering, 18(4), 4247–4263.

[58] Zander, T. O., & Kothe, C. (2011). Towards passive brain–computer interfaces: Applying brain–computer interface technology to human–machine systems in general. Journal of Neural Engineering, 8(2), 025005.

RIGHTS & PERMISSIONS

2023 2023 The Authors. PsyCh Journal published by Institute of Psychology, Chinese Academy of Sciences and John Wiley & Sons Australia, Ltd.