Predicting rock–paper–scissors choices based on single-trial EEG signals

Abstract

Decision prediction based on neurophysiological signals is of great application value in many real-life situations, especially in human–AI collaboration or counteraction. Single-trial analysis of electroencephalogram (EEG) signals is a very valuable step in the development of an online decision-prediction system. However, previous EEG-based decision-prediction methods focused mainly on averaged EEG signals of all decision-making trials to predict an individual's general decision tendency (e.g., risk seeking or aversion) over a period rather than on a specific decision response in a single trial. In the present study, we used a rock–paper–scissors game, which is a common multichoice decision-making task, to explore how to predict participants' single-trial choice with EEG signals. Forty participants, comprising 20 females and 20 males, played the game with a computer player for 330 trials. Considering that the decision-making process of this game involves multiple brain regions and neural networks, we proposed a new algorithm named common spatial pattern-attractor metagene (CSP-AM) to extract CSP features from different frequency bands of EEG signals that occurred during decision making. The results showed that a multilayer perceptron classifier achieved an accuracy significantly exceeding the chance level among 88.57% (31 of 35) of participants, verifying the classification ability of CSP features in multichoice decision-making prediction. We believe that the CSP-AM algorithm could be used in the development of proactive AI systems.

Key words： attractor metagene; common spatial pattern; decision making; electroencephalogram (EEG); single-trial prediction

References

1 Ang, K. K., Chin, Z. Y., Wang, C., Guan, C., & Zhang, H. (2012). Filter Bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b. Frontiers in Neuroscience, 6, 39.

2 Ang, K. K., Chin, Z. Y., Zhang, H., & Guan, C. (2008). Filter Bank common spatial pattern (FBCSP) in brain-computer Interface. IEEE International Joint Conference on Neural Networks, Hong Kong, China, 2008, 2390–2397.

3 Anumanchipalli, G. K., Chartier, J., & Chang, E. F. (2019). Speech synthesis from neural decoding of spoken sentences. Nature, 568(7753), 493–498.

4 Awais, M., & Henrich, D. (2010). Human-robot collaboration by intention recognition using probabilistic state machines. In 19th International Workshop on Robotics in Alpe-Adria-Danube Region (RAAD 2010) (pp. 75–80). Budapest, Hangary, 2010.

5 Bartra, O., McGuire, J. T., & Kable, J. W. (2013). The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. NeuroImage, 76, 412–427.

6 Bhushan, V., Saha, G., Lindsen, J., Shimojo, S., & Bhattacharya, J. (2012). How we choose one over another: Predicting trial-by-trial preference decision. PLoS One, 7(8), e43351.

7 Blankertz, B., Tomioka, R., Lemm, S., Kawanabe, M., & Muller, K. (2008). Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Processing Magazine, 25(1), 41–56.

8 Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.

9 Busemeyer, J. R., Gluth, S., Rieskamp, J., & Turner, B. M. (2019). Cognitive and neural bases of multi-attribute, multi-alternative, Value-based Decisions. Trends in Cognitive Sciences, 23(3), 251–263.

10 Chen, Z., Hu, X., Chen, Q., & Feng, T. (2019). Altered structural and functional brain network overall organization predict human intertemporal decision-making. Human Brain Mapping, 40(1), 306–328.

11 Chong, A. Y.-L., Chan, F. T. S., & Ooi, K.-B. (2012). Predicting consumer decisions to adopt mobile commerce: Cross country empirical examination between China and Malaysia. Decision Support Systems, 53(1), 34–43.

12 Combrisson, E., & Jerbi, K. (2015). Exceeding chance level by chance: The caveat of theoretical chance levels in brain signal classification and statistical assessment of decoding accuracy. Journal of Neuroscience Methods, 250, 126–136.

13 Critchley, H. D., Elliott, R., Mathias, C. J., & Dolan, R. J. (2000). Neural activity relating to generation and representation of galvanic skin conductance responses: A functional magnetic resonance imaging study. The Journal of Neuroscience, 20(8), 3033–3040.

14 Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems, 2(4), 303–314.

15 Davis, C. E., Hauf, J. D., Wu, D. Q., & Everhart, D. E. (2011). Brain function with complex decision making using electroencephalography. International Journal of Psychophysiology, 79(2), 175–183.

16 Delorme, A., & Makeig, S. (2004). EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134(1), 9–21.

17 Ditterich, J. (2010). A comparison between mechanisms of multi-alternative perceptual decision making: Ability to explain human behavior, predictions for neurophysiology, and relationship with decision theory. Frontiers in Neuroscience, 4, 184.

18 Dodds, C. M., Morein-Zamir, S., & Robbins, T. W. (2011). Dissociating inhibition, attention, and response control in the frontoparietal network using functional magnetic resonance imaging. Cerebral Cortex, 21(5), 1155–1165.

19 Engel, A. K., & Fries, P. (2010). Beta-band oscillations—Signalling the status quo? Current Opinion in Neurobiology, 20(2), 156–165.

20 Foxe, J. J., & Snyder, A. C. (2011). The role of alpha-band brain oscillations as a sensory suppression mechanism during selective attention. Frontiers in Psychology, 2, 154.

21 Galar, M., Fernández, A., Barrenechea, E., Bustince, H., & Herrera, F. (2011). An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and one-vs-all schemes. Pattern Recognition, 44(8), 1761–1776.

22 Güth, W., Schmittberger, R., & Schwarze, B. (1982). An experimental analysis of ultimatum bargaining. Journal of Economic Behavior & Organization, 3(4), 367–388.

23 Hosseini, M.-P., Hosseini, A., & Ahi, K. (2021). A review on machine learning for EEG signal processing in bioengineering. IEEE Reviews in Biomedical Engineering, 14, 204–218.

24 Hsu, W.-Y. (2013). Embedded prediction in feature extraction: Application to single-trial EEG discrimination. Clinical EEG and Neuroscience, 44(1), 31–38.

25 Klimesch, W. (2012). Alpha-band oscillations, attention, and controlled access to stored information. Trends in Cognitive Sciences, 16(12), 606–617.

26 Knerr, S., Personnaz, L., & Dreyfus, G. (1990). Single-layer learning revisited: A stepwise procedure for building and training a neural network. In F. F. Soulié & J. Hérault (Eds.), Neurocomputing (pp. 41–50). Springer.

27 Krain, A. L., Wilson, A. M., Arbuckle, R., Castellanos, F. X., & Milham, M. P. (2006). Distinct neural mechanisms of risk and ambiguity: A meta-analysis of decision-making. NeuroImage, 32(1), 477–484.

28 Lawhern, V. J., Solon, A. J., Waytowich, N. R., Gordon, S. M., Hung, C. P., & Lance, B. J. (2018). EEGNet: A compact convolutional neural network for EEG-based brain–computer interfaces. Journal of Neural Engineering, 15(5), 056013.

29 Liu, H., & Yu, L. (2005). Toward integrating feature selection algorithms for classification and clustering. IEEE Transactions on Knowledge and Data Engineering, 17(4), 491–502.

30 Lorena, A. C., de Carvalho, A. C. P. L. F., & Gama, J. M. P. (2008). A review on the combination of binary classifiers in multiclass problems. Artificial Intelligence Review, 30(1–4), 19–37.

31 Lotte, F., Bougrain, L., Cichocki, A., Clerc, M., Congedo, M., Rakotomamonjy, A., & Yger, F. (2018). A review of classification algorithms for EEG-based brain–computer interfaces: A 10 year update. Journal of Neural Engineering, 15(3), 031005.

32 Lu, H., Eng, H.-L., Guan, C., Plataniotis, K. N., & Venetsanopoulos, A. N. (2010). Regularized common spatial pattern with aggregation for EEG classification in small-sample setting. IEEE Transactions on Biomedical Engineering, 57(12), 2936–2946.

33 Lv, C., Wang, Q., Chen, C., Qiu, J., Xue, G., & He, Q. (2019). The regional homogeneity patterns of the dorsal medial prefrontal cortex predict individual differences in decision impulsivity. NeuroImage, 200, 556–561.

34 MuÈller-Gerking, J., Pfurtscheller, G., & Flyvbjerg, H. (1999). Designing optimal spatial filters for single-trial EEG classification in a movement task. Clinical Neurophysiology, 110(5), 787–798.

35 Nguyen, T., Hettiarachchi, I., Khatami, A., Gordon-Brown, L., Lim, C. P., & Nahavandi, S. (2018). Classification of multi-class BCI data by common spatial pattern and fuzzy system. IEEE Access, 6, 27873–27884.

36 Ou Yang, T.-H., Cheng, W.-Y., Zheng, T., Maurer, M. A., & Anastassiou, D. (2014). Breast cancer prognostic biomarker using attractor metagenes and the FGD3–SUSD3 metagene. Cancer Epidemiology Biomarkers & Prevention, 23(12), 2850–2856.

37 Page, K. (2012). The four principles: Can they be measured and do they predict ethical decision making? BMC Medical Ethics, 13(1), 10.

38 Park, Y., & Chung, W. (2019). Selective feature generation method based on time domain parameters and correlation coefficients for filter-Bank-CSP BCI systems. Sensors, 19(17), 3769.

39 Pei, Y., Luo, Z., Zhao, H., Xu, D., Li, W., Yan, Y., Yan, H., Xie, L., Xu, M., & Yin, E. (2022). A tensor-based frequency features combination method for brain–computer interfaces. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 30, 465–475.

40 Ratcliff, R., Philiastides, M. G., & Sajda, P. (2009). Quality of evidence for perceptual decision making is indexed by trial-to-trial variability of the EEG. Proceedings of the National Academy of Sciences, 106(16), 6539–6544.

41 Rokach, L., & Maimon, O. (2013). Data mining with decision trees (Vol. 81). World Scientific.

42 Sathya, R., & Abraham, A. (2013). Comparison of supervised and unsupervised learning algorithms for pattern classification. International Journal of Advanced Research in Artificial Intelligence, 2(2), 34–38.

43 Selim, S., Tantawi, M. M., Shedeed, H. A., & Badr, A. (2018). A CSP\AM-BA-SVM approach for motor imagery BCI system. IEEE Access, 6, 49192–49208.

44 Si, Y., Jiang, L., Tao, Q., Chen, C., Li, F., Jiang, Y., Zhang, T., Cao, X., Wan, F., Yao, D., & Xu, P. (2019). Predicting individual decision-making responses based on the functional connectivity of resting-state EEG. Journal of Neural Engineering, 16(6), 066025.

45 Si, Y., Li, F., Duan, K., Tao, Q., Li, C., Cao, Z., Zhang, Y., Biswal, B., Li, P., Yao, D., & Xu, P. (2020). Predicting individual decision-making responses based on single-trial EEG. NeuroImage, 206, 116333.

46 Sun, Q., Zhang, H., Zhang, J., & Zhang, X. (2018). Why Can't we accurately predict Others' decisions? Prediction discrepancy in risky decision-making. Frontiers in Psychology, 9, 2190.

47 Suri, G., Sheppes, G., & Gross, J. J. (2013). Predicting affective choice. Journal of Experimental Psychology: General, 142(3), 627–632.

48 Suzuki, S., Harasawa, N., Ueno, K., Gardner, J. L., Ichinohe, N., Haruno, M., Cheng, K., & Nakahara, H. (2012). Learning to simulate Others' decisions. Neuron, 74(6), 1125–1137.

49 Szolnoki, A., Mobilia, M., Jiang, L.-L., Szczesny, B., Rucklidge, A. M., & Perc, M. (2014). Cyclic dominance in evolutionary games: A review. Journal of the Royal Society Interface, 11(100), 20140735.

50 Tabar, Y. R., & Halici, U. (2017). A novel deep learning approach for classification of EEG motor imagery signals. Journal of Neural Engineering, 14(1), 016003.

51 Tanabe, J., Thompson, L., Claus, E., Dalwani, M., Hutchison, K., & Banich, M. T. (2007). Prefrontal cortex activity is reduced in gambling and nongambling substance users during decision-making. Human Brain Mapping, 28(12), 1276–1286.

52 Wang, L., Huang, W., Li, Y., Evans, J., & He, S. (2020). Multi-AI competing and winning against humans in iterated rock-paper-scissors game. Scientific Reports, 10(1), 13873.

53 Wang, Z., Zhang, J., He, Y., & Zhang, J. (2022). EEG emotion recognition using multichannel weighted multiscale permutation entropy. Applied Intelligence, 52(10), 12064–12076.

54 Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., & Vaughan, T. M. (2002). Brain–computer interfaces for communication and control. Clinical Neurophysiology, 113(6), 767–791.

55 Wyart, V., de Gardelle, V., Scholl, J., & Summerfield, C. (2012). Rhythmic fluctuations in evidence accumulation during decision making in the human brain. Neuron, 76(4), 847–858.

56 Xu, D., & Li, M. (2022). A dual alignment-based multi-source domain adaptation framework for motor imagery EEG classification. Applied Intelligence, 53, 10766–10788.

57 Yin, X., Meng, M., She, Q., Gao, Y., & Luo, Z. (2021). Optimal channel-based sparse time-frequency blocks common spatial pattern feature extraction method for motor imagery classification. Mathematical Biosciences and Engineering, 18(4), 4247–4263.

58 Zander, T. O., & Kothe, C. (2011). Towards passive brain–computer interfaces: Applying brain–computer interface technology to human–machine systems in general. Journal of Neural Engineering, 8(2), 025005.