Predicting rock–paper–scissors choices based on single-trial EEG signals

  • Zetong He 1 ,
  • Lidan Cui 2 ,
  • Shunmin Zhang 1 ,
  • Guibing He , 1
Expand
  • 1. Department of Psychology and Behavioral Sciences, Zhejiang University, Hangzhou, China
  • 2. College of Computer Science and Technology, Zhejiang University, Hangzhou, China
gbhe@zju.edu.cn

Received date: 15 Mar 2023

Accepted date: 17 Aug 2023

Published date: 20 Jan 2024

Copyright

2023 2023 The Authors. PsyCh Journal published by Institute of Psychology, Chinese Academy of Sciences and John Wiley & Sons Australia, Ltd.

Abstract

Decision prediction based on neurophysiological signals is of great application value in many real-life situations, especially in human–AI collaboration or counteraction. Single-trial analysis of electroencephalogram (EEG) signals is a very valuable step in the development of an online decision-prediction system. However, previous EEG-based decision-prediction methods focused mainly on averaged EEG signals of all decision-making trials to predict an individual's general decision tendency (e.g., risk seeking or aversion) over a period rather than on a specific decision response in a single trial. In the present study, we used a rock–paper–scissors game, which is a common multichoice decision-making task, to explore how to predict participants' single-trial choice with EEG signals. Forty participants, comprising 20 females and 20 males, played the game with a computer player for 330 trials. Considering that the decision-making process of this game involves multiple brain regions and neural networks, we proposed a new algorithm named common spatial pattern-attractor metagene (CSP-AM) to extract CSP features from different frequency bands of EEG signals that occurred during decision making. The results showed that a multilayer perceptron classifier achieved an accuracy significantly exceeding the chance level among 88.57% (31 of 35) of participants, verifying the classification ability of CSP features in multichoice decision-making prediction. We believe that the CSP-AM algorithm could be used in the development of proactive AI systems.

Cite this article

Zetong He , Lidan Cui , Shunmin Zhang , Guibing He . Predicting rock–paper–scissors choices based on single-trial EEG signals[J]. Psych Journal, 2024 , 13(1) : 19 -30 . DOI: 10.1002/pchj.688

1
Ang, K. K., Chin, Z. Y., Wang, C., Guan, C., & Zhang, H. (2012). Filter Bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b. Frontiers in Neuroscience, 6, 39.

DOI

2
Ang, K. K., Chin, Z. Y., Zhang, H., & Guan, C. (2008). Filter Bank common spatial pattern (FBCSP) in brain-computer Interface. IEEE International Joint Conference on Neural Networks, Hong Kong, China, 2008, 2390–2397.

DOI

3
Anumanchipalli, G. K., Chartier, J., & Chang, E. F. (2019). Speech synthesis from neural decoding of spoken sentences. Nature, 568(7753), 493–498.

DOI

4
Awais, M., & Henrich, D. (2010). Human-robot collaboration by intention recognition using probabilistic state machines. In 19th International Workshop on Robotics in Alpe-Adria-Danube Region (RAAD 2010) (pp. 75–80). Budapest, Hangary, 2010.

DOI

5
Bartra, O., McGuire, J. T., & Kable, J. W. (2013). The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. NeuroImage, 76, 412–427.

DOI

6
Bhushan, V., Saha, G., Lindsen, J., Shimojo, S., & Bhattacharya, J. (2012). How we choose one over another: Predicting trial-by-trial preference decision. PLoS One, 7(8), e43351.

DOI

7
Blankertz, B., Tomioka, R., Lemm, S., Kawanabe, M., & Muller, K. (2008). Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Processing Magazine, 25(1), 41–56.

DOI

8
Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.

DOI

9
Busemeyer, J. R., Gluth, S., Rieskamp, J., & Turner, B. M. (2019). Cognitive and neural bases of multi-attribute, multi-alternative, Value-based Decisions. Trends in Cognitive Sciences, 23(3), 251–263.

DOI

10
Chen, Z., Hu, X., Chen, Q., & Feng, T. (2019). Altered structural and functional brain network overall organization predict human intertemporal decision-making. Human Brain Mapping, 40(1), 306–328.

DOI

11
Chong, A. Y.-L., Chan, F. T. S., & Ooi, K.-B. (2012). Predicting consumer decisions to adopt mobile commerce: Cross country empirical examination between China and Malaysia. Decision Support Systems, 53(1), 34–43.

DOI

12
Combrisson, E., & Jerbi, K. (2015). Exceeding chance level by chance: The caveat of theoretical chance levels in brain signal classification and statistical assessment of decoding accuracy. Journal of Neuroscience Methods, 250, 126–136.

DOI

13
Critchley, H. D., Elliott, R., Mathias, C. J., & Dolan, R. J. (2000). Neural activity relating to generation and representation of galvanic skin conductance responses: A functional magnetic resonance imaging study. The Journal of Neuroscience, 20(8), 3033–3040.

DOI

14
Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems, 2(4), 303–314.

DOI

15
Davis, C. E., Hauf, J. D., Wu, D. Q., & Everhart, D. E. (2011). Brain function with complex decision making using electroencephalography. International Journal of Psychophysiology, 79(2), 175–183.

DOI

16
Delorme, A., & Makeig, S. (2004). EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134(1), 9–21.

DOI

17
Ditterich, J. (2010). A comparison between mechanisms of multi-alternative perceptual decision making: Ability to explain human behavior, predictions for neurophysiology, and relationship with decision theory. Frontiers in Neuroscience, 4, 184.

DOI

18
Dodds, C. M., Morein-Zamir, S., & Robbins, T. W. (2011). Dissociating inhibition, attention, and response control in the frontoparietal network using functional magnetic resonance imaging. Cerebral Cortex, 21(5), 1155–1165.

DOI

19
Engel, A. K., & Fries, P. (2010). Beta-band oscillations—Signalling the status quo? Current Opinion in Neurobiology, 20(2), 156–165.

DOI

20
Foxe, J. J., & Snyder, A. C. (2011). The role of alpha-band brain oscillations as a sensory suppression mechanism during selective attention. Frontiers in Psychology, 2, 154.

DOI

21
Galar, M., Fernández, A., Barrenechea, E., Bustince, H., & Herrera, F. (2011). An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and one-vs-all schemes. Pattern Recognition, 44(8), 1761–1776.

DOI

22
Güth, W., Schmittberger, R., & Schwarze, B. (1982). An experimental analysis of ultimatum bargaining. Journal of Economic Behavior & Organization, 3(4), 367–388.

DOI

23
Hosseini, M.-P., Hosseini, A., & Ahi, K. (2021). A review on machine learning for EEG signal processing in bioengineering. IEEE Reviews in Biomedical Engineering, 14, 204–218.

DOI

24
Hsu, W.-Y. (2013). Embedded prediction in feature extraction: Application to single-trial EEG discrimination. Clinical EEG and Neuroscience, 44(1), 31–38.

DOI

25
Klimesch, W. (2012). Alpha-band oscillations, attention, and controlled access to stored information. Trends in Cognitive Sciences, 16(12), 606–617.

DOI

26
Knerr, S., Personnaz, L., & Dreyfus, G. (1990). Single-layer learning revisited: A stepwise procedure for building and training a neural network. In F. F. Soulié & J. Hérault (Eds.), Neurocomputing (pp. 41–50). Springer.

DOI

27
Krain, A. L., Wilson, A. M., Arbuckle, R., Castellanos, F. X., & Milham, M. P. (2006). Distinct neural mechanisms of risk and ambiguity: A meta-analysis of decision-making. NeuroImage, 32(1), 477–484.

DOI

28
Lawhern, V. J., Solon, A. J., Waytowich, N. R., Gordon, S. M., Hung, C. P., & Lance, B. J. (2018). EEGNet: A compact convolutional neural network for EEG-based brain–computer interfaces. Journal of Neural Engineering, 15(5), 056013.

DOI

29
Liu, H., & Yu, L. (2005). Toward integrating feature selection algorithms for classification and clustering. IEEE Transactions on Knowledge and Data Engineering, 17(4), 491–502.

DOI

30
Lorena, A. C., de Carvalho, A. C. P. L. F., & Gama, J. M. P. (2008). A review on the combination of binary classifiers in multiclass problems. Artificial Intelligence Review, 30(1–4), 19–37.

DOI

31
Lotte, F., Bougrain, L., Cichocki, A., Clerc, M., Congedo, M., Rakotomamonjy, A., & Yger, F. (2018). A review of classification algorithms for EEG-based brain–computer interfaces: A 10 year update. Journal of Neural Engineering, 15(3), 031005.

DOI

32
Lu, H., Eng, H.-L., Guan, C., Plataniotis, K. N., & Venetsanopoulos, A. N. (2010). Regularized common spatial pattern with aggregation for EEG classification in small-sample setting. IEEE Transactions on Biomedical Engineering, 57(12), 2936–2946.

DOI

33
Lv, C., Wang, Q., Chen, C., Qiu, J., Xue, G., & He, Q. (2019). The regional homogeneity patterns of the dorsal medial prefrontal cortex predict individual differences in decision impulsivity. NeuroImage, 200, 556–561.

DOI

34
MuÈller-Gerking, J., Pfurtscheller, G., & Flyvbjerg, H. (1999). Designing optimal spatial filters for single-trial EEG classification in a movement task. Clinical Neurophysiology, 110(5), 787–798.

DOI

35
Nguyen, T., Hettiarachchi, I., Khatami, A., Gordon-Brown, L., Lim, C. P., & Nahavandi, S. (2018). Classification of multi-class BCI data by common spatial pattern and fuzzy system. IEEE Access, 6, 27873–27884.

DOI

36
Ou Yang, T.-H., Cheng, W.-Y., Zheng, T., Maurer, M. A., & Anastassiou, D. (2014). Breast cancer prognostic biomarker using attractor metagenes and the FGD3–SUSD3 metagene. Cancer Epidemiology Biomarkers & Prevention, 23(12), 2850–2856.

DOI

37
Page, K. (2012). The four principles: Can they be measured and do they predict ethical decision making? BMC Medical Ethics, 13(1), 10.

DOI

38
Park, Y., & Chung, W. (2019). Selective feature generation method based on time domain parameters and correlation coefficients for filter-Bank-CSP BCI systems. Sensors, 19(17), 3769.

DOI

39
Pei, Y., Luo, Z., Zhao, H., Xu, D., Li, W., Yan, Y., Yan, H., Xie, L., Xu, M., & Yin, E. (2022). A tensor-based frequency features combination method for brain–computer interfaces. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 30, 465–475.

DOI

40
Ratcliff, R., Philiastides, M. G., & Sajda, P. (2009). Quality of evidence for perceptual decision making is indexed by trial-to-trial variability of the EEG. Proceedings of the National Academy of Sciences, 106(16), 6539–6544.

DOI

41
Rokach, L., & Maimon, O. (2013). Data mining with decision trees (Vol. 81). World Scientific.

DOI

42
Sathya, R., & Abraham, A. (2013). Comparison of supervised and unsupervised learning algorithms for pattern classification. International Journal of Advanced Research in Artificial Intelligence, 2(2), 34–38.

DOI

43
Selim, S., Tantawi, M. M., Shedeed, H. A., & Badr, A. (2018). A CSP\AM-BA-SVM approach for motor imagery BCI system. IEEE Access, 6, 49192–49208.

DOI

44
Si, Y., Jiang, L., Tao, Q., Chen, C., Li, F., Jiang, Y., Zhang, T., Cao, X., Wan, F., Yao, D., & Xu, P. (2019). Predicting individual decision-making responses based on the functional connectivity of resting-state EEG. Journal of Neural Engineering, 16(6), 066025.

DOI

45
Si, Y., Li, F., Duan, K., Tao, Q., Li, C., Cao, Z., Zhang, Y., Biswal, B., Li, P., Yao, D., & Xu, P. (2020). Predicting individual decision-making responses based on single-trial EEG. NeuroImage, 206, 116333.

DOI

46
Sun, Q., Zhang, H., Zhang, J., & Zhang, X. (2018). Why Can't we accurately predict Others' decisions? Prediction discrepancy in risky decision-making. Frontiers in Psychology, 9, 2190.

DOI

47
Suri, G., Sheppes, G., & Gross, J. J. (2013). Predicting affective choice. Journal of Experimental Psychology: General, 142(3), 627–632.

DOI

48
Suzuki, S., Harasawa, N., Ueno, K., Gardner, J. L., Ichinohe, N., Haruno, M., Cheng, K., & Nakahara, H. (2012). Learning to simulate Others' decisions. Neuron, 74(6), 1125–1137.

DOI

49
Szolnoki, A., Mobilia, M., Jiang, L.-L., Szczesny, B., Rucklidge, A. M., & Perc, M. (2014). Cyclic dominance in evolutionary games: A review. Journal of the Royal Society Interface, 11(100), 20140735.

DOI

50
Tabar, Y. R., & Halici, U. (2017). A novel deep learning approach for classification of EEG motor imagery signals. Journal of Neural Engineering, 14(1), 016003.

DOI

51
Tanabe, J., Thompson, L., Claus, E., Dalwani, M., Hutchison, K., & Banich, M. T. (2007). Prefrontal cortex activity is reduced in gambling and nongambling substance users during decision-making. Human Brain Mapping, 28(12), 1276–1286.

DOI

52
Wang, L., Huang, W., Li, Y., Evans, J., & He, S. (2020). Multi-AI competing and winning against humans in iterated rock-paper-scissors game. Scientific Reports, 10(1), 13873.

DOI

53
Wang, Z., Zhang, J., He, Y., & Zhang, J. (2022). EEG emotion recognition using multichannel weighted multiscale permutation entropy. Applied Intelligence, 52(10), 12064–12076.

DOI

54
Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., & Vaughan, T. M. (2002). Brain–computer interfaces for communication and control. Clinical Neurophysiology, 113(6), 767–791.

DOI

55
Wyart, V., de Gardelle, V., Scholl, J., & Summerfield, C. (2012). Rhythmic fluctuations in evidence accumulation during decision making in the human brain. Neuron, 76(4), 847–858.

DOI

56
Xu, D., & Li, M. (2022). A dual alignment-based multi-source domain adaptation framework for motor imagery EEG classification. Applied Intelligence, 53, 10766–10788.

DOI

57
Yin, X., Meng, M., She, Q., Gao, Y., & Luo, Z. (2021). Optimal channel-based sparse time-frequency blocks common spatial pattern feature extraction method for motor imagery classification. Mathematical Biosciences and Engineering, 18(4), 4247–4263.

DOI

58
Zander, T. O., & Kothe, C. (2011). Towards passive brain–computer interfaces: Applying brain–computer interface technology to human–machine systems in general. Journal of Neural Engineering, 8(2), 025005.

DOI

Options
Outlines

/