A probabilistic generative model for tracking multi-knowledge concept mastery probability

Abstract

Knowledge tracing aims to track students’ knowledge status over time to predict students’ future performance accurately. In a real environment, teachers expect knowledge tracing models to provide the interpretable result of knowledge status. Markov chain-based knowledge tracing (MCKT) models, such as Bayesian Knowledge Tracing, can track knowledge concept mastery probability over time. However, as the number of tracked knowledge concepts increases, the time complexity of MCKT predicting student performance increases exponentially (also called explaining away problem). When the number of tracked knowledge concepts is large, we cannot utilize MCKT to track knowledge concept mastery probability over time. In addition, the existing MCKT models only consider the relationship between students’ knowledge status and problems when modeling students’ responses but ignore the relationship between knowledge concepts in the same problem. To address these challenges, we propose an inTerpretable pRobAbilistiC gEnerative moDel (TRACED), which can track students’ numerous knowledge concepts mastery probabilities over time. To solve explain away problem, we design long and short-term memory (LSTM)-based networks to approximate the posterior distribution, predict students’ future performance, and propose a heuristic algorithm to train LSTMs and probabilistic graphical model jointly. To better model students’ exercise responses, we proposed a logarithmic linear model with three interactive strategies, which models students’ exercise responses by considering the relationship among students’ knowledge status, knowledge concept, and problems. We conduct experiments with four real-world datasets in three knowledge-driven tasks. The experimental results show that TRACED outperforms existing knowledge tracing methods in predicting students’ future performance and can learn the relationship among students, knowledge concepts, and problems from students’ exercise sequences. We also conduct several case studies. The case studies show that TRACED exhibits excellent interpretability and thus has the potential for personalized automatic feedback in the real-world educational environment.

Graphical abstract

Keywords

probabilistic graphical model / deep learning / knowledge tracing / learner modeling

Hengyu Liu received a BS degree in computer science from Northeastern University, China in 2017. He is currently a PhD student studying computer software and theory at Northeastern University, China. His research interests include machine learning, deep learning, graph model, cognitive diagnosis and knowledge tracing

Tiancheng Zhang received a PhD degree in computer software and theory from Northeastern University (NEU) , China. He is currently an associate professor in the School of Computer Science and Engineering at NEU, China. His research interests include big data analysis, spatiotemporal data management, and deep learning

Fan Li received a BE degree in computer science from Qinghai University, China in 2019. He is currently a Master's student at the School of Computer Science and Engineering, Northeastern University, China. His research interest lies in artificial intelligence in education

Minghe Yu received the BS degree in computer science and technology from Northeastern University, China in 2012, and the PhD degree in computer science and technology from Tsinghua University, China in 2018. Since 2018, she has been a lecturer with the Software College, Northeastern University, China. Her research interests include big data, information retrieval, and data mining

Ge Yu received the PhD degree in computer science from Kyushu University, Japan in 1996. He is currently a professor and a PhD Supervisor at Northeastern University, China. His research interests include distributed and parallel databases, OLAP and data warehousing, data integration, and graph data management. He is a member of ACM and a Fellow of the China Computer Federation (CCF)

References

[1] Liu Q, Tong S, Liu C, Zhao H, Chen E, Ma H, Wang S. Exploiting cognitive structure for adaptive learning. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2019, 627−635

[2] Wu R, Liu Q, Liu Y, Chen E, Su Y, Chen Z, Hu G. Cognitive modelling for predicting examinee performance. In: Proceedings of the 24th International Conference on Artificial Intelligence. 2015, 1017−1024

[3] Ai F, Chen Y, Guo Y, Zhao Y, Wang Z, Fu G, Wang G. Concept-aware deep knowledge tracing and exercise recommendation in an online learning system. In: Proceedings of the 12th International Conference on Educational Data Mining. 2019

[4] Piech C, Bassen J, Huang J, Ganguli S, Sahami M, Guibas L. Deep knowledge tracing. In: Proceedings of the 28th International Conference on Neural Information Processing Systems. 2015, 505−513

[5] Chen Y, Liu Q, Huang Z, Wu L, Chen E, Wu R, Su Y, Hu G. Tracking knowledge proficiency of students with educational priors. In: Proceedings of 2017 ACM on Conference on Information and Knowledge Management. 2017, 989−998

[6] Sun S, Hu X, Bu C, Liu F, Zhang Y, Luo W. Genetic algorithm for Bayesian knowledge tracing: a practical application. In: Proceedings of the 13th International Conference on Sensing and Imaging. 2022, 282−293

[7] Wong T L, Zou D, Cheng G, Tang J K T, Cai Y, Wang F L . Enhancing skill prediction through generalising Bayesian knowledge tracing. International Journal of Mobile Learning and Organisation, 2021, 15( 4): 358–373

[8] Liu F, Hu X, Bu C, Yu K . Fuzzy Bayesian knowledge tracing. IEEE Transactions on Fuzzy Systems, 2022, 30( 7): 2412–2425

[9] Zhang K, Yao Y . A three learning states Bayesian knowledge tracing model. Knowledge-Based Systems, 2018, 148: 189–201

[10] Wellman M P, Henrion M . Explaining ’explaining away’. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1993, 15( 3): 287–292

[11] van der Linden W J, Hambleton R K. Handbook of Modern Item Response Theory. New York: Springer, 2013

[12] Cen H, Koedinger K, Junker B. Learning factors analysis–A general method for cognitive model evaluation and improvement. In: Proceedings of the 8th International Conference on Intelligent Tutoring Systems. 2006, 164−175

[13] Pavlik Jr P I, Cen H, Koedinger K R. Performance factors analysis–A new alternative to knowledge tracing. In: Proceedings of the Artificial Intelligence in Education: Building Learning Systems that Care: From Knowledge Representation to Affective Modelling, Proceedings of the 14th International Conference on Artificial Intelligence in Education. 2009

[14] Vie J J, Kashima H. Knowledge tracing machines: factorization machines for knowledge tracing. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, the 31st Innovative Applications of Artificial Intelligence Conference, IAAI 2019, the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019. 2019, 750−757

[15] Choffin B, Popineau F, Bourda Y, Vie J J. DAS3H: modeling student learning and forgetting for optimally scheduling distributed practice of skills. In: Proceedings of the 12th International Conference on Educational Data Mining. 2019

[16] Corbett A T, Anderson J R . Knowledge tracing: modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 1994, 4( 4): 253–278

[17] Gorgun G, Bulut O. Considering disengaged responses in Bayesian and deep knowledge tracing. In: Proceedings of the 23rd International Conference on Artificial Intelligence in Education. 2022, 591−594

[18] de la Torre J . Dina model and parameter estimation: a didactic. Journal of Educational and Behavioral Statistics, 2009, 34( 1): 115–130

[19] Liu H, Zhang T, Li F, Gu Y, Yu G . Tracking knowledge structures and proficiencies of students with learning transfer. IEEE Access, 2021, 9: 55413–55421

[20] Ausubel D P. Educational Psychology: A Cognitive View. New York: Holt, Rinehart and Winston, 1968

[21] Su Y, Liu Q, Liu Q, Huang Z, Yin Y, Chen E, Ding C, Wei S, Hu G. Exercise-enhanced sequential modeling for student performance prediction. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence and 30th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence. 2018, 297

[22] Liu Q, Huang Z, Yin Y, Chen E, Xiong H, Su Y, Hu G . EKT: exercise-aware knowledge tracing for student performance prediction. IEEE Transactions on Knowledge and Data Engineering, 2021, 33( 1): 100–115

[23] Lu Y, Wang D, Meng Q, Chen P. Towards interpretable deep learning models for knowledge tracing. In: Proceedings of the 21st International Conference on Artificial Intelligence in Education. 2020, 185−190

[24] Zhang J, Shi X, King I, Yeung D Y. Dynamic key-value memory networks for knowledge tracing. In: Proceedings of the 26th International Conference on World Wide Web. 2017, 765−774

[25] Sun X, Zhao X, Ma Y, Yuan X, He F, Feng J. Muti-behavior features based knowledge tracking using decision tree improved DKVMN. In: Proceedings of the ACM Turing Celebration Conference-China. 2019, 72

[26] Pandey S, Karypis G. A self attentive model for knowledge tracing. In: Proceedings of the 12th International Conference on Educational Data Mining. 2019

[27] Pandey S, Srivastava J. RKT: relation-aware self-attention for knowledge tracing. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020, 1205−1214

[28] Ghosh A, Heffernan N, Lan A S. Context-aware attentive knowledge tracing. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 2330−2339

[29] Zhu J, Yu W, Zheng Z, Huang C, Tang Y, Fung G P C. Learning from interpretable analysis: Attention-based knowledge tracing. In: Proceedings of the 21st International Conference on Artificial Intelligence in Education. 2020, 364−368

[30] Yu M, Li F, Liu H, Zhang T, Yu G . ContextKT: a context-based method for knowledge tracing. Applied Sciences, 2022, 12( 17): 8822

[31] Heng-yu L I U, Tian-cheng Z, Pei-wen W U, Ge Y U. A review of knowledge tracking. Journal of East China Normal University (Natural Science), 2019, 2019(5): 1

[32] Anzanello M J, Fogliatto F S. Learning curve models and applications: literature review and research directions. International Journal of Industrial Ergonomics, 2011, 41(5): 573−583

[33] von Foerster H. Understanding Understanding: Essays on Cybernetics and Cognition. Springer Science & Business Media, 2007

[34] Rabiner L R . A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 1989, 77( 2): 257–286

[35] Hinton G E, Dayan P, Frey B J, Neal R M . The “wake-sleep” algorithm for unsupervised neural networks. Science, 1995, 268( 5214): 1158–1161

[36] Fogarty J, Baker R S, Hudson S E. Case studies in the use of ROC curve analysis for sensor-based estimates in human computer interaction. In: Proceedings of Graphics Interface 2005. 2005, 129−136

[37] Liu H, Fu Q, Du L, Zhang T, Yu G, Han S, Zhang D. Learning rate perturbation: a generic plugin of learning rate schedule towards flatter local minima. In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 2022, 4234−4238

[38] Embretson S E, Reise S P. Item Response Theory. New York: Psychology Press, 2013

[39] Lindsey R V, Shroyer J D, Pashler H, Mozer M C . Improving students’ long-term knowledge retention through personalized review. Psychological Science, 2014, 25( 3): 639–647

[40] Mozer M C, Lindsey R V. Predicting and improving memory retention: psychological theory matters in the big data era. In: Jones M N, ed. Big Data in Cognitive Science. London: Routledge, Taylor & Francis Group, 2017, 34−64

[41] Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics. 2010, 249−256

[42] Mikolov T, Sutskever I, Chen K, Corrado G, Dean J. Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013, 3111−3119

Acknowledgements

The work described in this paper was supported by the National Natural Science Foundation of China (Grant Nos. 62272093, 62137001, U1811261, and 61902055).

RIGHTS & PERMISSIONS

2024 Higher Education Press