Elamur Research⌕
☰
home / papers / 2020 / Context-Aware Attentive Knowledge Tracing

Context-Aware Attentive Knowledge Tracing

Paper · 2020 · Aritra Ghosh, Neil Heffernan and Andrew S. Lan

Summary

Knowledge tracing (KT) forecasts how a learner will answer upcoming assessment questions from their earlier answers. Deep models such as DKT and DKVMN predict accurately but are opaque. Classic approaches like BKT and IRT-based models are interpretable but weaker. The authors introduce attentive knowledge tracing (AKT), built entirely from attention networks. Two self-attentive encoders produce contextualized representations of questions and of question–response pairs. An attention-based knowledge retriever links the current question to the learner's history, and a feed-forward network outputs the predicted response. The attention is monotonic: an exponential decay over a context-aware time distance reduces the weight of old interactions, mimicking forgetting. Question embeddings are regularized with the Rasch model, so items on the same concept differ through a learned difficulty scalar without a large parameter count. On several real-world benchmark datasets, AKT outperforms prior KT methods, by as much as 6% AUC in some cases. Case studies show that its attention patterns and difficulty estimates are meaningful. This makes the model useful for automated feedback and question recommendation in personalized learning.

Key points

  • AKT has four parts: a question encoder, a knowledge encoder, a knowledge retriever, and a response prediction network. Baselines include DKT, DKVMN, and SAKT. In the authors' experiments, SAKT did not beat DKT or DKVMN.
  • Attention weights combine question similarity with an exponential decay over a context-aware distance. Learners' attention concentrates on recent practice of the same concept, echoing hand-crafted count features used in DASH and PFA-style models.
  • The Rasch model-based embeddings give each question a difficulty parameter (µ_q) within its concept. On ASSISTments2009, t-SNE plots show questions of one concept arranged along a curve from easy to hard.
  • Learned difficulties match intuition. For "Probability of a Single Event," the easy, average, and hard items received µ_q values of −0.0515, 0.0088, and 0.0548.
  • Code and datasets were announced at github.com/arghosh/AKT. Future work includes using question text and testing the model on language-learning data that involves memory decay.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Massachusetts AmherstAmherst Center · United States122 works · 69,574 citations🎓Worcester Polytechnic InstituteWorcester · United States14 works · 1,103 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksInformation Systems · 1,888 worksComputer Science Applications · 184 works

Cite

Cite as Aritra Ghosh, Neil Heffernan and Andrew S. Lan. 2020. Context-Aware Attentive Knowledge Tracing. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1145/3394486.3403282 · arXiv 2007.12324

@inproceedings{ghosh2020context,
  title = {{Context-Aware Attentive Knowledge Tracing}},
  author = {Aritra Ghosh, Neil Heffernan and Andrew S. Lan},
  booktitle = {Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery \& Data Mining},
  year = {2020},
  doi = {10.1145/3394486.3403282},
  eprint = {2007.12324},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1145/3394486.3403282}
}

References

36 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hao Cen, Kenneth Koedinger, and Brian Junker. 2006. Learning factors analysis– A general method for cognitive model evaluation and improvement. In Proc. International Conference on Intelligent Tutoring Systems. 164–175.
  2. Benoît Choffin, Fabrice Popineau, Yolaine Bourda, and Jill-Jênn Vie. 2019. DAS3H: Modeling student learning and forgetting for pptimally scheduling distributed practice of skills. In Proc. International Conference on Educational Data Mining. 29–38. find
  3. Albert Corbett and John Anderson. 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-adapted Interaction 4, 4 (Dec. 1994), 253–278. find
  4. Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proc. International Conference on Artificial Intelligence and Statistics. 249–256. find
  5. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. find
  6. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition. 770–778.
  7. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation 9, 8 (Nov. 1997), 1735–1780. find
  8. MM Khajah, Y Huang, JP González-Brenes, MC Mozer, and P Brusilovsky. 2014. Integrating knowledge tracing and item response theory: A tale of two frameworks. In Proc. International Workshop on Personalization Approaches in Learning Environments, Vol. 1181. 7–15. find
  9. Mohammad Khajah, Robert Lindsey, and Michael Mozer. 2016. How deep is knowledge tracing?. In Proc. International Conference on Educational Data Mining. 94–101.
  10. Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proc. International Conference on Learning Representations.
  11. Andrew Lan and Richard Baraniuk. 2016. A Contextual Bandits Framework for Personalized Learning Action Selection. In Proc. International Conference on Educational Data Mining. 424–429. find
  12. Andrew Lan, Tom Goldstein, Richard Baraniuk, and Christoph Studer. 2016. Dealbreaker: A nonlinear latent variable model for educational data. In Proc. International Conference on Machine Learning. 266–275. find
  13. Andrew Lan, Christoph Studer, and Richard Baraniuk. 2014. Time-varying learning and content analytics via sparse factor analysis. In Proc. ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 452–461. find
  14. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450 (July 2016).
  15. Robert Lindsey, Jeffery Shroyer, Harold Pashler, and Michael Mozer. 2014. Improving students’ long-term knowledge retention through personalized review. Psychological Science 25, 3 (Jan. 2014), 639–647. find
  16. Frederick Lord. 1980. Applications of Item Response Theory to Practical Testing Problems. Erlbaum Associates. find
  17. Michael Mozer, Denis Kazakov, and Robert Lindsey. 2017. Discrete-event continuous-time recurrent nets. arXiv preprint arXiv:1710.04110 (Oct. 2017). — [arXiv:1710.04110](https://arxiv.org/abs/1710.04110) find
  18. Shalini Pandey and George Karypis. 2019. A self attentive model for knowledge tracing. In Proc. International Conference on Educational Data Mining. 384–389.
  19. Zachary Pardos and Neil Heffernan. 2010. Modeling individualization in a Bayesian networks implementation of knowledge tracing. In Proc. International Conference on User Modeling, Adaptation, and Personalization. 255–266. find
  20. Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013. On the difficulty of training recurrent neural networks. In Proc. International Conference on Machine Learning. 1310–1318. find
  21. Harold Pashler, Nicholas Cepeda, Robert Lindsey, Ed Vul, and Michael Mozer. 2009. Predicting the optimal spacing of study: A multiscale context model of memory. In Proc. Conference on Advances in Neural Information Processing Systems. 1321–1329. find
  22. Philip Pavlik Jr, Hao Cen, and Kenneth Koedinger. 2009. Performance factors analysis–A new alternative to knowledge tracing. In Proc. International Conference on Artificial Intelligence in Education. 531–538. find
  23. Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas J Guibas, and Jascha Sohl-Dickstein. 2015. Deep knowledge tracing. In Proc. Conference on Advances in Neural Information Processing Systems. 505–513.
  24. Chris Piech, Jonathan Huang, Andy Nguyen, Mike Phulsuksombati, Mehran Sahami, and Leonidas Guibas. 2015. Learning Program Embeddings to Propagate Feedback on Student Code. In Proc. International Conference on Machine Learning. 1093–1102. find
  25. Georg Rasch. 1993. Probabilistic Models for Some Intelligence and Attainment Tests. MESA Press. find
  26. Siddharth Reddy, Igor Labutov, Siddhartha Banerjee, and Thorsten Joachims. 2016. Unbounded human learning: Optimal scheduling for spaced repetition. In Proc. ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1815–1824. find
  27. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research 15, 1 (June 2014), 1929– 1958.
  28. Laurens Van Der Maaten. 2014. Accelerating t-SNE using tree-based algorithms. The Journal of Machine Learning Research 15, 1 (2014), 3221–3245. find
  29. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proc. Conference on Advances in Neural Information Processing Systems. 5998–6008.
  30. Jill-Jênn Vie and Hisashi Kashima. 2019. Knowledge tracing machines: Factorization machines for knowledge tracing. In Proc. AAAI Conference on Artificial Intelligence, Vol. 33. 750–757. find
  31. Kevin Wilson, Yan Karklin, Bojian Han, and Chaitanya Ekanadham. 2016. Back to the basics: Bayesian extensions of IRT outperform neural networks for proficiency estimation. In Proc. International Conference on Educational Data Mining. 539–544. find
  32. Beverly Park Woolf. 2010. Building Intelligent Interactive Tutors: Student-centered Strategies for Revolutionizing E-learning. Morgan Kaufmann. find
  33. Xiaolu Xiong, Siyuan Zhao, Eric G Van Inwegen, and Joseph E Beck. 2016. Going deeper with deep knowledge tracing. International Educational Data Mining Society (2016).
  34. Chun-Kit Yeung and Dit-Yan Yeung. 2018. Addressing two problems in deep knowledge tracing via prediction-consistent regularization. In Proc. ACM Conference on Learning at Scale. ACM, 5. find
  35. Michael Yudelson, Kenneth Koedinger, and Geoffrey Gordon. 2013. Individualized Bayesian knowledge tracing models. In Proc. International Conference on Artificial Intelligence in Education. 171–180. find
  36. Jiani Zhang, Xingjian Shi, Irwin King, and Dit-Yan Yeung. 2017. Dynamic key-value memory networks for knowledge tracing. In Proc. International Conference on World Wide Web. 765–774.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ghosh-2020-context-aware-attentive-knowledge-tracing/