📄 Learning gain differences between ChatGPT and human tutor generated algebra hints

Paper · 2023 · Pardos

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

🏛️ Institutions

🎓University of California, BerkeleyBerkeley · United States243 works · 251,774 citations

🧭 Fields, topics and concepts

Cite

Cite as Pardos. 2023. Learning gain differences between ChatGPT and human tutor generated algebra hints. arXiv (Cornell University).

The paper itself, not this page. BibTeX from the library's record of the work. arXiv 2302.06871

@misc{pardos2023learning,
  title = {{Learning gain differences between ChatGPT and human tutor generated algebra hints}},
  author = {Pardos},
  year = {2023},
  eprint = {2302.06871},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2302.06871}
}

References

30 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Vincent Aleven, Bruce M McLaren, Jonathan Sewall, and Kenneth R Koedinger. 2006. The cognitive tutor authoring tools (CTAT): Preliminary evaluation of efficiency gains. In International Conference on Intelligent Tutoring Systems. Springer, 61–70. find
2. Tiffany Barnes and John Stamper. 2008. Toward automatic hint generation for logic proof tutoring using historical student data. In Intelligent Tutoring Systems: 9th International Conference, ITS 2008, Montreal, Canada, June 23-27, 2008 Proceedings 9. Springer, 373–382. find
3. Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021).
4. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
5. Milo Buwalda, Johan Jeuring, and Nico Naus. 2018. Use Expert Knowledge Instead of Data: Generating Hints for Hour of Code Exercises. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale (London, United Kingdom) (L@S ’18). Association for Computing Machinery, New York, NY, USA, Arti… — [doi:10.1145/3231644.3231690](https://doi.org/10.1145/3231644.3231690) find
6. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. Advances in neural information processing systems 30 (2017). find
7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
8. Roberto Gozalo-Brizuela and Eduardo C Garrido-Merchan. 2023. ChatGPT is not all you need. A State of the Art Review of large Generative AI models. arXiv preprint arXiv:2301.04655 (2023). — [arXiv:2301.04655](https://arxiv.org/abs/2301.04655) find
9. Ryung S Kim, Rob Weitz, Neil T Heffernan, and Nathan Krach. 2009. Tutored problem solving vs.“pure” worked examples. In Cognitive Science Society. 3121– 3126. find
10. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models. arXiv preprint arXiv:2301.12597 (2023).
11. Stephen MacNeil, Andrew Tran, Juho Leinonen, Paul Denny, Joanne Kim, Arto Hellas, Seth Bernstein, and Sami Sarsa. 2022. Automatically Generating CS Learning Materials with Large Language Models. arXiv preprint arXiv:2212.05113 (2022). — [arXiv:2212.05113](https://arxiv.org/abs/2212.05113) find
12. Stephen MacNeil, Andrew Tran, Dan Mogil, Seth Bernstein, Erin Ross, and Ziheng Huang. 2022. Generating diverse code explanations using the gpt-3 large language model. In Proceedings of the 2022 ACM Conference on International Computing Education Research-Volume 2. 37–39. find
13. Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On Faithfulness and Factuality in Abstractive Summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Online, 1906–1919. https://doi.org… ↗
14. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155 (2022).
15. Eleanor O’Rourke, Eric Butler, Armando Díaz Tolentino, and Zoran Popović. 2019. Automatic generation of problems and explanations for an intelligent algebra tutor. In Artificial Intelligence in Education: 20th International Conference, AIED 2019, Chicago, IL, USA, June 25-29, 2019, Proceedings, Part… find
16. Chris Piech, Mehran Sahami, Jonathan Huang, and Leonidas Guibas. 2015. Autonomously generating hints by inferring problem solving policies. In Proceedings of the second (2015) acm conference on learning@ scale. 195–204. find
17. Thomas W Price, Yihuan Dong, and Tiffany Barnes. 2016. Generating data-driven hints for open-ended programming. International Educational Data Mining Society (2016). find
18. Thomas W Price, Yihuan Dong, Rui Zhi, Benjamin Paaßen, Nicholas Lytle, Veronica Cateté, and Tiffany Barnes. 2019. A comparison of the quality of data-driven programming hint generation algorithms. International Journal of Artificial Intelligence in Education 29 (2019), 368–395. find
19. Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018. Improving language understanding by generative pre-training. (2018). find
20. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9. find
21. Leena Razzaq, Jozsef Patvarczki, Shane F Almeida, Manasi Vartak, Mingyu Feng, Neil T Heffernan, and Kenneth R Koedinger. 2009. The Assistment Builder: Supporting the life cycle of tutoring system content creation. IEEE Transactions on Learning Technologies 2, 2 (2009), 157–166. find
22. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 (2019).
23. Kelly Rivers and Kenneth R Koedinger. 2014. Automating hint generation with solution space path construction. In Intelligent Tutoring Systems: 12th International Conference, ITS 2014, Honolulu, HI, USA, June 5-9, 2014. Proceedings 12. Springer, 329–339. find
24. Rohan Roy Choudhury, Hezheng Yin, and Armando Fox. 2016. Scale-driven automatic hint generation for coding style. In Intelligent Tutoring Systems: 13th International Conference, ITS 2016, Zagreb, Croatia, June 7-10, 2016. Proceedings 13. Springer, 122–132. find
25. Jürgen Rudolph, Samson Tan, and Shannon Tan. 2023. ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning and Teaching 6, 1 (2023). find
26. Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. Retrieval augmentation reduces hallucination in conversation. arXiv preprint arXiv:2104.07567 (2021).
27. John Stamper, Michael Eagle, Tiffany Barnes, and Marvin Croy. 2013. Experimental evaluation of automatic hint generation for a logic tutor. International Journal of Artificial Intelligence in Education 22, 1-2 (2013), 3–17. find
28. H Holden Thorp. 2023. ChatGPT is fun, but not an author. , 313–313 pages. find
29. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
30. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171 (2022).

Updated 3 November 2026

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pardos-2023-learning-gain-differences-between-chatgpt/