Elamur Research⌕
☰
home / papers / 2023 / EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Education

EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Education

Paper · 2023 · Yuhao Dan, Zhikai Lei, Yiyang Gu et al.

Summary

General-purpose LLMs are poorly suited to education. They lack subject and pedagogical knowledge, their knowledge goes stale, and they tend to hallucinate. EduChat is an open-source chatbot built to serve teachers, students and parents. The authors draw on educational and psychological theory and train it in two stages. First, the model is further pre-trained on Chinese textbooks, exam question banks, classical poetry and psychology literature, plus about 4 million cleaned bilingual instructions. Second, it is tuned on roughly 500 thousand education-specific instructions. These target four skills: retrieval-augmented open QA with a self-check step that filters unhelpful search results, fine-grained essay assessment, Socratic tutoring, and counselling-style emotional support informed by REBT/ABC theory. System prompts toggle tools such as web search. On C-Eval, the LLaMa-13B-based model beats Chinese-Alpaca-13B, which shares the same base, and retrieval gives a further large boost. The released code, data and weights give researchers a reusable template for adapting LLMs to the education domain.

Key points

  • Pre-training data: educational textbooks and question banks from Chinese middle and high school exams, over 70,000 Chinese poems with author and background notes, and 60 selected psychology books. Fundamental instructions come from Alpaca, BELLE, GPT4All, Open-Assistant, FLANCoT, Firefly, MOSS, COIG, LIMA and ShareGPT.
  • Emotional-support data: ESConv was translated into Chinese as ESConv-zh. The authors then expanded it with simulated multi-agent dialogues and real Chinese counselling records.
  • Retrieval-augmented QA: ChatGPT judged whether retrieved content was useful and generated self-checked answers. The resulting data was manually verified.
  • C-Eval results: EduChat averages 40.7 versus 33.3 for Chinese-Alpaca-13B. With retrieval it reaches 49.3, approaching ChatGPT's 54.4.
  • Release: code, data and model weights are available on GitHub (icalk-nlp/EduChat) and Hugging Face (ecnu-icalk), with an online demo at educhat.top.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Fudan UniversityShanghai · China167 works · 20,987 citations🎓East China Normal UniversityShanghai · China50 works · 2,174 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksInformation Systems · 1,437 worksMedicine · 241 worksHealth Informatics · 184 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksDialogue systems · 624 works

Cite

Cite as Yuhao Dan, Zhikai Lei, Yiyang Gu, Yong Li, Jianghao Yin, Jiaju Lin, Linhao Ye, Zhiyan Tie, Yougen Zhou, Yilei Wang, Aimin Zhou, Ze Zhou, Qin Chen, Jie Zhou, Liang He and Xipeng Qiu. 2023. EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Education. arXiv:2308.02773.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2308.02773 · arXiv 2308.02773

@misc{dan2023educhat,
  title = {{EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Education}},
  author = {Yuhao Dan, Zhikai Lei, Yiyang Gu, Yong Li, Jianghao Yin, Jiaju Lin, Linhao Ye, Zhiyan Tie, Yougen Zhou, Yilei Wang, Aimin Zhou, Ze Zhou, Qin Chen, Jie Zhou, Liang He and Xipeng Qiu},
  year = {2023},
  doi = {10.48550/arxiv.2308.02773},
  eprint = {2308.02773},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2308.02773}
}

References

24 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alexis Baladn, Ignacio Sastre, Luis Chiruzzo, and Aiala Ros. 2023. RETUYT-InCo at BEA 2023 shared task: Tuning open-source LLMs for generating teacher responses. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 756–765, Toronto, Can… find
  2. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
  3. Jiaxi Cui, Zongjia Li, Yang Yan, Bohua Chen, and Li Yuan. 2023. Chatlaw: Open-source legal large language model with integrated external knowledge bases. ArXiv, abs/2306.16092.
  4. Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2022. Glm: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 320–335…
  5. Albert Ellis. 1991. The revised ABC’s of rational-emotive therapy (RET). Journal of Rational-Emotive and Cognitive-Behavior Therapy, 9(3):139–172. find
  6. Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Jiayi Lei, Yao Fu, Maosong Sun, and Junxian He. 2023. Ceval: A multi-level multi-discipline chinese evaluation suite for foundation models. arXiv preprint arXiv:2305.08322.
  7. Yunjie Ji, Yong Deng, Yan Gong, Yiping Peng, Qiang Niu, Lei Zhang, Baochang Ma, and Xiangang Li. 2023. Exploring the impact of instruction data scaling on large language models: An empirical study on real-world use cases. arXiv preprint arXiv:2303.14742.
  8. Yunxiang Li, Zihan Li, Kai Zhang, Ruilong Dan, Steve Jiang, and You Zhang. 2023. Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge. Cureus, 15(6).
  9. Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, and Minlie Huang. 2021. Towards emotional support dialog systems. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lan…
  10. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084.
  11. John Schulman, B Zoph, C Kim, J Hilton, J Menick, J Weng, JFC Uribe, L Fedus, L Metz, M Pokorny, et al. 2022. ChatGPT: Optimizing language models for dialogue. In OpenAI blog. find
  12. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020. Learning to summarize with human feedback. In Advances in Neural Information Processing Systems, volume 33, pages 3008–3021. find
  13. Tianxiang Sun, Xiaotian Zhang, Zhengfu He, Peng Li, Qinyuan Cheng, Hang Yan, Xiangyang Liu, Yunfan Shao, Qiong Tang, Xingjian Zhao, Ke Chen, Yining Zheng, Zhejian Zhou, Ruixiao Li, Jun Zhan, Yunhua Zhou, Linyang Li, Xiaogui Yang, Lingling Wu, Zhangyue Yin, Xuanjing Huang, and Xipeng Qiu. 2023. Moss:… find
  14. Anaïs Tack, Ekaterina Kochmar, Zheng Yuan, Serge Bibauw, and Chris Piech. 2023. find
  15. The BEA 2023 shared task on generating AI teacher responses in educational dialogues. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 785–795, Toronto, Canada. Association for Computational Linguistics. find
  16. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023a. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  17. Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Daniel M. Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Bria… — Llama 2: Open Foundation and Fine-Tuned Chat Models
  18. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022. Finetuned language models are zero-shot learners. In The Tenth International Conference on Learning Representations.
  19. Hongyang Yang, Xiao-Yang Liu, and Chris Wang. 2023. Fingpt: Open-source financial large language models. ArXiv, abs/2306.06031.
  20. Ge Zhang, Yemin Shi, Ruibo Liu, Ruibin Yuan, Yizhi Li, Siwei Dong, Yu Shu, Zhaoqun Li, Zekun Wang, Chenghua Lin, Wenhao Huang, and Jie Fu. 2023a. Chinese open instruction generalist: A preliminary release. find
  21. Hongbo Zhang, Junying Chen, Feng Jiang, Fei Yu, Zhihong Chen, Jianquan Li, Guimin Chen, Xiangbo Wu, Zhiyi Zhang, Qingying Xiao, Xiang Wan, Benyou Wang, and Haizhou Li. 2023b. Huatuogpt, towards taming language model to be a doctor. ArXiv, abs/2305.15075.
  22. Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. 2023a. Lima: Less is more for alignment. find
  23. Jie Zhou, Pei Ke, Xipeng Qiu, Minlie Huang, and Jun-ping Zhang. 2023b. Chatgpt: Potential, prospects, and limitations. Frontiers of Information Technology & Electronic Engineering, pages 1–6. find
  24. Tool Skill Retrieval Self-check General Psychology Socrates Retrieval-Augmented Open QA True True True False False Fine-grained Essay Assessment True/False True/False True False False Psychology-based Emotional Support False False False True False Socratic Teaching False False False False True *Tabl… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/dan-2023-educhat-large-scale-language-model/