Elamur Research⌕
☰
home / papers / 2025 / FEAT: A Preference Feedback Dataset through a Cost-Effective Auto-Generation and Labeling Framework for English AI Tutoring

FEAT: A Preference Feedback Dataset through a Cost-Effective Auto-Generation and Labeling Framework for English AI Tutoring

Paper · 2025 · Hyein Seo, Taewook Hwang, Yohan Lee et al.

Summary

AI tutors for English need teacher feedback data that is both plentiful and good. Collecting and ranking it by hand is slow and expensive. The authors present FEAT, a framework that uses LLMs to build preference data for this feedback, and they release three datasets. DIRECT-Manual (DM) extends DIRECT: five feedback candidates from different sources are ranked by humans. DIRECT-Generated (DG) turns MCTest reading-comprehension items into tutoring dialogues. In DG, an LLM writes feedback twice, once guided by pedagogical criteria and once without them, and the criteria-guided version is treated as preferred. DIRECT-Augmented (DA) is DG with a small share of DM mixed in. To check label quality, they train pairwise ranking models on small Llama and Qwen models and score them against human rankings with RBO. Models trained only on DG come close to those trained on DM. Adding just 5–10% of DM to DG can beat training on all of DM. This suggests strong feedback rankers can be built with very little human annotation.

Key points

  • Feedback criteria. FEAT uses five criteria taken from Seo et al. (2025): Correct, Revealing, Guidance, Diagnostic, and Encouragement.
  • DM construction. Candidates come from Human, DIRECT, PrepTutor, GPT-3.5 and GPT-4. Annotators rank them on Correct and Revealing, and Correct wins when only one criterion is met. The ranked lists are expanded into chosen/rejected pairs.
  • Dataset sizes. DM has 5,025 training and 475 test items. DG has 3,996 training and 444 test items.
  • Rankers. Five approaches are compared: binary classifier, reward model, DPO, RankNet, and a majority-vote Ensemble. They run on Llama-1B, Llama-1B-IT, Llama-3B, Llama-3B-IT and Qwen-3B-IT.
  • Main results.
    • DM→DM reaches RBO 0.77–0.80; DG→DM peaks around 0.76.
    • With Llama-3B-IT, DA containing only 5–10% DM surpasses DM→DM.
    • Training with five criteria instead of two improves results, by up to +11.41% for DPO on Llama-1B.

Interesting ideas

  • I1. Free preference labels come from prompting the same LLM with and without explicit quality criteria and calling the criteria-guided output "better"; this gives contrastive pairs with no human ranking.
  • I2. A large silver set plus a small slice (5–10%) of gold data can outperform the full gold set, so the gold budget is best spent as a seed rather than as the whole corpus.
  • I3. Majority voting over several differently trained rankers smoothed out the instability of any single approach across model architectures.
  • I4. With only five candidates, RBO stays near 0.4 even for completely different rankings, so the metric's floor must be known before reading scores.
  • I5. The "Revealing" criterion rewards feedback that guides a student toward the answer without stating it, with correctness ranked above it in a strict priority order.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): LLM teacher tiers could produce explanations or frame readings with and without our UD/VerbNet constraints, yielding cheap preference pairs for training repair or ranking components.
  • I2 (learn): when training the perceptron parser or gparse on witness-agreed silver trees, inject a small hand-verified treebank fraction and sweep its ratio instead of chasing more gold.
  • I3 (learn): this backs the idea of combining several witness parsers by vote when producing silver data, especially where a single parser's behaviour is erratic.
  • I4 (learn): our evaluations should report chance-level baselines for any ranking metric on short lists, as a negative control against inflated numbers.
  • I5 (answer): when Mova explains a wrong answer on fable QA or school tests, it could point to the relevant sentence or entity in the text world rather than give the solution, checking accuracy first.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Chungnam National UniversityDaejeon · South Korea5 works · 13 citations🔬Electronics and Telecommunications Research InstituteDaejeon · South Korea3 works · 5 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,263 works

Cite

Cite as Hyein Seo, Taewook Hwang, Yohan Lee and Sangkeun Jung. 2025. FEAT: A Preference Feedback Dataset through a Cost-Effective Auto-Generation and Labeling Framework for English AI Tutoring. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-short.45

@inproceedings{seo2025feat,
  title = {{FEAT: A Preference Feedback Dataset through a Cost-Effective Auto-Generation and Labeling Framework for English AI Tutoring}},
  author = {Hyein Seo, Taewook Hwang, Yohan Lee and Sangkeun Jung},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-short.45},
  url = {https://doi.org/10.18653/v1/2025.acl-short.45}
}

References

53 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. ↗
  2. Nischal Ashok Kumar and Andrew Lan. 2024. Improving socratic question generation using data augmentation and preference optimization. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), pages 108–118, Mexico City, Mexico. Association for Computational Linguistics. ↗
  3. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609. ↗
  4. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. ↗
  5. Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. 2005. Learning to rank using gradient descent. In Proceedings of the 22nd International Conference on Machine Learning, ICML ’05, page 89–96, New York, NY, USA. Association for Computing Machinery.
  6. Christopher Burges, Robert Ragno, and Quoc Le. 2006. Learning to rank with nonsmooth cost functions. In Advances in Neural Information Processing Systems, volume 19. MIT Press. ↗
  7. Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. 2007. Learning to rank: from pairwise approach to listwise approach. In Proceedings of the 24th International Conference on Machine Learning, ICML ’07, page 129–136, New York, NY, USA. Association for Computing Machinery. find
  8. Nadine Chang, Francesco Ferroni, Michael J Tarr, Martial Hebert, and Deva Ramanan. 2023. Thinking like an annotator: Generation of dataset labeling instructions. arXiv preprint arXiv:2306.14035. ↗
  9. Koby Crammer and Yoram Singer. 2001. Pranking with ranking. In Advances in Neural Information Processing Systems, volume 14. MIT Press. ↗
  10. Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guotong Xie, Ruobing Xie, Yankai Lin, et al. 2023. Ultrafeedback: Boosting language models with scaled ai feedback. arXiv preprint arXiv:2310.01377. ↗
  11. Wei Dai, Jionghao Lin, Hua Jin, Tongguang Li, Yi-Shan Tsai, Dragan Gašević, and Guanliang Chen. 2023. Can large language models provide feedback to students? a case study on chatgpt. In 2023 IEEE International Conference on Advanced Learning Technologies (ICALT), pages 323–325. find
  12. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. ↗
  13. Wanyong Feng, Jaewook Lee, Hunter McNichols, Alexander Scarlatos, Digory Smith, Simon Woodhead, Nancy Ornelas, and Andrew Lan. 2024. Exploring automated distractor generation for math multiple-choice questions via large language models. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3067–3082, Mexico City, Mexico. Association for Computational Linguistics. find
  14. Davide Fossati. 2008. The role of positive feedback in intelligent tutoring systems. In Proceedings of the ACL-08: HLT Student Research Workshop, pages 31–36. find
  15. Yoav Freund, Raj Iyer, Robert E Schapire, and Yoram Singer. 2003. An efficient boosting algorithm for combining preferences. Journal of machine learning research, 4(Nov):933–969. find
  16. Wensheng Gan, Zhenlian Qi, Jiayang Wu, and Jerry Chun-Wei Lin. 2023. Large language models in education: Vision and opportunities. In 2023 IEEE International Conference on Big Data (BigData), pages 4776–4785. find
  17. John Hattie and Helen Timperley. 2007. The power of feedback. Review of educational research, 77(1):81– 112. ↗
  18. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685. ↗
  19. Jin-Xia Huang, Yohan Lee, and Oh-Woog Kwon. 2023. Direct: Toward dialogue-based reading comprehension tutoring. IEEE Access, 11:8978–8987. ↗
  20. Jaeho Jeon and Seongyong Lee. 2023. Large language models in education: A focus on the complementary relationship between human teachers and chatgpt. Education and Information Technologies, 28(12):15873–15892. ↗
  21. Thorsten Joachims. 2002. Optimizing search engines using clickthrough data. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’02, page 133–142, New York, NY, USA. Association for Computing Machinery. find
  22. Gerd Kortemeyer. 2024. Performance of the pre-trained large language model gpt-4 on automated short answer grading. Discover Artificial Intelligence, 4(1):47. ↗
  23. Charles Koutcheme, Nicola Dainese, Arto Hellas, Sami Sarsa, Juho Leinonen, Syed Ashraf, and Paul Denny. 2024. Evaluating language models for generating and judging programming feedback. find
  24. Yanyan Lan, Yadong Zhu, Jiafeng Guo, Shuzi Niu, and Xueqi Cheng. 2014. Position-aware listmle: a sequential learning process for ranking. In Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence, UAI’14, page 449–458, Arlington, Virginia, USA. AUAI Press. find
  25. Yu-Ju Lan and Nian-Shing Chen. 2024. Teachers’ agency in the era of llm and generative ai. Educational Technology & Society, 27(1):I–XVIII. find
  26. Jaewook Lee, Digory Smith, Simon Woodhead, and Andrew Lan. 2024. Math multiple choice question generation via human-large language model collaboration. In Proceedings of the 17th International Conference on Educational Data Mining, pages 941– 946, Atlanta, Georgia, USA. International Educational Data Mining Society. ↗
  27. Ping Li, Qiang Wu, and Christopher Burges. 2007. Mcrank: Learning to rank using multiple classification and gradient boosting. In Advances in Neural Information Processing Systems, volume 20. Curran Associates, Inc. find
  28. Wencke Liermann, Jin-Xia Huang, Yohan Lee, and Kong Joo Lee. 2024. More insightful feedback for tutoring: Enhancing generation mechanisms and automatic evaluation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 10838–10851, Miami, Florida, USA. Association for Computational Linguistics. find
  29. Haochen Liu, Zitao Liu, Zhongqin Wu, and Jiliang Tang. 2020. Personalized multimodal feedback generation in education. In Proceedings of the 28th International Conference on Computational Linguistics, pages 1826–1840, Barcelona, Spain (Online). International Committee on Computational Linguistics. find
  30. Haohao Luo, Yang Deng, Ying Shen, See-Kiong Ng, and Tat-Seng Chua. 2024a. Chain-of-exemplar: Enhancing distractor generation for multimodal educational question generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7978–7993, Bangkok, Thailand. Association for Computational Linguistics. find
  31. Jian Luo, Xuanang Chen, Ben He, and Le Sun. 2024b. PRP-graph: Pairwise ranking prompting to LLMs with graph aggregation for effective text re-ranking. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5766–5776, Bangkok, Thailand. Association for Computational Linguistics. find
  32. Wenting Ma, Olusola O Adesope, John C Nesbit, and Qing Liu. 2014. Intelligent tutoring systems and learning outcomes: A meta-analysis. Journal of educational psychology, 106(4):901. ↗
  33. Jennifer Meyer, Thorben Jansen, Ronja Schiller, Lucas W. Liebenow, Marlene Steinbach, Andrea Horbach, and Johanna Fleckenstein. 2024. Using llms to bring evidence-based feedback into the classroom: Ai-generated feedback increases secondary students’ text revision, motivation, and positive emotions. Computers and Education: Artificial Intelligence, 6:100199. ↗
  34. Nikahat Mulla and Prachi Gharpure. 2023. Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications. Progress in Artificial Intelligence, 12(1):1–32. ↗
  35. David J Nicol and Debra Macfarlane-Dick. 2006. Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in higher education, 31(2):199–218. ↗
  36. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, volume 35, pages 27730–27744. Curran Associates, Inc. ↗
  37. Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Le Yan, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, Xuanhui Wang, and Michael Bendersky. 2024. Large language models are effective text rankers with pairwise ranking prompting. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 1504–1518, Mexico City, Mexico. Association for Computational Linguistics. ↗
  38. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems, volume 36, pages 53728–53741. Curran Associates, Inc. ↗
  39. Matthew Richardson, Christopher J.C. Burges, and Erin Renshaw. 2013. MCTest: A challenge dataset for the open-domain machine comprehension of text. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 193–203, Seattle, Washington, USA. Association for Computational Linguistics. ↗
  40. Alexander Scarlatos, Wanyong Feng, Andrew Lan, Simon Woodhead, and Digory Smith. 2024a. Improving automated distractor generation for math multiple-choice questions with overgenerate-and-rank. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), pages 222–231, Mexico City, Mexico. Association for Computational Linguistics. ↗
  41. Alexander Scarlatos, Digory Smith, Simon Woodhead, and Andrew Lan. 2024b. Improving the validity of automatically generated feedback via reinforcement learning. find
  42. Innovative Use of NLP for Building Educational Applications (BEA 2024), pages 283–298, Mexico City, Mexico. Association for Computational Linguistics. find
  43. Jacob Steiss, Tamara Tate, Steve Graham, Jazmin Cruz, Michael Hebert, Jiali Wang, Youngsun Moon, Waverly Tseng, Mark Warschauer, and Carol Booth Olson. 2024. Comparing the quality of human and chatgpt feedback of students’ writing. Learning and Instruction, 91:101894. ↗
  44. Michael Taylor, John Guiver, Stephen Robertson, and Tom Minka. 2008. Softrank: optimizing non-smooth rank metrics. In Proceedings of the 2008 International Conference on Web Search and Data Mining, WSDM ’08, page 77–86, New York, NY, USA. Association for Computing Machinery. find
  45. Ming-Feng Tsai, Tie-Yan Liu, Tao Qin, Hsin-Hsi Chen, and Wei-Ying Ma. 2007. Frank: a ranking method with fidelity loss. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’07, page 383–390, New York, NY, USA. Association for Computing Machinery. find
  46. Shen Wang, Tianlong Xu, Hang Li, Chaoli Zhang, Joleen Liang, Jiliang Tang, Philip S Yu, and Qingsong Wen. 2024. Large language models for education: A survey and outlook. arXiv preprint arXiv:2403.18105. ↗
  47. Zichao Wang, Jakob Valdez, Debshila Basu Mallick, and Richard G. Baraniuk. 2022. Towards human-like educational question generation with large language models. In Artificial Intelligence in Education, pages 153–166, Cham. Springer International Publishing. find
  48. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2019. Huggingface’s transformers: State-ofthe-art natural language processing. arXiv preprint arXiv:1910.03771. ↗
  49. Zhaohui Zheng, Keke Chen, Gordon Sun, and Hongyuan Zha. 2007. A regression framework for learning ranking functions using relative relevance judgments. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’07, page 287–294, New York, NY, USA. Association for Computing Machinery. find
  50. Alexander Scarlatos, Digory Smith, Simon Woodhead, and Andrew Lan. 2024c. Improving the validity of automatically generated feedback via reinforcement learning. In International Conference on Artificial Intelligence in Education, pages 280–294. Springer. find
  51. Hyein Seo, Taewook Hwang, Jeesu Jung, Hyeonseok Kang, Hyuk Namgoong, Yohan Lee, and Sangkeun Jung. 2025. Large language models as evaluators in education: Verification of feedback consistency and accuracy. Applied Sciences, 15(2):671. ↗
  52. Valerie J Shute. 2008. Focus on formative feedback. Review of educational research, 78(1):153–189. ↗
  53. Maja Stahl, Leon Biermann, Andreas Nehring, and Henning Wachsmuth. 2024. Exploring LLM prompting strategies for joint essay scoring and feedback generation. In Proceedings of the 19th Workshop on ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/seo-2025-feat-preference-feedback-dataset-through/