Elamur Research⌕
☰
home / papers / 2026 / Controllable Spoken Dialogue Generation: An LLM-Driven Grading System for K-12 Non-Native English Learners

Controllable Spoken Dialogue Generation: An LLM-Driven Grading System for K-12 Non-Native English Learners

Paper · 2026 · Haidong Yuan, Haokun Zhao, Wanshi Xu et al.

Summary

The paper addresses a mismatch between general-purpose LLMs and young non-native English learners: chatbot replies often use words above the learner's level. Using China's Standards of English (CSE) and the national curriculum as the reference framework, the authors build a four-level (L1–L4) grading scheme driven mainly by vocabulary difficulty. They compile graded word lists from textbooks and create a multi-turn teacher–student dialogue corpus. Two Qwen2.5-32B models generate the dialogues under vocabulary-constrained decoding. deepseek-v3 then fixes grammar and repetition, and OOV re-checks, rule-based filters and human review follow. Their main method, DDPO (Diversity Driven Policy Optimization), extends GRPO to multi-turn dialogue. It mixes a quality reward with two diversity rewards so the policy does not collapse into bland, repetitive answers. Experiments show that DDPO keeps vocabulary violations low while preserving diversity and naturalness, which plain constrained decoding and SFT fail to do together. This matters for building level-appropriate speaking tutors, and the framework is meant to carry over to other curricula.

Key points

  • Four proficiency tiers (L1–L4) aligned with the CSE rather than CEFR. Inflected forms (tense, plural) are assigned the same level as their root, and proper nouns are removed from the lists to avoid false OOV flags.
  • Corpus pipeline:
    • topics are taken from school exams and textbooks;
    • two vocabulary-constrained Qwen2.5-32B models simulate the teacher and the student;
    • deepseek-v3 rewrites the dialogues without changing their content or vocabulary;
    • a second OOV check, rule-based screening and human review complete the process.
  • DDPO samples G multi-turn trajectories from a fixed history. Multi-sample diversity is computed over the G first-turn replies, and multi-turn diversity over the turns within one trajectory. Both are combined with a task quality reward via dynamic weighting.
  • Ablation on Llama3.1-8B:
    • a GRPO baseline using only the vocabulary constraint reaches the lowest violation rate (2.57%), but its diversity drops to 0.39, semantic richness to 3.28, and average quality to 3.95, which the authors call entropy collapse;
    • adding a model-based quality scorer helps quality, but diversity stays at 0.53 and OOV rises somewhat.
  • OOV ratios are pushed below 10% at every level, with the largest gain at L3. The conclusion claims robustness across Qwen and Llama, while the limitations section mentions only Llama-3.1-8B. The authors also list three further weaknesses: reliance on a user simulator, higher training cost, and static word lists that mishandle derivations and compounds.

Interesting ideas

  • I1. Generate under a hard lexical constraint, let an unconstrained model repair the fluency damage, then re-verify the constraint, so neither the constraint nor fluency is traded away.
  • I2. Measure diversity on two axes: across parallel samples for the same context, and across successive turns of one trajectory. Then reward both, so the model avoids both identical answers and self-repetition.
  • I3. The ablation shows that optimizing only the constraint metric gives the best constraint score and the worst usefulness, so a constraint score should never be reported alone.
  • I4. Grade words at the lemma level and exclude proper nouns, while noting that static lists wrongly flag derived words. Level checking should therefore look through morphology, not surface strings.
  • I5. Simulate a teacher model and a student model conversing on curriculum topics at a set level to mass-produce graded training dialogues.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (back): Mova's graph-to-text generator could produce level-restricted text this way, with a repair step whose output is re-checked against the allowed lemma set before acceptance.
  • I2 (learn): in self-play or LLM-teacher tiers, both diversity signals could be tracked to detect when a trained component collapses into a narrow set of safe outputs.
  • I3 (learn): evaluation could pair every compliance metric with a quality or diversity counter-metric, acting as a negative control against degenerate "safe" solutions.
  • I4 (tokens): the en crate's lemmatizer and morphology could back a graded-vocabulary checker that maps inflections and derivations to their base before lookup.
  • I5 (answer): graded L1–L4 dialogues or questions produced this way could serve as level-stratified test sets for Mova's school-course readers, showing at which proficiency tier comprehension breaks.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China315 works · 43,268 citations🎓Fudan UniversityShanghai · China202 works · 22,509 citations🏢TencentShenzhen · China163 works · 21,032 citations

Also on the byline: China University of Political Science and Law.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 3Research subjects and languages the work is about.Language models & pretraining · 3,894 worksEnglish · 2,379 worksDialogue systems · 654 works

Cite

Cite as Haidong Yuan, Haokun Zhao, Wanshi Xu, Songjun Cao, Qingyu Zhou, Long Ma and Hongjie Fan. 2026. Controllable Spoken Dialogue Generation: An LLM-Driven Grading System for K-12 Non-Native English Learners. arXiv:2604.22542.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2604.22542 · arXiv 2604.22542

@misc{yuan2026controllable,
  title = {{Controllable Spoken Dialogue Generation: An LLM-Driven Grading System for K-12 Non-Native English Learners}},
  author = {Haidong Yuan, Haokun Zhao, Wanshi Xu, Songjun Cao, Qingyu Zhou, Long Ma and Hongjie Fan},
  year = {2026},
  doi = {10.48550/arxiv.2604.22542},
  eprint = {2604.22542},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2604.22542}
}

References

24 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmad Zaki Amiruddin, Mohammad Taufiq Abdul Ghani, Wan Ab Aziz Wan Daud, Nur Hafezah Hussein, Nur Hazimah Mat Yamin, and Yasmin See Abdullah. 2025. Empowering language learners: Innovations in cefr-based language learning applications. Edelweiss Applied Science and Technology, 9(2):352– 359. ↗
  2. Yingming Gao, Baorian Nuchged, Ya Li, and Linkai Peng. 2023. An investigation of applying large language models to spoken language learning. Applied Sciences, 14(1):224. ↗
  3. Jeroen Geertzen, Theodora Alexopoulou, Anna Korhonen, and 1 others. 2013. Automatic linguistic annotation of large scale l2 databases: The ef-cambridge open language database (efcamdat). In Proceedings of the 31st Second Language Research Forum. Somerville, MA: Cascadilla Proceedings Project, pages 240–254. find
  4. Marina Bacher, Sabrina Bacher, and Kateryna Binyt-ska. 2024. A didactical approach to fostering language sensitivity in multilingual classroom settings: The common european framework of reference for Dominik Glandorf, Peng Cui, Detmar Meurers, and Mrinmaya Sachan. 2025. Grammar control in dialogue response generation for language learning chatbots. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 9820– 9839, Albuquerque, New Mexico. Association for Computational Linguistics. erence for languages. The Asia-Pacific Education Researcher, 31(6):667–677. find
  5. Kun Qian, Ryan Shea, Yu Li, Luke Kutszik Fryer, and Zhou Yu. 2023. User adaptive language learning chatbots with a curriculum. In International Conference on Artificial Intelligence in Education, pages 308–313. Springer. find
  6. Jeremy Redlich and Steven Pattison. 2024. Combining cefr and 4c clil frameworks for principled literature in language teaching. NOBEL: Journal of Literature and Language Teaching, 15(1):1–21. ↗
  7. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948. ↗
  8. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. ↗
  9. Desta Haileselassie Hagos, Rick Battle, and Danda B Rawat. 2024. Recent advances in generative ai and large language models: Current status, challenges, and perspectives. IEEE Transactions on Artificial Intelligence. ↗
  10. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. ↗
  11. Nadezhda Stanislavovna Lagutina, Kseniya Vladimirovna Lagutina, Anasta-sya Mikhailovna Brederman, and Natalia Nikolaevna Kasatkina. 2023. Text classification by cefr levels using machine learning methods and bert language model. Modeling and Analysis of Information Systems, 30(3):202–213. ↗
  12. Gladys Tyen, Mark Brenchley, Andrew Caines, and Paula Buttery. 2022. Towards an open-domain chatbot for language practice. In Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022), pages 234–249. find
  13. Shuning Li. 2024. The challenges and dilemmas of spoken english education in the context of the compulsory education system in china. Transactions on Social Science Education and Humanities Research, 11:629–636. ↗
  14. Satoru Uchida, Yuki Arase, and Tomoyuki Kajiwara. 2024. Profiling English sentences based on CEFR levels. ITL-International Journal of Applied Linguistics (Belgium), pages 103–126. ↗
  15. Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. DailyDialog: A manually labelled multi-turn dialogue dataset. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 986–995, Taipei, Taiwan. Asian Federation of Natural Language Processing. ↗
  16. Jingxuan Wu and Carsten Roever. 2021. Proficiency and preference organization in second language mandarin chinese refusals. The Modern Language Journal, 105(4):897–918. ↗
  17. Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. A new dataset and method for automatically grading esol texts. In Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies, pages 180–189.
  18. Jionghao Lin, Zifei Han, Danielle R Thomas, Ashish Gurung, Shivang Gupta, Vincent Aleven, and Kenneth R Koedinger. 2024. How can i get it right? using gpt to rephrase incorrect trainee responses. International journal of artificial intelligence in education, pages 1–27. ↗
  19. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, and 1 others. 2025. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476. ↗
  20. Yun Long, Haifeng Luo, and Yu Zhang. 2024. Evaluating large language models in analysing classroom dialogue. npj Science of Learning, 9(1):60. find
  21. Hanqing Zhang, Haolin Song, Shaoyu Li, Ming Zhou, and Dawei Song. 2023. A survey of controllable text generation using transformer-based pre-trained language models. ACM Computing Surveys, 56(3):1– 37. ↗
  22. John Mendonça, Isabel Trancoso, and Alon Lavie. 2024. Soda-eval: Open-domain dialogue evaluation in the age of LLMs. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 11687– 11708, Miami, Florida, USA. Association for Computational Linguistics. find
  23. Jixiao Zhang and Chunsheng Zuo. 2025. Grpo-lead: A difficulty-aware reinforcement learning approach for concise mathematical reasoning in language models. arXiv preprint arXiv:2504.09696. ↗
  24. Chuan Peng, Jianda Liu, and Hongwen Cai. 2022. Aligning china’s standards of english language ability with the common european framework of ref-B Main Training and Testing Parameters Algorithm 1 Vocabulary-Constraint Quality Reward Function Training Settings find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yuan-2026-controllable-spoken-dialogue-generation-llm/