Elamur Research⌕
☰
home / papers / 2026 / RABIT: Rationale-Based Distillation Towards Interpretable Automatic Speaking Assessment via a Small Language Model

RABIT: Rationale-Based Distillation Towards Interpretable Automatic Speaking Assessment via a Small Language Model

Paper · 2026 · Bi-Cheng Yan, Hong-Yun Lin, Fu-An Chao et al.

Summary

Automatic speaking assessment (ASA) systems usually feed hand-crafted features into neural graders. These graders can score learners well, but they cannot explain their scores, which limits their teaching value. RABIT addresses this with a two-stage knowledge distillation framework. First, a teacher LLM is prompted with the scoring rubric, an instruction, the expert-assigned score, the test questions, and an ASR transcript of the learner's answer. The teacher returns rationales covering grammar, content, and overall proficiency. Rationales whose implied score disagrees with the expert label are discarded. Second, a small language model with two output heads learns both tasks at once: it predicts a holistic proficiency score and generates the rationale autoregressively. Prompt vectors help stabilize the score estimation. On the picture-description part of the GEPT dataset, RABIT beat several strong baselines, and adding rationale generation during training improved scoring accuracy. The work suggests that compact models can deliver ASA that is both explainable and efficient.

Key points

  • The teachers are LLaMA3.1-8B and LLaMA3.2-3B. Whisper-large-v3 transcribes responses, and scores fall into five categories (1–5).
  • SmolLM2-135M-Instruct is the default student and outperformed its 360M counterpart on all reported metrics, including on unseen-prompt test data.
  • The authors also tried multimodal models (Phi-4-multimodal, QwenAudio) that generate rationales directly from audio, but these produced hallucinated feedback.
  • In qualitative comparisons, the student's rationales mostly matched the teacher's, though the student tended to flag errors rather than rewrite whole sentences.
  • The data are anonymized GEPT responses provided by LTTC. Limitations include dependence on LLM rationale quality and ignoring the picture prompts' visual content.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓National Taiwan Normal UniversityTaipei · Taiwan7 works · 119 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Bi-Cheng Yan, Hong-Yun Lin, Fu-An Chao, Jiun-Ting Li and Berlin Chen. 2026. RABIT: Rationale-Based Distillation Towards Interpretable Automatic Speaking Assessment via a Small Language Model. Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.bea-1.9

@inproceedings{yan2026rabit,
  title = {{RABIT: Rationale-Based Distillation Towards Interpretable Automatic Speaking Assessment via a Small Language Model}},
  author = {Bi-Cheng Yan, Hong-Yun Lin, Fu-An Chao, Jiun-Ting Li and Berlin Chen},
  booktitle = {Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.bea-1.9},
  url = {https://doi.org/10.18653/v1/2026.bea-1.9}
}

References

31 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, and 1 others. 2025. Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras. arXiv preprint arXiv:2503.…
  2. Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Gabriel Martín Blázquez, Guilherme Penedo, Lewis Tunstall, Andrés Marafioti, Hynek Kydlíček, Agustín Piqueres Lajarín, Vaibhav Srivastav, and 1 others. 2025. Smollm2: When smol goes big–data-centric training of a small language model. arXiv preprint ar…
  3. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations.
  4. Advances in neural information processing systems (NIPS), 33:12449–12460. find
  5. Stefano Bannò and Marco Matassoni. 2023. Proficiency assessment of l2 spoken english using wav2vec 2.0. In IEEE Spoken Language Technology Workshop (SLT), pages 1088–1095. IEEE.
  6. Fu-An Chao, Tien-Hong Lo, Tzu-I Wu, Yao-Ting Sung, and Berlin Chen. 2022. 3m: An effective multi-view, multi-granularity, and multi-aspect modeling approach to english pronunciation assessment. In Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), p… find
  7. Lei Chen, Keelan Evanini, and Xie Sun. 2010. Assessment of non-native speech using vowel space characteristics. In IEEE Spoken Language Technology Workshop, pages 139–144. IEEE. find
  8. Nancy F Chen and Haizhou Li. 2016. Computer-assisted pronunciation training: From pronunciation scoring towards spoken language learning. In Proceedings of Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), pages 1–7. IEEE. find
  9. SeongYeub Chu, Jong Woo Kim, Bryan Wong, and Mun Yong Yi. 2025. Rationale behind essay scores: Enhancing s-llm’s multi-trait essay scoring with rationale generated by llms. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 5796–5814. find
  10. Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou. 2023. Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models. arXiv preprint arXiv:2311.07919.
  11. Larry Davis and John M Norris. 2024. Challenges and innovations in speaking assessment. Routledge. find
  12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the conference of the North American chapter of the association for computational linguistics: human language technologies (NAAC…
  13. Keelan Evanini, Maurice Cogan Hauck, and Kenji Hakuta. 2017. Approaches to automated scoring of speaking for k–12 english language proficiency assessments. ETS Research Report Series, 2017(1):1– 11. find
  14. Keelan Evanini and Xinhao Wang. 2013. Automated speech scoring for non-native middle school students with multiple task types. In Proceedings of the Interspeech (INTERSPEECH), pages 2435–2439. find
  15. Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alex Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. 2023. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes. In Findings of the Association for Computati…
  16. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, and 1 others. 2022. Lora: Low-rank adaptation of large language models. Iclr, 1(2):3.
  17. Minhyeok Lee. 2023. Gelu activation function in deep learning: a comprehensive mathematical analysis and performance. arXiv preprint arXiv:2305.12073. — [arXiv:2305.12073](https://arxiv.org/abs/2305.12073) find
  18. Jiun-Ting Li, Tien-Hong Lo, Bi-Cheng Yan, Yung-Chang Hsu, and Berlin Chen. 2023. Graph-enhanced transformer architecture with novel use of cefr vocabulary profile and filled pauses in automated speaking assessment. In SLaTE, pages 109–113. find
  19. Jiun-Ting Li, Bi-Cheng Yan, Tien-Hong Lo, Yi-Cheng Wang, Yung-Chang Hsu, and Berlin Chen. 2024. Automated speaking assessment of conversation tests with novel graph-based modeling on spoken response coherence. In 2024 IEEE Spoken Language Technology Workshop (SLT), pages 825–832. IEEE. find
  20. Tien-Hong Lo, Fu-An Chao, Tzu-I Wu, Yao-Ting Sung, and Berlin Chen. 2024. An effective automated speaking assessment approach to mitigating data scarcity and imbalanced distribution. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 1352–1362. find
  21. Russell Moore, Andrew Caines, Calbert Graham, and Paula Buttery. 2015. Incremental dependency parsing and disfluency detection in spoken learner english. In International Conference on Text, Speech, and Dialogue (TSD), pages 470–479. Springer. find
  22. Zhanming Shen, Hao Chen, Yulei Tang, Shaolin Zhu, Wentao Ye, Xiaomeng Hu, Haobo Wang, Gang Chen, and Junbo Zhao. 2025. Cycle-instruct: Fully seed-free instruction tuning via dual self-training and cycle consistency. In Proceedings of the Conference on Empirical Methods in Natural Language Processing… find
  23. Yaman Kumar Singla, Avyakt Gupta, Shaurya Bagga, Changyou Chen, Balaji Krishnamurthy, and Rajiv Ratn Shah. 2021. Speaker-conditioned hierarchical modeling for automated speech scoring. In Proceedings of the ACM international conference on information & knowledge management (CIKM), pages 1681–1691. find
  24. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  25. Bi-Cheng Yan, Ming-Kang Tsai, and Berlin Chen. 2025. Muffin: Multifaceted pronunciation feedback model with interactive hierarchical neural modeling. IEEE Transactions on Audio, Speech and Language Processing. find
  26. Klaus Zechner and Keelan Evanini. 2019. Automated speaking assessment: Using language technologies to score spontaneous speech. Routledge. find
  27. Klaus Zechner, Derrick Higgins, Xiaoming Xi, and David M Williamson. 2009. Automatic scoring of non-native spontaneous speech in tests of spoken english. Speech communication, 51(10):883–895. find
  28. Wen-Hsuan Peng, Sally Chen, and Berlin Chen. 2024. Enhancing automatic speech assessment leveraging heterogeneous features and soft labels for ordinal classification. In IEEE Spoken Language Technology Workshop (SLT), pages 945–952. IEEE. find
  29. Yao Qian, Patrick Lange, Keelan Evanini, Robert Pugh, Rutuja Ubale, Matthew Mulholland, and Xinhao Wang. 2019. Neural approaches to automated speech scoring of monologue and dialogue responses. In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages… find
  30. Yao Qian, Rutuja Ubale, Matthew Mulholland, Keelan Evanini, and Xinhao Wang. 2018. A prompt-aware neural network approach to content-based scoring of non-native spontaneous speech. In IEEE Spoken Language Technology Workshop (SLT), pages 979– 986. IEEE. find
  31. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International Conference on Machine Learning (ICML), pages 28492–28518. PMLR.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yan-2026-rabit-rationale-based-distillation-interpretable/