Elamur Research⌕
☰
home / papers / 2026 / HFT at BEA 2026 Shared Task 2: Blunt-Edge Models for Hybrid Grading

HFT at BEA 2026 Shared Task 2: Blunt-Edge Models for Hybrid Grading

Paper · 2026 · Ulrike Pado

Summary

The paper addresses automatic grading of German short answers in the BEA 2026 Shared Task 2. It deliberately builds a low-cost, replicable "blunt-edge" grader rather than an optimized system. Seven open-source LLMs hosted by GWDG Göttingen (e.g., gpt-oss-120b, Llama-3.3-70B-Instruct, Qwen3, Mistral-Large-3, Gemma-3, SauerkrautLM) were each tested with four German zero-shot prompts of increasing complexity. On their own, these graders are mediocre. They reach about 0.48 QWK, are competitive in the Unseen Question track, and rank last in the Unseen Answer track.

The author then simulates a hybrid workflow:

  • The model's more precise class ("Incorrect") is accepted without review.
  • Answers predicted "Correct" go to human review.

This approach cuts manual effort substantially while matching or beating cutting-edge systems, especially when little task information is available. The work matters for practical use for three reasons:

  • It keeps a human in the loop, which is relevant under the EU AI Act.
  • It protects data privacy by relying on open-source models.
  • It shows teachers recurring patterns in student answers.

The main caveat is that the remaining errors fall entirely on students wrongly marked incorrect.

Key points

  • Setup. Seven models were run at temperature 0.01 through RAGAS. They were tested with four prompts: Baseline, Fan et al., Role, and a committee-of-experts prompt from Wang et al.
  • Prompt and model results. Simpler prompts that add a little context worked best. The Wang et al. prompt sharply hurt some models. The top trial score was Llama with the Fan et al. prompt (0.4927 QWK).
  • Test results. GPT-OSS with the Fan et al. prompt kept about 0.48 QWK, placing 5th of 9 in the 2-way Unseen Question task and last in Unseen Answer.
  • Rubric coverage. Adding the "Partially Correct" rubrics, counted as Incorrect, raised QWK by about 0.03 across all configurations.
  • Hybrid simulation. On 7072 training answers, 5103 "Incorrect" predictions were accepted and 1969 were reviewed, cutting human workload by 71%.
    • If human review is perfect, overall error is 12% and weighted F1 is 0.896.
    • If reviewers err on 15% of the answers they check, F1 is 0.828, which would rank 1st in Unseen Questions and 8th in Unseen Answers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Stuttgart Technical University of Applied SciencesStuttgart · Germany2 works · 1 citation

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Ulrike Pado. 2026. HFT at BEA 2026 Shared Task 2: Blunt-Edge Models for Hybrid Grading. Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.bea-1.86

@inproceedings{pado2026hft,
  title = {{HFT at BEA 2026 Shared Task 2: Blunt-Edge Models for Hybrid Grading}},
  author = {Ulrike Pado},
  booktitle = {Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.bea-1.86},
  url = {https://doi.org/10.18653/v1/2026.bea-1.86}
}

References

22 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Xiaoyu Bai and Manfred Stede. 2023. A survey of current machine learning approaches to student free-text evaluation for intelligent tutoring. Int. J. Artif. Intell. Educ., 33(4):992–1030. find
  2. Sumit Basu, Chuck Jacobs, and Lucy Vanderwende. 2013. Powergrading: a clustering approach to amplify human effort for short answer grading. Transactions of the Association for Computational Linguistics, 1:391–402. find
  3. Steven Burrows, Iryna Gurevych, and Benno Stein. 2015. The eras and trends of automatic short answer grading. International Journal of Artificial Intelligence in Education, 25:60–117. find
  4. Yuming Fan, Chuangchuang Tan, and Wenyu Song. 2025. BJTU at BEA 2025 shared task: Task-aware prompt tuning and data augmentation for evaluating AI math tutors. In Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025), pages 1073–1077, Vienna, Aust… find
  5. Anna Filighera, Siddharth Parihar, Tim Steuer, Tobias Meuser, and Sebastian Ochs. 2022. Your answer is incorrect... would you like to know why? Introducing a bilingual short answer feedback dataset. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, pages 857… find
  6. Sebastian Gombert, Zhifan Sun, Fabian Zehner, Jannik Lossjew, Tobias Wyrwich, Berrit Katharina Czinczel, David Bednorz, Sascha Bernholt, Knut Neumann, Ute Harms, Aiso Heinze, and Hendrik Drachsler. 2026. Report on the BEA 2026 Shared Task on Rubric-based Short Answer Scoring for German. In Proceedin…
  7. Santiago Góngora, Ignacio Sastre, Santiago Robaina, Ignacio Remersaro, Luis Chiruzzo, and Aiala Rosá. 2025. RETUYT-INCO at BEA 2025 shared task: How far can lightweight models go in AI-powered tutor evaluation? In Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational App… find
  8. Andrea Horbach, Sebastian Stennmanns, and Torsten Zesch. 2018. Cross-lingual content scoring. In Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 410–419. find
  9. Alexander Kwako, Susan Lottridge, and Christopher Ormerod. 2026. Using confidence modeling to optimize overall score quality in hybrid scoring systems. Educational Measurement: Issues and Practice, 45(2):e70019. find
  10. Tim Metzler, Paul G. Plöger, and Jörn Hees. 2024. Computer-assisted short answer grading using large language models and rubrics. In INFORMATIK, Lecture Notes in Informatics (LNI). Gesellschaft für Informatik. find
  11. Detmar Meurers, Ramon Ziai, Niels Ott, and Stacey Bailey. 2011. Integrating parallel analysis modules to evaluate the meaning of answers to reading comprehension questions. International Journal of Continuing Engineering Education and Life-Long Learning, 21(4):355–369. find
  12. Margot Mieskes and Ulrike Padó. 2018. Work smart - reducing effort in short-answer grading. In Proceedings of the 7th Workshop on NLP for Computer Assisted Language Learning, pages 57–68. find
  13. Tanya Nazaretsky, Moriah Ariely, Mutlu Cukurova, and Giora Alexandron. 2022. Teachers’ trust in AI-powered educational technology and a professional development program to improve it. British Journal of Educational Technology, 53(4):914–931. find
  14. Ulrike Padó. 2016. Get semantic with me! The usefulness of different feature types for short-answer grading. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2186–2195. find
  15. Ulrike Padó. 2026. Hybrid evaluation of tutor dialogues. In Agentic learning Ecoystems and ITS, 22nd International Conference (ITS 2026). Springer Nature Switzerland. To appear. find
  16. Ulrike Padó, Yunus Eryilmaz, and Larissa Kirschner. 2024. Short-answer grading for German: Addressing the challenges. International Journal of Artificial Intelligence in Education, 34(4). find
  17. Ulrike Padó and Cornelia Kiefer. 2015. Short answer grading: When sorting helps and when it doesn’t. In Proceedings of the Workshop on NLP for Computer-Aided Language Learning, pages 42––50, Vilnius, Lithuania. find
  18. J. Schneider, R. Richner, and M. Riser. 2023. Towards trustworthy autograding of short, multi-lingual, multi-type answers. Int J Artif Intell Educ 33, 33:88–118. find
  19. Sebastian Speiser and Annegret Weng. 2024. Enhancing short answer grading with OpenAI APIs. In IEEE International Conference on IT in Higher Education and Training (ITHET). find
  20. Pierpaolo Vittorini, Stefano Menini, and Sara Tonelli. 2021. An AI-based system for formative and summative assessment in Data Science courses. International Journal of Artificial Intelligence in Education, 31:159–185. find
  21. Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. 2024. Unleashing the emergent cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration. In Proceedings of the 2024 Conference of the North American Chapter of the Association…
  22. Fabian Zehner, Christine Sälzer, and Frank Goldhammer. 2016. Automatic coding of short text responses via clustering in educational assessment. Educational and psychological measurement, 2(76):280– 303. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pado-2026-hft-bea-2026-shared-task/