Elamur Research⌕
☰
home / papers / 2026 / CEFR-Cymraeg: A Dataset and Baseline Models for Language Proficiency Assessment in Welsh

CEFR-Cymraeg: A Dataset and Baseline Models for Language Proficiency Assessment in Welsh

Summary

Most work on automatic CEFR level prediction targets major languages, and Welsh has had almost no openly available CEFR-labelled text. The authors fill this gap with CEFR-Cymraeg, a corpus of 2,658 texts taken from the adult "Learn Welsh" coursebooks of the National Centre for Learning Welsh, covering levels A1 to B2. Native-speaker annotators extracted segments manually or semi-automatically, and a teacher of Welsh as a second language checked them. The texts are stored at four granularities (sentence, dialogue, paragraph, document) in the UniversalCEFR format for cross-lingual compatibility. The authors frame level assignment as multi-class classification and fine-tune EuroBERT and the decoder model BritLLM-3B. Because the training set is small, they add back-translated copies that stay highly similar to the originals. The strongest system reaches a weighted F1 of 0.83. Manual error inspection suggests that many misclassifications reflect legitimate variation in the teaching materials rather than model faults. The resource gives Welsh its first CEFR benchmark and supports educational tools, corpus research and policy goals such as Cymraeg 2050.

Key points

  • CEFR-Cymraeg contains 2,658 gold-labelled entries for levels A1–B2. C1–C2 are not included because no digital teaching materials exist for those levels.
  • The source material comes only from the South Wales variant of the coursebooks (Mynediad, Sylfaen, Canolradd, Uwch 1–3), mixing version 1 and version 2 editions. Formal permission was obtained to redistribute the extracted text.
  • The baselines are EuroBERT, fine-tuned with a classification head, and BritLLM-3B, a causal decoder, both evaluated across folds. Training data is augmented with Welsh→English→Welsh back-translation.
  • The best model achieves a weighted F1 of 0.83. The authors highlight large-capacity BritLLM as handling the low-resource setting well.
  • The dataset and models are released in the Hugging Face collection cardiffnlp/welsh-cefr.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

1 paper of our library cites it (of the 140673 whose bibliographies we read); each has a page here.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2dvuy5ucr9g2

@inproceedings{waqar2026cefr,
  title = {{CEFR-Cymraeg: A Dataset and Baseline Models for Language Proficiency Assessment in Welsh}},
  author = {Waqar, Eeshan and Davies, Jonathan and Knight, Dawn and Alva-Manchego, Fernando},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/2dvuy5ucr9g2},
  url = {https://doi.org/10.63317/2dvuy5ucr9g2}
}

References

17 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Nicolas Boizard, Hippolyte Gisserot-Boukhlef, Duarte Miguel Alves, Andre Martins, Ayoub Hammal, Caio Corro, CELINE HUDELOT, Emmanuel Malherbe, Etienne Malaboeuf, Fanny Jourdan, Gabriel Hautreux, João Alves, Kevin El Haddad, Manuel Faysse, Maxime Peyrard, Nuno M Guerreiro, Patrick Fernandes, Ricardo… find
  2. Zoltán Dörnyei. 1997. Psychological processes in cooperative language learning: Group dynamics and motivation. The Modern Language Journal, 81(4):482–493. find
  3. Neus Figueras. 2012. The impact of the CEFR. ELT journal, 66(4):477–485. find
  4. Nicolás Gutiérrez-Rolón, Jonathan Davies, Tomos Williams, Dawn Knight, and Fernando Alva-Manchego. 2026. Proffiliadur: Welsh language text profiling toolkit. In Proceedings of the Fifteenth Language Resources and Evaluation Conference, Palma, Mallorca. European Language Resources Association. find our card
  5. Claudia Harsch. 2014. General Language Proficiency Revisited: Current and Future Issues. Language Assessment Quarterly, 11(2):152– 169. find
  6. Joseph Marvin Imperial, Abdullah Barayan, Regina Stodden, Rodrigo Wilkens, Ricardo Munoz Sanchez, Lingyun Gao, Melissa Torgbi, Dawn Knight, Gail Forey, Reka R. Jablonkai, Ekaterina Kochmar, Robert Reynolds, Eugenio Ribeiro, Horacio Saggion, Elena Volodina, Sowmya Vajjala, Thomas Francois, Fernando A… find
  7. Marcin Junczys-Dowmunt, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, André F. T. Martins, and Alexandra Birch. 2018. Marian: Fast neural machine translation in C++. CoRR, abs/1804.00344. find our card
  8. Stephen D. Krashen. 1982. Acquiring a second language. World Englishes, 1(3):97–101. find
  9. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational… find our card
  10. Maja Popović. 2015. chrF: character n-gram Fscore for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics. find
  11. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,… find
  12. Kevin P. Scannell. 2007. The crúbadán project: Corpus building for under-resourced languages. find
  13. Shivalika Singh, Freddie Vargus, Daniel Dsouza, Börje F. Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura OMahony, Mike Zhang, Ramith Hettiarachchi, Joseph Wilson, Marina Machado, Luisa Souza Moura, Dominik Krzemiński, Hakimeh Fadaei, Irem Ergün, Ifeo… find our card
  14. Sowmya Vajjala and Ivana Lucic. 2018. Onestopenglish corpus: A new corpus for automatic readability assessment and text simplification. pages 297–304. find our card
  15. Sowmya Vajjala and Taraka Rama. 2018a. Experiments with universal CEFR classification. In Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 147–153, New Orleans, Louisiana. Association for Computational Linguistics. find
  16. Sowmya Vajjala and Taraka Rama. 2018b. Experiments with universal CEFR classification. CoRR, abs/1804.06636. find
  17. Menglin Xia, Ekaterina Kochmar, and Ted Briscoe. 2016. Text readability assessment for second language learners. In Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications, pages 12–22, San Diego, CA. Association for Computational Linguistics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/waqar-2026-cefr-cymraeg-dataset-baseline-models/