Elamur Research⌕
☰
home / papers / 2026 / uogal at BEA 2026 Shared Task 1: Ensemble of Multilingual Encoders with NMT Augmentation for L1-Aware Vocabulary Difficulty Prediction

uogal at BEA 2026 Shared Task 1: Ensemble of Multilingual Encoders with NMT Augmentation for L1-Aware Vocabulary Difficulty Prediction

Paper · 2026 · Bernardo Stearns, John P. McCrae, Thomas Gaillat et al.

Summary

The uogal team took part in the BEA 2026 shared task, which asks systems to predict vocabulary difficulty while taking the learner's first language into account. The task is to estimate how hard an English word is for learners whose first language is Spanish, German or Mandarin Chinese. The data come from the British Council's Knowledge-based Vocabulary Lists (KVL), where each word has a GLMM difficulty score. The authors compared three system families on the closed track:

  • tabular features fed to XGBoost/LightGBM,
  • fully fine-tuned multilingual encoders,
  • Pythia decoders adapted to learner English, used as artificial learners.

Each family was tested with and without English contexts generated by machine translation. The best and simplest setup was a stacked ensemble of eight encoders: four strong base models plus versions of the same four whose input also contained six NMT translations of the L1 context. This system took second place in the closed track and clearly beat the organizers' baseline. The results suggest that large multilingual encoders can learn L1-specific difficulty without being told the L1 explicitly, and that translation-based augmentation adds useful diversity to an ensemble.

Key points

  • Data: KVL contains 6,768 parallel items per L1, with about 6,091 for training and 677 for development. Inputs are the English word, a letter clue, the L1 translation and an L1 context sentence.
  • Encoder comparison: 11 encoders were tested. The top four were E5-Large, XLM-R Large, RemBERT and InfoXLM-Large, with RemBERT the strongest single model on dev (RMSE 0.980).
  • NMT augmentation: OPUS-MT, NLLB-600M and NLLB-1.3B each produced a free and a forced decode. This improved RMSE for three of the four models, though it slightly hurt results for Chinese.
  • Ensemble results: The eight-model ensemble used per-language weights fitted with Elastic-Net and Nelder-Mead stacking. It reached dev RMSE of 0.891/0.902 and test Pearson of 0.859 (Es), 0.869 (De) and 0.880 (Cn).
  • Artificial learners: Pythia models from 70M to 1.4B parameters were trained on EFCAMDAT and adapted with LoRA. Their RMSE improved steadily with size (1.512 → 1.030) but never matched the encoders.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Ollscoil na Gaillimhe – University of GalwayGalway · Ireland23 works · 2,229 citations🔬Insight SFI Research Centre for Data AnalyticsDublin · Ireland9 works · 543 citations🔬Linguistique Ingénierie et Didactique des LanguesRennes · France4 works · 14 citations🎓Université Rennes 2Rennes · France3 works · 9 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Machine translation · 1,087 worksMultilingual & cross-lingual NLP · 828 works

Cite

Cite as Bernardo Stearns, John P. McCrae, Thomas Gaillat and Jefkine Kafunah. 2026. uogal at BEA 2026 Shared Task 1: Ensemble of Multilingual Encoders with NMT Augmentation for L1-Aware Vocabulary Difficulty Prediction. Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.bea-1.75

@inproceedings{stearns2026uogal,
  title = {{uogal at BEA 2026 Shared Task 1: Ensemble of Multilingual Encoders with NMT Augmentation for L1-Aware Vocabulary Difficulty Prediction}},
  author = {Bernardo Stearns, John P. McCrae, Thomas Gaillat and Jefkine Kafunah},
  booktitle = {Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.bea-1.75},
  url = {https://doi.org/10.18653/v1/2026.bea-1.75}
}

References

41 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tatsuya Aoyama and Nathan Schneider. 2024. Modeling nonnative sentence processing with L2 language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 4927–4940, Miami, Florida, USA. Association for Computational Linguistics. find
  2. Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, Usvsn Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar Van Der Wal. 2023. Pythia: A suite for analyzing large language models across… find
  3. BEA 2026 shared task: Vocabulary difficulty prediction for English learners. https://www.britishcouncil.org/ data-science-and-insights/bea2026st. Data Science and Insights, British Council. Accessed 2026-04-22. ↗
  4. Christopher Bryant, Mariano Felice, Øistein E. Andersen, and Ted Briscoe. 2019. The BEA-2019 shared task on grammatical error correction. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 52–75, Florence, Italy. Association for Computatio…
  5. Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 785–794. find
  6. Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, and Ming Zhou. 2021a. InfoXLM: An information-theoretic framework for cross-lingual language model pre-training. In Proceedings of the 2021 Conference of the North American Chapter of the Asso…
  7. Zewen Chi, Li Dong, Bo Zheng, Shaohan Huang, Xian-Ling Mao, Heyan Huang, and Furu Wei. 2021b. Improving pretrained cross-lingual language models via self-labeled word alignment. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Join… find
  8. Hyung Won Chung, Thibault Fevry, Henry Tsai, Melvin Johnson, and Sebastian Ruder. 2021. Rethinking embedding coupling in pre-trained language models. In International Conference on Learning Representations.
  9. Jonathan H. Clark, Dan Garrette, Iulia Turc, and John Wieting. 2022. Canine: Pre-training an efficient tokenization-free encoder for language representation. Transactions of the Association for Computational Linguistics, 10:73–91.
  10. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association fo… find
  11. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  12. Matthew S. Dryer and Martin Haspelmath, editors. 2013. WALS Online (v2020.4). Zenodo. find
  13. Christiane Fellbaum, editor. 1998. WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA. find
  14. Jeroen Geertzen, Theodora Alexopoulou, and Anna Korhonen. 2013. Automatic linguistic annotation of large scale L2 databases: The EF-Cambridge Open Language Database (EFCAMDAT). In Proceedings of the 31st Second Language Research Forum (SLRF). find
  15. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  16. Jingying Hu and Yan Cong. 2025. Modeling Chinese L2 writing development: The LLM-surprisal perspective. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pages 172– 183, Albuquerque, New Mexico, USA. Association for Computational Linguistics. find
  17. Scott Jarvis and Aneta Pavlenko. 2008. Crosslinguistic influence in language and cognition, 1 edition. Routledge, New York. find
  18. Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. LightGBM: A highly efficient gradient boosting decision tree. In Advances in Neural Information Processing Systems, volume 30. find
  19. Batia Laufer. 1997. The lexical plight in second language reading: Words you don’t know, words you think you know, and words you can’t guess. In James Coady and Thomas Huckin, editors, Second Language Vocabulary Acquisition, pages 20–34. Cambridge University Press. find
  20. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692.
  21. David R. Mortensen, Patrick Littell, Akash Bharadwaj, Kartik Goyal, Chris Dyer, and Lori Levin. 2016. Pan-Phon: A resource for mapping IPA segments to articulatory feature vectors. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages… find
  22. John A. Nelder and Roger Mead. 1965. A simplex method for function minimization. The Computer Journal, 7(4):308–313. find
  23. NLLB Team, Marta R. Costa-jussà, James Cross, Onur Celebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Baez, Gabriel Birber, Johan Boscolo, and 51 others. 2022. No l… find
  24. Gustavo Paetzold and Lucia Specia. 2016. find
  25. SemEval 2016 task 11: Complex word identification. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 560–569, San Diego, California. Association for Computational Linguistics.
  26. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesnay. 2011. Scikit-learn:… find
  27. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, pages 3982–3992.
  28. Nils Reimers and Iryna Gurevych. 2020. Making monolingual sentence embeddings multilingual using knowledge distillation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).
  29. Norbert Schmitt, Karen Dunn, Barry O’Sullivan, Laurence Anthony, and Benjamin Kremmel. 2021. Introducing knowledge-based vocabulary lists (KVL). TESOL Journal, 12(4):e622. find
  30. Matthew Shardlow, Fernando Alva-Manchego, Riza Batista-Navarro, Stefan Bott, Saul Calderon Ramirez, Rémi Cardon, Thomas François, Akio Hayakawa, Andrea Horbach, Anna Hülsing, Yusuke Ide, Joseph Marvin Imperial, Adam Nohejl, Kai North, Laura Occhipinti, Nelson Peréz Rojas, Nishat Raihan, Tharindu Ran… find
  31. The BEA 2024 shared task on the multilingual lexical simplification pipeline. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), pages 571–589, Mexico City, Mexico. Association for Computational Linguistics. find
  32. Matthew Shardlow, Richard Evans, Gustavo Henrique Paetzold, and Marcos Zampieri. 2021. SemEval-2021 task 1: Lexical complexity prediction. In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1–16, Online. Association for Computational Linguistics.
  33. Robyn Speer. 2022. wordfreq: v3.0. find
  34. Felix Stahlberg and Shankar Kumar. 2021. Synthetic data generation for grammatical error correction with tagged corruption models. In Proceedings of the 16th workshop on innovative use of NLP for building educational applications, pages 37–47. find
  35. Bernardo Stearns, Nicolas Ballier, Thomas Gaillat, Andrew Simpkin, and John P. McCrae. 2024. Evaluating the generalisation of an artificial learner. In Proceedings of the 13th Workshop on Natural Language Processing for Computer Assisted Language Learning, pages 199–208, Rennes, France. LiU Electron… find
  36. Jörg Tiedemann and Santhosh Thottingal. 2020. OPUS-MT – building open translation services for the world. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 479–480, Lisboa, Portugal. European Association for Machine Translation. find
  37. Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern,… — SciPy 1.0 — Fundamental Algorithms for Scientific Computing in Python
  38. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533.
  39. Seid Muhie Yimam, Chris Biemann, Shervin Malmasi, Gustavo Paetzold, Lucia Specia, Sanja Štajner, Anaïs Tack, and Marcos Zampieri. 2018. A report on the complex word identification shared task 2018. In Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applicatio… find
  40. Hui Zou and Trevor Hastie. 2005. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(2):301–320. find
  41. Configuration r RMSE Es De Cn Avg r RMSE all_concat_als_xgb all_features_lgbm all_features_xgb all_concat_ipa_xgb all_features_full_xgb all_ipa_phonetic_xgb cvo_allconcat_xgb all_features_lr cvo_allconcat_ridge cvo_embedding_ridge als_nonmt_pythia410m_xgb cefr_wordnet_only_xgb ipa_phonetic_only_xgb… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/stearns-2026-uogal-bea-2026-shared-task/