Elamur Research⌕
☰
home / papers / 2026 / Multilingual Communication in the Asylum Context: Evaluating LLM-Based Machine Translation with Fuzzy Match Augmentation and Adaptive NMT across Resource Conditions under Low-Data Constraints

Multilingual Communication in the Asylum Context: Evaluating LLM-Based Machine Translation with Fuzzy Match Augmentation and Adaptive NMT across Resource Conditions under Low-Data Constraints

Summary

Asylum reception services must communicate across many languages, some of them low-resource, and they often have very little in-domain parallel data. Drawing on data produced by MaTIAS, a project whose name stands for Machine Translation to Inform Asylum Seekers, the authors compare two ways of adding domain knowledge to MT across 14 target languages with different resource levels.

The first approach is in-context learning with LLMs, where prompts are enriched with fuzzy matches (FM) retrieved from a tiny translation memory. This is tested on open-source translation models and on the commercial Gemini Pro, against random examples and zero-shot prompting. The second approach is adaptive NMT with ModernMT, run with and without domain adaptation. Systems are scored with ChrF, BLEU and COMET.

FM-based example selection consistently beat the other prompting strategies, especially for low-resource languages. ModernMT with adaptation stayed strongest overall, but Gemini Pro came close. Because the best-performing LLM is a commercial service, this exposes a quality-versus-data-sovereignty dilemma for privacy-sensitive settings. The results also show that per-language evaluation is essential.

Key points

  • The translation memory holds only 358 sentences. The dataset has 200 messages (781 sentences) from asylum reception communication, with 423 test sentences per target language.
  • The models are TranslateGemma-4b/12b/27b, TowerPlus-9b, Gemini Pro and ModernMT. The open-source models were run inference-only on two A100 GPUs via vLLM.
  • Gemini Pro with FM prompts outscored adaptive ModernMT on 6 of 14 languages. NMT kept a clear lead for most low-resource languages, with Tigrinya a notable exception.
  • Farsi, although classed as high-resource, had the second-lowest scores. This suggests factors like script, training-data representation and domain matter beyond resourcedness.
  • COMET is unreliable for Pashto, Somali and Tigrinya: its correlation with ChrF was near zero for the low-resource group. No human evaluation was performed.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics: Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{moerman2026multilingual,
  title = {{Multilingual Communication in the Asylum Context: Evaluating LLM-Based Machine Translation with Fuzzy Match Augmentation and Adaptive NMT across Resource Conditions under Low-Data Constraints}},
  author = {Moerman, Thomas and Tezcan, Arda and Macken, Lieve},
  booktitle = {Proceedings of the 26th Annual Conference of the European Association for Machine Translation (Volume 1)},
  year = {2026},
  url = {https://aclanthology.org/2026.eamt-1.3/}
}

References

45 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alves, Duarte M., Nuno M. Guerreiro, João Alves, José Pombal, Ricardo Rei, José G. C. de Souza, Pierre Colombo, and André F. T. Martins. 2023. Steering large language models for machine translation with finetuning and in-context learning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11127–11148, Singapore, December. Association for Computational Linguistics. find
  2. Bertoldi, Nicola, Davide Caroselli, and Marcello Federico. 2018. The ModernMT project. In Proceedings of the 21st Annual Conference of the European Association for Machine Translation, page 365, Alicante, Spain, May. European Association for Machine Translation. find
  3. Bouthors, Maxime, Josep Crego, and François Yvon. 2023. Towards example-based NMT with multi-Levenshtein transformers. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1830–1846, Singapore, December. Association for Computational Linguistics. https://github.com/vllm-project/vllm ↗
  4. Briva-Iglesias, Vicent and Sharon O’Brien. 2026. Human-centered AI language technology (HCAILT): An empathetic design framework for reliable, safe and trustworthy multilingual communication. International Journal of Human– Computer Interaction, pages 1–15. Advance online publication. ↗
  5. Bulté, Bram and Arda Tezcan. 2019. Neural fuzzy repair: Integrating fuzzy matches into neural machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1800–1809, Florence, Italy, July. Association for Computational Linguistics. find
  6. Cao, Qian and Deyi Xiong. 2018. Encoding gated translation memory into neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3042–3047, Brussels, Belgium, October-November. Association for Computational Linguistics. find
  7. Farajian, M. Amin, Marco Turchi, Matteo Negri, and Marcello Federico. 2017. Multi-domain neural machine translation through unsupervised adaptation. In Proceedings of the Second Conference on Machine Translation, pages 127–137, Copenhagen, Denmark, September. Association for Computational Linguistics. find our card
  8. Feng, Yang, Shiyue Zhang, Andi Zhang, Dong Wang, and Andrew Abel. 2017. Memory-augmented neural machine translation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1390–1399, Copenhagen, Denmark, September. Association for Computational Linguistics. ↗
  9. Finkelstein, Mara, Isaac Caswell, Tobias Domhan, Jan-Thorsten Peter, Juraj Juraska, Parker Riley, Daniel Deutsch, Geza Kovacs, Cole Dilanni, Colin Cherry, Eleftheria Briakou, Elizabeth Nielsen, Jiaming Luo, Kat Black, Ryan Mullins, Sweta Agrawal, Wenda Xu, Erin Kats, Stephane Jaskiewicz, Markus Freitag, and David Vilar. 2026. TranslateGemma technical report. arXiv preprint arXiv:2601.09012. ↗
  10. Gemini Team, Google. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261. ↗ our card
  11. He, Qiuxiang, Guoping Huang, Qu Cui, Li Li, and Lemao Liu. 2021. Fast and accurate neural machine translation with translation memory. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3170–3180, Online, August. Association for Computational Linguistics. find
  12. Higashiyama, Shohei. 2024. Results of the WAT/WMT 2024 shared task on patent translation. In Proceedings of the Ninth Conference on Machine Translation, pages 118–123, Miami, Florida, USA, November. Association for Computational Linguistics. ↗
  13. Johnson, Jeff, Matthijs Douze, and Hervé Jégou. 2021. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535–547. find our card
  14. Joshi, Pratik, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The state and fate of linguistic diversity and inclusion in the NLP world. In Jurafsky, Dan, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293, Online, July. Association for Computational Linguistics. find our card
  15. Keet, C. Maria and Langa Khumalo. 2023. Contextualising levels of language resourcedness that affect NLP tasks. arXiv preprint arXiv:2309.17035. ↗
  16. Kocmi, Tom, Eleftherios Avramidis, Rachel Bawden, Ondřej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Philipp Koehn, Benjamin Marie, Christof Monz, Makoto Morishita, Kenton Murray, Masaaki Nagata, Toshiaki Nakazawa, Martin Popel, Maja Popović, Mariya Shmatova, and Jun Suzuki. 2023. Findings of the 2023 conference on machine translation (WMT23): LLMs are here but not quite there yet. In Proceedings of the Eighth Conference on Machine Translation, pages 1–42, Singapore, December. Association for Computational Linguistics. find
  17. Kocmi, Tom, Eleftherios Avramidis, Rachel Bawden, Ondřej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Marzena Karpinska, Philipp Koehn, Benjamin Marie, Christof Monz, Kenton Murray, Masaaki Nagata, Martin Popel, Maja Popović, Mariya Shmatova, Steinthór Steingrı́msson, and Vilém Zouhar. 2024. Findings of the WMT24 general machine translation shared task: The LLM era is here but MT is not solved yet. In Proceedings of the Ninth Conference on Machine Translation, pages 1–46, Miami, Florida, USA, November. Association for Computational Linguistics. find our card
  18. Kocmi, Tom, Sweta Agrawal, Ekaterina Artemova, Eleftherios Avramidis, Eleftheria Briakou, Pinzhen Chen, Marzieh Fadaee, Markus Freitag, Roman Grundkiewicz, Yupeng Hou, Philipp Koehn, Julia Kreutzer, Saab Mansour, Stefano Perrella, Lorenzo Proietti, Parker Riley, Eduardo Sánchez, Patricia Schmidtova, Mariya Shmatova, and Vilém Zouhar. 2025a. Findings of the WMT25 multilingual instruction shared task: Persistent hurdles in reasoning, generation, and evaluation. In Proceedings of the Tenth Conference on Machine Translation, pages 414–435, Suzhou, China, November. Association for Computational Linguistics. find
  19. Kocmi, Tom, Ekaterina Artemova, Eleftherios Avramidis, Rachel Bawden, Ondřej Bojar, Konstantin Dranch, Anton Dvorkovich, Sergey Dukanov, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Marzena Karpinska, Philipp Koehn, Howard Lakougna, Jessica Lundin, Christof Monz, Kenton Murray, Masaaki Nagata, Stefano Perrella, Lorenzo Proietti, Martin Popel, Maja Popović, Parker Riley, Mariya Shmatova, Steinthór Steingrı́msson, Lisa Yankovskaya, and Vilém Zouhar. 2025b. Findings of the WMT25 general machine translation shared task: Time to stop evaluating on easy test sets. In Proceedings of the Tenth Conference on Machine Translation, pages 355–413, Suzhou, China, November. Association for Computational Linguistics. find our card
  20. Koehn, Philipp. 2004. Statistical significance tests for machine translation evaluation. In Lin, Dekang and Dekai Wu, editors, Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pages 388–395, Barcelona, Spain, July. Association for Computational Linguistics. find
  21. Lavie, Alon, Greg Hanneman, Sweta Agrawal, Diptesh Kanojia, Chi-Kiu Lo, Vilém Zouhar, Frederic Blain, Chrysoula Zerva, Eleftherios Avramidis, Sourabh Deoghare, Archchana Sindhujan, Jiayi Wang, David Ifeoluwa Adelani, Brian Thompson, Tom Kocmi, Markus Freitag, and Daniel Deutsch. 2025. Findings of the WMT25 shared task on automated translation evaluation systems: Linguistic diversity is challenging and references still help. In Proceedings of the Tenth Conference on Machine Translation, pages 436–483, Suzhou, China, November. Association for Computational Linguistics. find
  22. Macken, Lieve, Margot Fonteyne, Arda Tezcan, Ella van Hest, Katrijn Maryns, and July De Wilde. 2025. Machine translation in asylum reception centres: system selection and multilingual quality evaluation. Revista Tradumàtica: translation technologies, (23):326–349. ↗
  23. Mager, Manuel, Elisabeth Mager, Katharina Kann, and Ngoc Thang Vu. 2023. Ethical considerations for machine translation of indigenous languages: Giving a voice to the speakers. In Rogers, Anna, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4871–4897, Toronto, Canada, July. Association for Computational Linguistics. find
  24. Moerman, Thomas, Tom Vanallemeersch, Sara Szoc, and Arda Tezcan. 2025. Tailoring machine translation for scientific literature through topic filtering and fuzzy match augmentation. In Tsunakawa, Takashi, Katsuhito Sudoh, and Isao Goto, editors, Proceedings of the Eleventh Workshop on Patent and Scientific Literature Translation (PSLT 2025), pages 13–26, Geneva, Switzerland, June. European Association for Machine Translation. find
  25. Moslem, Yasmin, Rejwanul Haque, John D. Kelleher, and Andy Way. 2023a. Adaptive machine translation with large language models. In Proceedings of the 24th Annual Conference of the European Association for Machine Translation, pages 227–237, Tampere, Finland, June. European Association for Machine Translation. ↗ our card
  26. Moslem, Yasmin, Rejwanul Haque, and Andy Way. 2023b. Fine-tuning large language models for adaptive machine translation. arXiv preprint arXiv:2312.12740. ↗
  27. Neves, Mariana, Cristian Grozea, Philippe Thomas, Roland Roller, Rachel Bawden, Aurélie Névéol, Steffen Castle, Vanessa Bonato, Giorgio Maria Di Nunzio, Federica Vezzani, Maika Vicente Navarro, Lana Yeganova, and Antonio Jimeno Yepes. 2024. Findings of the WMT 2024 biomedical translation shared task: Test sets on abstract level. In Proceedings of the Ninth Conference on Machine Translation, pages 124–138, Miami, Florida, USA, November. Association for Computational Linguistics. find
  28. NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang. 2022. No Language Left Behind: Scaling Human-Centered Machine Translation. arXiv preprint arXiv:2207.04672. ↗
  29. Ortega, John, Felipe Sánchez-Martı́nez, and Mikel L. Forcada. 2016. Fuzzy-match repair using black-box machine translation systems: what can be expected? In Green, Spence and Lane Schwartz, editors, Conferences of the Association for Machine Translation in the Americas: MT Researchers’ Track, pages 27– 39, Austin, TX, USA, October. The Association for Machine Translation in the Americas. find
  30. Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA, July. Association for Computational Linguistics. find our card
  31. Popović, Maja. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal, September. Association for Computational Linguistics. find our card
  32. Rei, Ricardo, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. COMET: A neural framework for MT evaluation. In Webber, Bonnie, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685–2702, Online, November. Association for Computational Linguistics. find our card
  33. Rei, Ricardo, Nuno M. Guerreiro, José Pombal, João Alves, Pedro Teixeirinha, Amin Farajian, and André F. T. Martins. 2025. Tower+: Bridging generality and translation specialization in multilingual LLMs. arXiv preprint arXiv:2506.17080. ↗
  34. Salim, Luis Frentzen, Esteban Carlin, Alexandre Mor-invil, Xi Ai, and Lun-Wei Ku. 2026. Beyond many-shot translation: Scaling in-context demonstrations for low-resource machine translation. arXiv preprint arXiv:2602.04764. ↗
  35. Semenov, Kirill, Xu Huang, Vilém Zouhar, Nathaniel Berger, Dawei Zhu, Arturo Oncevay, and Pinzhen Chen. 2025. Findings of the WMT25 terminology translation task: Terminology is useful especially for good MTs. In Proceedings of the Tenth Conference on Machine Translation, pages 554–576, Suzhou, China, November. Association for Computational Linguistics. find
  36. Snover, Matthew, Bonnie Dorr, Rich Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers, pages 223–231, Cambridge, Massachusetts, USA, August. Association for Machine Translation in the Americas. find our card
  37. Strubell, Emma, Ananya Ganesh, and Andrew McCallum. 2019. Energy and policy considerations for deep learning in NLP. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3645–3650, Florence, Italy, July. Association for Computational Linguistics. find our card
  38. Tezcan, Arda, Bram Bulté, and Bram Vanroy. 2021. Towards a better integration of fuzzy matches in neural machine translation through data augmentation. Informatics, 8(1):7. ↗
  39. Tezcan, Arda, Alina Skidanova, and Thomas Moerman. 2024. Improving fuzzy match augmented neural machine translation in specialised domains through synthetic data. The Prague Bulletin of Mathematical Linguistics, 122(122):9–42. ↗
  40. Valdez, Susana and Ana Guerberof-Arenas. 2025. “Google Translate is our best friend here”: A vignette-based interview study on machine translation use for health communication. Translation Spaces, 14(2):253–276. ↗
  41. Valdez, Susana, Floor van Heeswijk, and Noa Warren. 2025. Machine translation at the hospital: Healthcare professionals’ perspectives on use, appropriateness, and policy. Revista Tradumàtica: translation technologies, (23):244–265. ↗
  42. Vieira, Lucas Nunes. 2024. Machine translation and migration. In Maher, Brigid, Loredana Polezzi, and Rita Wilson, editors, The Routledge Handbook of Translation and Migration, pages 221–234. Routledge, London. ↗
  43. Wang, Wenhui, Hangbo Bao, Shaohan Huang, Li Dong, and Furu Wei. 2021. MiniLMv2: Multi-head self-attention relation distillation for compressing pre-trained transformers. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2140–2151, Online, August. Association for Computational Linguistics. find
  44. Wassie, Aman Kassahun, Mahdi Molaei, and Yasmin Moslem. 2024. Domain-specific translation with open-source large language models: Resource-oriented analysis. arXiv preprint arXiv:2412.05862. ↗
  45. Xu, Jitao, Josep Crego, and Jean Senellart. 2020. Boosting neural machine translation with similar translations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1580–1590, Online, July. Association for Computational Linguistics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/moerman-2026-multilingual-communication-asylum-context-evaluating/