Elamur Research⌕
☰
home / papers / 2026 / On the Use of LLMs for Specialised Terminology: A Good Alternative to Corpora?

On the Use of LLMs for Specialised Terminology: A Good Alternative to Corpora?

Summary

Translators in specialised fields usually find target-language equivalents using parallel or comparable corpora and term bases. Building and searching such corpora is slow and technically demanding, and suitable data is often hard to obtain. This study tests whether widely used proprietary LLMs could take over part of this work for English→French terminology. The authors hand-picked 80 terms from NLP and another 80 from a contrasting field, Earth, Environmental and Planetary Sciences (EEPS). Sources included research articles translated by Master's students and the ARTES term base. They queried GPT-4o, GPT-5.2, Claude Sonnet 4.5 and DeepSeek under two prompting strategies, a terminology mode and a translation mode. An expert rated the proposed equivalents, and the models' self-reported confidence was also analysed. Claude Sonnet 4.5 scored highest in its preferred setup, while DeepSeek behaved most consistently across modes and domains. Confidence values were only a weak guide to correctness. The authors conclude that LLMs are useful for generating candidate terms quickly. However, their output must still be checked against corpora and human expertise, so for now they complement rather than replace those resources.

Key points

  • Setup: 4 proprietary models were tested on 160 terms (80 EEPS + 80 NLP). The same terms, ranging from simple to compound and complex, were used for every model and prompt.
  • Evaluation: Outputs were classified as Correct Main Equivalent (CME), Main Equivalent Identified (MEI) or Main Equivalent Not Identified (MENI). For example, the main French equivalent of low-resource language is langue peu dotée.
  • Model ranking and prompting:
    • Claude Sonnet 4.5 performed best (in terminology mode), and DeepSeek was the most stable.
    • GPT-5.2 came in the middle and GPT-4o performed weakest.
    • The GPT models did better in translation mode, while Claude did better in terminology mode, so no single prompting strategy works best for all models.
  • Confidence and domain: Models often reported high confidence even for partially correct or missed equivalents, with Claude adjusting its confidence somewhat better. The domain effect was moderate, with confidence slightly higher in EEPS.
  • Extra test and limitations:
    • Asking Claude to also supply a scientific source and an example sentence lowered its performance in both domains.
    • The study relies on a single expert annotator with no IAA measurement and covers only one language pair.
    • The work was funded by ANR project MaTOS.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{minder2026use,
  title = {{On the Use of LLMs for Specialised Terminology: A Good Alternative to Corpora?}},
  author = {Minder, Joachim and Wisniewski, Guillaume and Kübler, Natalie},
  booktitle = {Proceedings of the 26th Annual Conference of the European Association for Machine Translation (Volume 1)},
  year = {2026},
  url = {https://aclanthology.org/2026.eamt-1.55/}
}

References

35 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aston, Guy. 1999. Corpus use and learning to translate. In Textus. find
  2. Baker, Mona. 1999. The role of corpora in investigating the linguistic behaviour of professional translators. International Journal of Corpus Linguistics, 4:281–298. find
  3. Basturkmen, Helen and Catherine Elder. 2004. The practice of lsp. In Davies, Alan and Catherine Elder, editors, The Handbook of Applied Linguistics, chapter 27, pages 672–694. John Wiley & Sons. find
  4. Bernardini, Silvia. 2022. How to use corpora for translation. In The Routledge Handbook of Corpus Linguistics, page 14. Routledge, 2 edition. find
  5. Bowker, Lynne and Jennifer Pearson. 2002. Working with Specialized Language: A Practical Guide to Using Corpora. Routledge, 09. find
  6. Cabezas-Garcı́a, Melania and Pilar León-Araúz. 2023. Machine versus corpus-based translation of multiword terms. Digital Scholarship in the Humanities, 38(Supplement 1):6–16, 06. find
  7. Castagnoli, Sara, Dragos , Ciobanu, Kerstin Kunz, Natalie Kübler, and Alexandra Volanschi. 2011. Designing a learner translator corpus for training purposes. In Kübler, Natalie, editor, Corpora, Language, Teaching, and Resources: From Theory to Practice, Études contrastives, pages 221–248. Peter… find
  8. Corpas Pastor, Gloria. 2007. Lost in specialised translation: the corpus as an inexpensive and underexploited aid for language service providers. In Proceedings of Translating and the Computer 29, London, UK, November 29-30. Aslib. find
  9. European Master’s in Translation Network. 2022. Emt competence framework 2022. find
  10. Fernandes, Patrick, Daniel Deutsch, Mara Finkelstein, Parker Riley, André Martins, Graham Neubig, Ankush Garg, Jonathan Clark, Markus Freitag, and Orhan Firat. 2023. The devil is in the errors: Leveraging large language models for fine-grained machine translation evaluation. In Koehn, Philipp, Barr… find our card
  11. Gledhill, Christopher and Natalie Kübler. 2016. What can linguistic approaches bring to English for Specific Purposes? ASp - La revue du GERAS, (69):65– 95, March. find
  12. Gollin-Kies, Sandra, {David R.} Hall, and {Stephen H.} Moore. 2015. Language for specific purposes. Palgrave Macmillan, United Kingdom. find
  13. Granger, Sylviane and Marie-Aude Lefer, editors. 2022. Extending the Scope of Corpus-Based Translation Studies. Bloomsbury Advances in Translation. Bloomsbury Academic, London. find
  14. Hendy, Amr, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. 2023. How good are gpt models at machine translation? a comprehensive evaluation. find
  15. Jiao, Wenxiang, Wenxuan Wang, Jen tse Huang, Xing Wang, Shuming Shi, and Zhaopeng Tu. 2023. Is chatgpt a good translator? yes with gpt-4 as the engine. find our card
  16. Kocmi, Tom and Christian Federmann. 2023a. Gemba-mqm: Detecting translation quality error spans with gpt-4. find our card
  17. Kocmi, Tom and Christian Federmann. 2023b. Large language models are state-of-the-art evaluators of translation quality. find
  18. Kübler, Natalie and Mojca Pecman. 2011. ARTES: an online lexical database for research and teaching in specialized translation and communication. In ESS-LLI 2011, International Workshop on Lexical Resources (WoLeR), Ljubljana, Slovenia, August. find
  19. Kübler, Natalie. 2011. Working with different corpora in translation teaching. In Frankenberg-Garcia, Ana, Lynne Flowerdew, , and Guy Aston, editors, New Trends in Corpora and Language Learning, pages 62–80. Continuum. find
  20. Kübler, Natalie and Guy Aston. 2010. Using corpora in translation. In O’Keeffe, Anne and Michael Mc-Carthy, editors, The Routledge Handbook of Corpus Linguistics. Routledge, London, 1 edition. find
  21. Kübler, Natalie, Alexandra Mestivier, and Mojca Pecman. 2018. Teaching specialised translation through corpus linguistics: Translation quality assessment and methodology evaluation and enhancement by experimental approach. Meta, 63(3):807–825. find
  22. Kübler, Natalie, Hanna Martikainen, Alexandra Mestivier, and Mojca Pecman, 2024. Chapter 4. Post-editing neural machine translation in specialised languages: The role of corpora in the translation of phraseological structures, pages 57–78. John Benjamins Publishing Company. find
  23. Loock, Rudy. 2016. La Traductologie de Corpus. Presses Universitaires du Septentrion. find
  24. Lu, Qingyu, Baopu Qiu, Liang Ding, Kanjian Zhang, Tom Kocmi, and Dacheng Tao. 2024. Error analysis prompting enables human-like translation evaluation in large language models. find our card
  25. Lyu, Chenyang, Zefeng Du, Jitao Xu, Yitao Duan, Minghao Wu, Teresa Lynn, Alham Fikri Aji, Derek F. Wong, and Longyue Wang. 2024. A paradigm shift: The future of machine translation lies with large language models. In Calzolari, Nicoletta, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakt… find
  26. Mezeg, Adriana. 2020. Parallel corpora vs bilingual dictionaries: Their usefulness in translator training. In Granger, Sylviane and Marie-Aude Lefer, editors, Translating and Comparing Languages: Corpus-based Insights, volume 6 of Corpora and Language in Use Proceedings, pages 123–140. Presses unive… find
  27. Minder, Joachim, Guillaume Wisniewski, and Natalie Kübler. 2025. Testing LLMs’ capabilities in annotating translations based on an error typology designed for LSP translation: First experiments with ChatGPT. In Bouillon, Pierrette, Johanna Gerlach, Sabrina Girletti, Lise Volkart, Raphael Rubino, Ri… find
  28. Moosa, Ibraheem Muhammad, Rui Zhang, and Wenpeng Yin. 2024. Mt-ranker: Reference-free machine translation evaluation by inter-system ranking. find
  29. OpenAI. 2022. Introducing chatgpt. find
  30. Pecman, Mojca. 2025. Drafting definitions for emerging concepts and terms undergoing semantic shift within the artes knowledge base: A protocol for integrating llms into terminological analysis by experimental approach. Terminologija, 12. find
  31. Scarpa, Federica, 2020. Introducing Specialised Translation, pages 1–109. Palgrave Macmillan UK, 09. find
  32. Siu, Sai Cheong. 2023. Chatgpt and gpt-4 for professional translators: Exploring the potential of large language models in translation, May. Available at SSRN. find
  33. Vilar, David, Markus Freitag, Colin Cherry, Jiaming Luo, Viresh Ratnakar, and George Foster. 2023. Prompting palm for translation: Assessing strategies and performance. find
  34. Wang, Longyue, Chenyang Lyu, Tianbo Ji, Zhirui Zhang, Dian Yu, Shuming Shi, and Zhaopeng Tu. 2023. Document-level machine translation with large language models. find our card
  35. Xu, Wenda, Danqing Wang, Liangming Pan, Zhenqiao Song, Markus Freitag, William Yang Wang, and Lei Li. 2023. Instructscore: Explainable text generation evaluation with finegrained feedback. *Figure 8: 80 terms selected for the Natural Language Processing domain.* *Figure 9: 80 terms selected for the… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/minder-2026-use-llms-specialised-terminology-good/