🧑‍🔬 Tom Kocmi — all works

26 works · ← Tom Kocmi

1. Large Language Models Are State-of-the-Art Evaluators of Translation Quality ↗Proceedings of the 24th Annual Conference of the European Association for Machine Translation · no summary yetTom Kocmi, Christian Federmann2023 · 459 citations
2. Findings of the 2020 Conference on Machine Translation (WMT20)Conference on Machine Translation (WMT)Loïc Barrault, Magdalena Biesialska, Ondřej Bojar, et al.2020 · 403 citations
3. To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation ↗Proceedings of the Sixth Conference on Machine Translation · no summary yetTom Kocmi et al.2021 · 269 citations
4. Results of WMT22 Metrics Shared Task: Stop Using BLEU – Neural Metrics Are Better and More RobustMarkus Freitag, Ricardo Rei, Nitika Mathur, et al.Machine translation · 265 citations
5. GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4Conference on Machine Translation (WMT)Tom Kocmi, Christian Federmann2023 · 192 citations
6. Findings of the WMT24 General Machine Translation Shared Task: The LLM Era Is Here but MT Is Not Solved YetConference on Machine Translation (WMT)Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, et al.2024 · 178 citations
7. Results of WMT23 Metrics Shared Task: Metrics Might Be Guilty but References Are Not InnocentConference on Machine Translation (WMT)Markus Freitag, Nitika Mathur, Chi-kiu Lo, et al.2023 · 155 citations
8. Trivial Transfer Learning for Low-Resource Neural Machine TranslationConference on Machine Translation: Research PapersTom Kocmi, Ondřej Bojar2018 · 141 citations
9. Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language ModelsFindings of the Association for Computational LinguisticsQingyu Lu, Baopu Qiu, Liang Ding, et al.2024 · 136 citations
10. Are LLMs Breaking MT Metrics? Results of the WMT24 Metrics Shared TaskConference on Machine Translation (WMT)Markus Freitag, Nitika Mathur, Daniel Deutsch, et al.2024 · 107 citations
11. NTREX-128 – News Test References for MT Evaluation of 128 LanguagesWorkshop on Scaling Up Multilingual EvaluationChristian Federmann, Tom Kocmi, Ying Xin2022 · 104 citations
12. Error Span Annotation: A Balanced Approach for Human Evaluation of Machine TranslationConference on Machine Translation (WMT)Tom Kocmi, Vilém Zouhar, Eleftherios Avramidis, et al.2024 · 83 citations
13. Findings of the WMT25 General Machine Translation Shared Task: Time to Stop Evaluating on Easy Test SetsConference on Machine Translation (WMT)Tom Kocmi, Ekaterina Artemova, Eleftherios Avramidis, et al.2025 · 73 citations
14. Findings of the WMT25 Shared Task on Automated Translation Evaluation Systems: Linguistic Diversity is Challenging and References Still Help ↗Proceedings of the Tenth Conference on Machine Translation · no summary yetAlon Lavie et al.2025 · 39 citations
15. AI-Assisted Human Evaluation of Machine Translation ↗Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · no summary yetVilém Zouhar, Tom Kocmi, Mrinmaya Sachan2025 · 31 citations
16. Machine Translation Meta Evaluation through Translation Accuracy Challenge SetsComputational LinguisticsNikita Moghe, Arnisa Fazla, Chantal Amrhein, et al.2025 · 28 citations
17. MS-COMET: More and Better Human Judgements Improve Metric Performance ↗Proceedings of the Seventh Conference on Machine Translation (WMT) · no summary yetTom Kocmi, Hitokazu Matsushita, Christian Federmann2022 · 20 citations
18. Cometoid: Distilling Strong Reference-based Machine Translation Metrics into Even Stronger Quality Estimation Metrics ↗Proceedings of the Eighth Conference on Machine Translation · no summary yetThamme Gowda, Tom Kocmi, Marcin Junczys-Dowmunt2023 · 16 citations
19. SLIDE: Reference-free Evaluation for Machine Translation using a Sliding Document Window ↗Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers) · no summary yetVikas Raunak, Tom Kocmi, Matt Post2024 · 15 citations
20. Poor Man's Quality Estimation: Predicting Reference-Based MT Metrics Without the Reference ↗Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · no summary yetVilém Zouhar et al.2023 · 15 citations
21. Pearmut: Human Evaluation of Translation Made Trivial ↗arXiv:2601.02933 · no summary yetVilem Zouhar, Tom Kocmi2026 · 11 citations
22. Searching for a Higher Power in the Human Evaluation of MT ↗Proceedings of the Seventh Conference on Machine Translation (WMT) · no summary yetJohnny Wei, Tom Kocmi, Christian Federmann2022 · 10 citations
23. Evaluating Metrics for Document-context Evaluation in Machine Translation ↗Proceedings of the Eighth Conference on Machine Translation · no summary yetVikas Raunak, Tom Kocmi, Matt Post2023 · 9 citations
24. eBLEU: Unexpectedly Good Machine Translation Evaluation Using Simple Word Embeddings ↗Proceedings of the Eighth Conference on Machine Translation · no summary yetMuhammad ElNokrashy, Tom Kocmi2023 · 8 citations
25. On User Interfaces for Large-Scale Document-Level Human Evaluation of Machine Translation Outputs ↗Proceedings of the Workshop on Human Evaluation of NLP Systems (HumEval) · no summary yetRoman Grundkiewicz et al.2021 · 4 citations
26. Contrastive ESA: Human Evaluation of Multiple Translations at Once ↗arXiv:2607.26640 · no summary yetVilem Zouhar et al.2026 · 1 citation

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/authors/tom-kocmi/works/