🧑‍🔬 Markus Freitag — all works

41 works · ← Markus Freitag

1. Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine TranslationMarkus Freitag, George Foster, David Grangier, et al.Machine translation · 661 citations
2. Language Models are Multilingual Chain-of-Thought ReasonersarXiv:2210.03057Freda Shi, Mirac Suzgun, Markus Freitag, et al.2022 · 494 citations
3. Beam Search Strategies for Neural Machine TranslationWorkshop on Neural Machine TranslationMarkus Freitag, Yaser Al-Onaizan2017 · 386 citations
4. Results of WMT22 Metrics Shared Task: Stop Using BLEU – Neural Metrics Are Better and More RobustMarkus Freitag, Ricardo Rei, Nitika Mathur, et al.Machine translation · 265 citations
5. BLEU might be Guilty but References are not InnocentConference on Empirical Methods in Natural Language Processing (EMNLP)Markus Freitag, David Grangier, Isaac Caswell2020 · 225 citations
6. Results of the WMT20 Metrics Shared TaskConference on Machine Translation (WMT)Nitika Mathur, Johnny Wei, Markus Freitag, et al.2020 · 195 citations
7. Findings of the 2021 Conference on Machine Translation (WMT21)Conference on Machine Translation (WMT)Farhad Akhbardeh, Arkady Arkhangorodsky, Magdalena Biesialska, et al.2021 · 189 citations
8. Findings of the WMT24 General Machine Translation Shared Task: The LLM Era Is Here but MT Is Not Solved YetConference on Machine Translation (WMT)Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, et al.2024 · 178 citations
9. Results of the WMT21 Metrics Shared Task: Evaluating Metrics with Expert-based Human Evaluations on TED and News DomainConference on Machine Translation (WMT)Markus Freitag, Ricardo Rei, Nitika Mathur, et al.2021 · 169 citations
10. The Devil Is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation EvaluationConference on Machine Translation (WMT)Patrick Fernandes, Daniel Deutsch, Mara Finkelstein, et al.2023 · 158 citations
11. Results of WMT23 Metrics Shared Task: Metrics Might Be Guilty but References Are Not InnocentConference on Machine Translation (WMT)Markus Freitag, Nitika Mathur, Chi-kiu Lo, et al.2023 · 155 citations
12. INSTRUCTSCORE: Towards Explainable Text Generation Evaluation with Automatic FeedbackConference on Empirical Methods in Natural Language Processing (EMNLP)Wenda Xu, Danqing Wang, Liangming Pan, et al.2023 · 123 citations
13. High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural MetricsTransactions of the Association for Computational Linguistics (TACL)Markus Freitag, David Grangier, Qijun Tan, et al.2022 · 121 citations
14. MetricX-24: The Google Submission to the WMT 2024 Metrics Shared TaskConference on Machine Translation (WMT)Juraj Juraska, Daniel Deutsch, Mara Finkelstein, et al.2024 · 118 citations
15. MetricX-23: The Google Submission to the WMT 2023 Metrics Shared TaskConference on Machine Translation (WMT)Juraj Juraska, Mara Finkelstein, Daniel Deutsch, et al.2023 · 118 citations
16. APE at Scale and Its Implications on MT Evaluation BiasesConference on Machine Translation (WMT)Markus Freitag, Isaac Caswell, Scott Roy2019 · 108 citations
17. Are LLMs Breaking MT Metrics? Results of the WMT24 Metrics Shared TaskConference on Machine Translation (WMT)Markus Freitag, Nitika Mathur, Daniel Deutsch, et al.2024 · 107 citations
18. Ties Matter: Meta-Evaluating Modern Metrics with Pairwise Accuracy and Tie CalibrationConference on Empirical Methods in Natural Language Processing (EMNLP)Daniel Deutsch, George Foster, Markus Freitag2023 · 76 citations
19. Findings of the WMT25 General Machine Translation Shared Task: Time to Stop Evaluating on Easy Test SetsConference on Machine Translation (WMT)Tom Kocmi, Ekaterina Artemova, Eleftherios Avramidis, et al.2025 · 73 citations
20. Learning to Evaluate Translation Beyond English: BLEURT Submissions to the WMT Metrics 2020 Shared TaskConference on Machine Translation (WMT)Thibault Sellam, Amy Pu, Hyung Won Chung, et al.2020 · 55 citations
21. Translating Step-by-Step: Decomposing the Translation Process for Improved Translation Quality of Long-Form TextsConference on Machine Translation (WMT)Eleftheria Briakou, Jiaming Luo, Colin Cherry, et al.2024 · 46 citations
22. Findings of the WMT25 Shared Task on Automated Translation Evaluation Systems: Linguistic Diversity is Challenging and References Still Help ↗Proceedings of the Tenth Conference on Machine Translation · no summary yetAlon Lavie et al.2025 · 39 citations
23. Findings of the Quality Estimation Shared Task at WMT 2024: Are LLMs Closing the Gap in QE? ↗Proceedings of the Ninth Conference on Machine Translation · no summary yetChrysoula Zerva et al.2024 · 34 citations
24. MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding MethodsarXiv:2309.10966Mara Finkelstein; Subhajit Naskar; Mehdi Mirzazadeh; Apurva Shah;, Markus Freitag2023 · 32 citations
25. Training and Meta-Evaluating Machine Translation Evaluation Metrics at the Paragraph Level ↗Proceedings of the Eighth Conference on Machine Translation · no summary yetDaniel Deutsch et al.2023 · 25 citations
26. Human-Paraphrased References Improve Neural Machine Translation ↗Proceedings of the Fifth Conference on Machine Translation · no summary yetMarkus Freitag et al.2020 · 24 citations
27. Discriminative Reordering Extensions for Hierarchical Phrase-Based Machine Translation ↗Proceedings of the 16th Annual Conference of the European Association for Machine Translation · no summary yetMatthias Huck et al.2012 · 23 citations
28. KoBE: Knowledge-Based Machine Translation EvaluationZorik Gekhman, Roee Aharoni, Genady Beryozkin, et al.Machine translation · 20 citations
29. MetricX-25 and GemSpanEval: Google Translate Submissions to the WMT25 Evaluation Shared Task ↗Proceedings of the Tenth Conference on Machine Translation · no summary yetJuraj Juraska et al.2025 · 11 citations
30. Assessing Reference-Free Peer Evaluation for Machine Translation ↗Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · no summary yetSweta Agrawal et al.2021 · 11 citations
31. On the Implications of Verbose LLM Outputs: A Case Study in Translation Evaluation ↗arXiv:2410.00863 · no summary yetEleftheria Briakou et al.2024 · 9 citations
32. Quality-Aware Translation Models: Efficient Generation and Quality Estimation in a Single Model ↗Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · no summary yetChristian Tomani et al.2024 · 9 citations
33. Better punctuation prediction with hierarchical phrase-based translation ↗Proceedings of the 11th International Workshop on Spoken Language Translation: Papers · no summary yetStephan Peitz, Markus Freitag, Hermann Ney2014 · 6 citations
34. Overestimation in LLM Evaluation: A Controlled Large-Scale Study on Data Contamination's Impact on Machine Translation ↗arXiv:2501.18771 · no summary yetMuhammed Yusuf Kocyigit et al.2025 · 5 citations
35. Mind the Gap... or Not? How Translation Errors and Evaluation Details Skew Multilingual ResultsWorkshop on Generation, Evaluation and Metrics (GEM)Jan-Thorsten Peter, David Vilar, Tobias Domhan, et al.2026 · 4 citations
36. Generating Difficult-to-Translate TextsWorkshop on Multilingual Multicultural EvaluationVilém Zouhar, Wenda Xu, Parker Riley, et al.2026 · 3 citations
37. Feeding Two Birds or Favoring One? Adequacy-Fluency Tradeoffs in Evaluation and Meta-Evaluation of Machine Translation ↗arXiv:2509.20287 · no summary yetBehzad Shayegh et al.2025 · 2 citations
38. Feeding Two Birds or Favoring One? Adequacy–Fluency Tradeoffs in Evaluation and Meta-Evaluation of Machine Translation ↗Proceedings of the Tenth Conference on Machine Translation · no summary yetBehzad Shayegh et al.2025 · 2 citations
39. Penalizing Length: Uncovering Systematic Bias in Quality Estimation Metrics ↗arXiv:2510.22028 · no summary yetYilin Zhang et al.2025 · 2 citations
40. Learning from others' mistakes: Finetuning machine translation models with span-level error annotations ↗arXiv:2410.16509 · no summary yetLily H. Zhang et al.2024 · 1 citation
41. MQM Re-Annotation: A Technique for Collaborative Evaluation of Machine TranslationAnnual Meeting of the Association for Computational Linguistics (ACL)Parker Riley, Daniel Deutsch, Mara Finkelstein, et al.2026

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/authors/markus-freitag/works/