Elamur Research⌕
☰
home / papers / 2025 / Please Translate Again: Two Simple Experiments on Whether Human-Like Reasoning Helps Translation

Please Translate Again: Two Simple Experiments on Whether Human-Like Reasoning Helps Translation

Paper · 2025 · Di Wu, Seth Aycock and Christof Monz

Summary

The paper asks whether LLM translation actually benefits from human-style reasoning. In that approach, the model first analyses the source text (idioms, meanings and similar aspects), then drafts a translation, then refines it. The authors re-implement the four-step Step-by-step prompt of Briakou et al. (2024), a state-of-the-art method on WMT24. They compare it with a minimal alternative, "Translate again": the model translates directly and is then asked three more times, with the conversation history, to produce a better version. Both run on GPT-4o-mini and Gemini-2.0-Flash over WMT24++ at segment and paragraph level.

The first experiment finds that the decomposition step adds little. Its only gains are marginal and mostly at paragraph level with Gemini. Plain self-refinement accounts for most of the improvement, and one round of "translate again" gives the strongest scores. The second experiment uses an attribution check and shows that translations usually do follow the decomposition. However, this faithfulness hurts quality about as often as it helps, possibly because errors in the analysis carry over into the translation.

The work suggests that LLMs and human translators benefit from different strategies. It also argues that CoT-free refinement should be a standard baseline.

Key points

  • Test data: WMT24++, post-edited references, 8 directions from English (cs, de, fr, he, ja, ru, uk, zh).
    • Segment level: 960 shared source segments.
    • Paragraph level: 221 paragraphs, capped at 150 tokens.
  • Metrics: COMET-22 (reference-based) and CometKiwi-XL (reference-free) in the main results; XCOMET-XL and MetricX-23-XL in the appendix.
  • Faithfulness analysis:
    • GPT-4o-mini outputs are split into Improved, Comparable and Degraded groups using a ±0.5 COMET difference against direct translation.
    • GPT-4o judges whether each change can be traced back to the decomposition.
    • The share of decomposition-influenced translations is similar in the improved and degraded groups.
  • Main finding: The simple "Translate again" prompt matches or beats the Step-by-step baseline at both segment and paragraph level.
  • Released resource: Code plus about 223k segment and paragraph translations covering 8 language pairs, 2 models and 4 steps, including en→uk.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of AmsterdamAmsterdam · Netherlands144 works · 173,906 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Di Wu, Seth Aycock and Christof Monz. 2025. Please Translate Again: Two Simple Experiments on Whether Human-Like Reasoning Helps Translation. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.emnlp-main.1031

@inproceedings{wu2025please,
  title = {{Please Translate Again: Two Simple Experiments on Whether Human-Like Reasoning Helps Translation}},
  author = {Di Wu, Seth Aycock and Christof Monz},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.1031},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.1031}
}

References

28 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Seth Aycock, David Stap, Di Wu, Christof Monz, and Khalil Sima’an. 2025. Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book? In The Thirteenth International Conference on Learning Representations. find
  2. Eleftheria Briakou, Jiaming Luo, Colin Cherry, and Markus Freitag. 2024. Translating step-by-step: Decomposing the translation process for improved translation quality of long-form texts. In Proceedings of the Ninth Conference on Machine Translation, pages 1301–1317, Miami, Florida, USA. Association…
  3. Pinzhen Chen, Zhicheng Guo, Barry Haddow, and Kenneth Heafield. 2024a. Iterative translation refinement with large language models. In Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 1), pages 181–190, Sheffield, UK. European Association for Mach…
  4. Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. 2024b. Teaching large language models to self-debug. In The Twelfth International Conference on Learning Representations. find
  5. Linguistics: NAACL 2025, pages 3922–3938, Albuquerque, New Mexico. Association for Computational Linguistics. find
  6. Google. 2024. Introducing Gemini 2.0: Our new AI model for the agentic era. find
  7. Nuno M. Guerreiro, Ricardo Rei, Daan van Stigt, Luisa Coheur, Pierre Colombo, and André F. T. Martins. 2024. xcomet: Transparent machine translation evaluation through fine-grained error detection. Transactions of the Association for Computational Linguistics, 12:979–995.
  8. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  9. Zhiwei He, Tian Liang, Wenxiang Jiao, Zhuosheng Zhang, Yujiu Yang, Rui Wang, Zhaopeng Tu, Shuming Shi, and Xing Wang. 2024. Exploring human-like translation strategy with large language models. Transactions of the Association for Computational Linguistics, 12:229–246.
  10. Yichong Huang, Baohang Li, Xiaocheng Feng, Wenshuai Huo, Chengpeng Fu, Ting Liu, and Bing Qin. 2024. Aligning translation-specific understanding to general understanding in large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 5028–50… find
  11. Juraj Juraska, Mara Finkelstein, Daniel Deutsch, Aditya Siddhant, Mehdi Mirzazadeh, and Markus Freitag. 2023. MetricX-23: The Google submission to the WMT 2023 metrics shared task. In Proceedings of the Eighth Conference on Machine Translation, pages 756–767, Singapore. Association for Computational…
  12. Daniel Deutsch, Eleftheria Briakou, Isaac Caswell, Mara Finkelstein, Rebecca Galor, Juraj Juraska, Geza Kovacs, Alison Lui, Ricardo Rei, Jason Riesa, and 1 others. 2025. WMT24++: Expanding the language coverage of WMT24 to 55 languages & dialects. arXiv preprint arXiv:2502.12404.
  13. Dayeon Ki and Marine Carpuat. 2024. Guiding large language models to post-edit machine translation with error annotations. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 4253–4273, Mexico City, Mexico. Association for Computational Linguistics. find
  14. Daniel Deutsch, Juraj Juraska, Mara Finkelstein, and Markus Freitag. 2023. Training and meta-evaluating machine translation evaluation metrics at the paragraph level. In Proceedings of the Eighth Conference on Machine Translation, pages 996–1013, Singapore. Association for Computational Linguistics. find
  15. Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, Ondřej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Marzena Karpinska, Philipp Koehn, Benjamin Marie, Christof Monz, Kenton Murray, Masaaki Nagata, Martin Popel, Maja Popov… — Findings of the WMT24 General Machine Translation Shared Task: The LLM Era Is Here but MT Is Not Solved Yet
  16. Zhaopeng Feng, Yan Zhang, Hao Li, Bei Wu, Jiayu Liao, Wenqiang Liu, Jun Lang, Yang Feng, Jian Wu, and Zuozhu Liu. 2025. TEaR: Improving LLM-based machine translation with systematic self-refinement. In Findings of the Association for Computational Shuang Li, Jiangjie Chen, Siyu Yuan, Xinyi Wu, Hao Y… find
  17. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-refine: Iterative refinement with… find
  18. Brian Mossop. 2013. Revising and Editing for Translators, 3 edition. Routledge, London. find
  19. Advances in Neural Information Processing Systems, volume 35, pages 24824–24837. Curran Associates, Inc. find
  20. Minghao Wu, Jiahao Xu, Yulin Yuan, Gholamreza Haffari, Longyue Wan, Weihua Luo, and Kaifu Zhang. 2025. (Perhaps) beyond human translation: Harnessing multi-agent collaboration for translating ultra-long literary texts. Transactions of the Association for Computational Linguistics, 13:901–922.
  21. Wenda Xu, Daniel Deutsch, Mara Finkelstein, Juraj Juraska, Biao Zhang, Zhongtao Liu, William Yang Wang, Lei Li, and Markus Freitag. 2024. LLMRefine: Pinpointing and refining large language models via fine-grained actionable feedback. In Findings of the Association for Computational Linguistics: NAAC… find
  22. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393.
  23. OpenAI. 2024. GPT-4o mini: Advancing cost-efficient intelligence. find
  24. Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. 2024. Automatically correcting large language models: Surveying the landscape of diverse automated correction strategies. Transactions of the Association for Computational Linguistics, 12:484–506.
  25. Ricardo Rei, José G. C. de Souza, Duarte Alves, Chrysoula Zerva, Ana C Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and André F. T. Martins. 2022. COMET-22: Unbabel-IST 2022 submission for the metrics shared task. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages…
  26. Ricardo Rei, Nuno M. Guerreiro, José Pombal, Daan van Stigt, Marcos Treviso, Luisa Coheur, José G. C. de Souza, and André Martins. 2023. Scaling up CometKiwi: Unbabel-IST 2023 submission for the quality estimation shared task. In Proceedings of the Eighth Conference on Machine Translation, pages 841…
  27. Zayne Rea Sprague, Fangcong Yin, Juan Diego Rodriguez, Dongwei Jiang, Manya Wadhwa, Prasann Singhal, Xinyu Zhao, Xi Ye, Kyle Mahowald, and Greg Durrett. 2025. To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning. In The Thirteenth International Conference on Learning Re…
  28. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2025-please-translate-again-two-simple/