Elamur Research⌕
☰
home / papers / 2026 / Multi-Agent Debate for Machine Translation: A Case Study on English-Japanese Translation

Multi-Agent Debate for Machine Translation: A Case Study on English-Japanese Translation

Summary

Shen et al. ask whether multi-agent debate (MAD), in which several LLM agents deliberate before producing an answer, helps machine translation. Prior MAD work mostly targeted tasks with a single correct answer, while a translation can have many acceptable versions. Using English–Japanese as a demanding test pair, with large gaps in word order, ellipsis, honorifics and register, the authors adapt three debate designs: Pro/Con, Society of Mind (SoM) and Decoupled. Each design ends every round with a moderator who merges the agents' outputs, instead of using voting. The debates run on GPT-4.1 and are compared with zero-shot models, reasoning LLMs and a self-reflection prompt. The test data cover general-domain and culturally grounded material.

SoM, where each agent develops its own translation in parallel, performs best, especially for En→Ja. Its gains appear mostly in the first round. Later rounds often over-edit good translations and cause semantic drift. The study argues that agentic translation pipelines need ways to keep strong intermediate drafts and to stop unnecessary revision.

Key points

  • Data and evaluation: The test sets are WMT2023, KFTT (Japanese historical and cultural content) and IWSLT2017 (talks with preceding context). Quality is scored with BLEU, BERTScore and xCOMET, plus human evaluation and error-span analysis.
  • Baselines: The debates are compared with zero-shot GPT-4.1, o4-mini, DeepSeek-R1 and a self-reflection method. Pro/Con and Decoupled debate give little or no improvement over these baselines.
  • Model mixing: Mixing models such as DeepSeek-V3 and Gemini 1.5 Pro into the debater and moderator roles is competitive in round 1. These mixed setups become less stable in later rounds.
  • Agents and reasoning language: Increasing the number of agents from 2 to 8 steadily raises first-round xCOMET. Reasoning in Japanese slightly helps round 1 but leaves overall quality largely unchanged.
  • Context: On IWSLT2017, giving SoM 10 preceding sentences yields the highest xCOMET. Extra context helps deliberation more than it helps zero-shot translation. Code is released at github.com/ZhanShenYo/madmt.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics: Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{shen2026multi,
  title = {{Multi-Agent Debate for Machine Translation: A Case Study on English-Japanese Translation}},
  author = {Shen, Zhan and Naradowsky, Jason and Wang, Xiaotian and Miyao, Yusuke},
  booktitle = {Proceedings of the 26th Annual Conference of the European Association for Machine Translation (Volume 1)},
  year = {2026},
  url = {https://aclanthology.org/2026.eamt-1.16/}
}

References

39 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Briva-Iglesias, Vicent. 2025. Are ai agents the new machine translation frontier? challenges and opportunities of single-and multi-agent systems for multilingual digital communication. Proceedings of Machine Translation Summit XX: Volume 1, pages 365–377. find
  2. Brown, Tom, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901. find
  3. Castilho, Sheila, Joss Moorkens, Federico Gaspari, Iacer Calixto, John Tinsley, and Andy Way. 2017. Is neural machine translation the new state of the art? The Prague Bulletin of Mathematical Linguistics. find
  4. Cettolo, Mauro, Marcello Federico, Luisa Bentivogli, Jan Niehues, Sebastian Stüker, Katsuhito Sudoh, Koichiro Yoshino, and Christian Federmann. 2017. Overview of the iwslt 2017 evaluation campaign. In Proceedings of the 14th International Conference on Spoken Language Translation, pages 2–14. find our card
  5. Chowdhery, Aakanksha, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2023. Palm: Scaling language modeling with pathways. Journal of machine learning research, 24(240):1–113. find
  6. Estornell, Andrew, Jean-Francois Ton, Yuanshun Yao, and Yang Liu. 2024. Acc-debate: An actor-critic approach to multi-agent debate. arXiv preprint arXiv:2411.00053. — [arXiv:2411.00053](https://arxiv.org/abs/2411.00053) find
  7. Feng, Zhaopeng, Jiayuan Su, Jiamei Zheng, Jiahan Ren, Yan Zhang, Jian Wu, Hongwei Wang, and Zuozhu Liu. 2025. M-mad: Multidimensional multi-agent debate for advanced machine translation evaluation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1:… find our card
  8. Freitag, Markus, George Foster, David Grangier, Viresh Ratnakar, Qijun Tan, and Wolfgang Macherey. 2021. Experts, errors, and context: A large-scale study of human evaluation for machine translation. Transactions of the Association for Computational Linguistics, 9:1460– 1474. find our card
  9. Freitag, Markus, Nitika Mathur, Chi-kiu Lo, Eleftherios Avramidis, Ricardo Rei, Brian Thompson, Tom Kocmi, Frederic Blain, Daniel Deutsch, Craig Stewart, et al. 2023. Results of wmt23 metrics shared task: Metrics might be guilty but references are not innocent. In Proceedings of the Eighth Conferenc… find our card
  10. Fu, Yao, Hao Peng, Tushar Khot, and Mirella Lapata. 2023. Improving language model negotiation with self-play and in-context learning from ai feedback. arXiv preprint arXiv:2305.10142. find our card
  11. Guerreiro, Nuno M, Ricardo Rei, Daan van Stigt, Luisa Coheur, Pierre Colombo, and André FT Martins. 2024. xcomet: Transparent machine translation evaluation through fine-grained error detection. Transactions of the Association for Computational Linguistics, 12:979–995. find our card
  12. Guo, Daya, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948. find our card
  13. He, Zhiwei, Tian Liang, Wenxiang Jiao, Zhuosheng Zhang, Yujiu Yang, Rui Wang, Zhaopeng Tu, Shuming Shi, and Xing Wang. 2024. Exploring human-like translation strategy with large language models. Transactions of the Association for Computational Linguistics, 12:229–246. find our card
  14. Khan, Akbir, John Hughes, Dan Valentine, Laura Ruis, Kshitij Sachan, Ansh Radhakrishnan, Edward Grefenstette, Samuel R Bowman, Tim Rocktäschel, and Ethan Perez. 2024. Debating with more persuasive llms leads to more truthful answers. arXiv preprint arXiv:2402.06782. find
  15. Kocmi, Tom, Rachel Bawden, Ondřej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Thamme Gowda, Yvette Graham, Roman Grundkiewicz, Barry Haddow, et al. 2022. Findings of the 2022 conference on machine translation (wmt22). In Proceedings of the Seventh Conference on Machine Translation (W… find
  16. Liang, Tian, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. 2024. Encouraging divergent thinking in large language models through multi-agent debate. In Proceedings of the 2024 conference on empirical methods in natural language processing, pages 1… find our card
  17. Liu, Aixin, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. find
  18. Madaan, Aman, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2023. Self-refine: Iterative refinement with self-feedback. Advances in neural information processing systems, 36:46534–46594. find
  19. Minsky, Marvin. 1986. Society of mind. Simon and Schuster. find
  20. Mori, Junko. 2020. The cambridge handbook of japanese linguistics ed. by yoko hasegawa. The Journal of Japanese Studies, 46(2):536–540. find
  21. Neubig, Graham. 2011. The kyoto free translation task. http://www.phontron.com/kftt. ↗
  22. OpenAI. 2024. Openai research. https://openai.com/research. Accessed: 2025-07-25. ↗
  23. Ouyang, Long, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744. find
  24. Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318. find our card
  25. Shinn, Noah, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. Advances in neural information processing systems, 36:8634–8652. find our card
  26. Sin, King-kui, Xi Xuan, Chunyu Kit, Clara Ho-yan Chan, and Honic Ho-kin Ip. 2025. Solving the unsolvable: Translating case law in hong kong. arXiv preprint arXiv:2501.09444. find
  27. Team, Gemini, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530. find our card
  28. Team, Kimi, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. 2025. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599. find our card
  29. Touvron, Hugo, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971. find
  30. Tsujii, Jun’ichi. 1986. Future directions of machine translation. In Coling 1986 Volume 1: The 11th International Conference on Computational Linguistics. find
  31. Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30. find
  32. Wang, Xuezhi, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171. find our card
  33. Wei, Jason, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652. find
  34. Wei, Jason, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837. find our card
  35. Wu, Minghao, Jiahao Xu, Yulin Yuan, Gholamreza Haffari, Longyue Wan, Weihua Luo, and Kaifu Zhang. 2025. (perhaps) beyond human translation: Harnessing multi-agent collaboration for translating ultra-long literary texts. Transactions of the Association for Computational Linguistics, 13:901–922. find
  36. Xiong, Kai, Xiao Ding, Yixin Cao, Ting Liu, and Bing Qin. 2023. Examining inter-consistency of large language models collaboration: An in-depth analysis via debate. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7572–7590. find
  37. Yang, An, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388. find our card
  38. Yao, Shunyu, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809–11822. find
  39. Zhang, Tianyi, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shen-2026-multi-agent-debate-machine-translation/