Elamur Research⌕
☰
home / papers / 2025 / (Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts

(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts

Paper · 2025 · Minghao Wu, Jiahao Xu, Yulin Yuan et al.

Summary

Literary translation is still hard for MT systems. Novels rely on figurative language, cultural references and a distinctive authorial style, and these become harder to handle consistently across book-length documents. The authors present TRANSAGENTS, a framework in which LLM agents play the staff of a translation company: a CEO, Senior and Junior Editors, a Translator, a Localization Specialist and a Proofreader. In a preparation phase, the team is selected and a shared translation guideline is written. In an execution phase, chapters are translated, culturally adapted and proofread, and then pass a final editorial check. Standard reference-based scores are unreliable for literature, so the paper adds two evaluation protocols. Monolingual Human Preference (MHP) asks target-language readers to judge without the source text. Bilingual LLM Preference (BLP) has GPT-4 compare outputs against the original. TRANSAGENTS scores lowest on d-BLEU, which the authors attribute to references with little diversity. However, it wins on GEMBA-DA, and both readers and the LLM judge prefer it over human references and GPT-4 output. This suggests that agent collaboration helps most for long texts.

Key points

  • Agent profiles: GPT-4-TURBO generated 30 profiles per role, covering attributes such as nationality, education, experience and per-word rate. A "ghost agent" prompts the CEO to reconsider its choice of Senior Editor, a form of self-reflection.
  • Collaboration algorithms: Agents work together through an Addition-by-Subtraction scheme and a judgment-based loop, both with early exit. Raising the iteration limit M above 2 brings little or no gain and adds only about 10% runtime.
  • Cost: The system costs more time and money than plain GPT-4-1106-PREVIEW. It is still much cheaper and faster than professional translators, whose minimum rate is cited as 0.12 USD per word.
  • Short texts: On 200 sampled WMT2024 General MT examples (English–Chinese, English–German), TRANSAGENTS underperforms GPT-4O-MINI. Its output tends to be polished but drifts from the source and over-translates.
  • Case study: In a cultural-adaptation example, the localization step fixed the name and job-title order, and proofreading corrected terminology errors.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢Alibaba GroupHangzhou · China201 works · 17,173 citations🎓Nanyang Technological UniversitySingapore · Singapore184 works · 27,484 citations🎓Monash UniversityMelbourne · Australia64 works · 54,971 citations🎓University of MacauMacao · China43 works · 2,368 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

Cite

Cite as Minghao Wu, Jiahao Xu, Yulin Yuan, Gholamreza Haffari, Longyue Wan, Weihua Luo and Kaifu Zhang. 2025. (Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl.a.25

@article{wu2025perhaps,
  title = {{(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts}},
  author = {Minghao Wu, Jiahao Xu, Yulin Yuan, Gholamreza Haffari, Longyue Wan, Weihua Luo and Kaifu Zhang},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2025},
  doi = {10.1162/tacl.a.25},
  url = {https://doi.org/10.1162/tacl.a.25}
}

References

70 references, 35 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aleksandar Botev, Soham De, Samuel L. Smith, Anushan Fernando, George-Cristian Muraru, Ruba Haroun, Leonard Berrada, Razvan Pascanu, Pier Giuseppe Sessa, Robert Dadashi, Léonard Hussenot, Johan Ferret, Sertan Girgin, Olivier Bachem, Alek Andreev, Kathleen Kenealy, Thomas Mesnard, Cassidy Hardin, Su… find
  2. Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loı̈c Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Se… find
  3. Eleftheria Briakou, Jiaming Luo, Colin Cherry, and Markus Freitag. 2024. Translating step-by-step: Decomposing the translation process for improved translation quality of long-form texts. In Proceedings of the Ninth Conference on Machine Translation, pages 1301–1317, Miami, Florida, USA. Association…
  4. Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. 2023. Chateval: Towards better llm-based evaluators through multi-agent debate. CoRR, abs/2308.07201.
  5. Michael A. Covington and Joe D. McFall. 2010. Cutting the gordian knot: The moving-average type–token ratio (mattr). Journal of Quantitative Linguistics, 17(2):94–100. https://doi.org/10.1080/09296171003643098 — [doi:10.1080/09296171003643098](https://doi.org/10.1080/09296171003643098) ↗
  6. Yihong Dong, Xue Jiang, Zhi Jin, and Ge Li. 2023. Self-collaboration code generation via chatgpt. CoRR, abs/2304.07590.
  7. Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. 2023a. Improving factuality and reasoning in language models through multiagent debate. CoRR, abs/2305.14325.
  8. Zefeng Du, Wenxiang Jiao, Longyue Wang, Chenyang Lyu, Jianhui Pang, Leyang Cui, Kaiqiang Song, Derek F. Wong, Shuming Shi, and Zhaopeng Tu. 2023b. On extrapolation of long-text translation with large language models. https://www.researchgate.net/profile/Longyue-Wang/publication Xin Chan, Xiaoyang W… ↗
  9. CoRR, abs/2402.01680. https://doi.org/10.24963/ijcai.2024/890 — Large Language Model based Multi-Agents: A Survey of Progress and Challenges ↗
  10. Zhiwei He, Tian Liang, Wenxiang Jiao, Zhuosheng Zhang, Yujiu Yang, Rui Wang, Zhaopeng Tu, Shuming Shi, and Xing Wang. 2024. Exploring human-like translation strategy with large language models. Transactions of the Association for Computational Linguistics, 12:229–246. https://doi.org/10 .1162/tacl_a…
  11. Christian Herold and Hermann Ney. 2023. Improving long context document-level machine translation. In Proceedings of the 4th Workshop on Computational Approaches to Discourse (CODI 2023), pages 112–125, Toronto, Canada. Association for Computational Linguistics. https://doi.org/10 .18653/v1/2023.cod… find
  12. Markus Freitag, David Grangier, and Isaac Caswell. 2020. BLEU might be guilty but references are not innocent. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 61–71, Online. Association for Computational Linguistics. https://doi.org/10 .18653/…
  13. Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, and Chenglin Wu. 2023. Metagpt: Meta programming for multi-agent collaborative framework. CoRR, abs/2308.00352.
  14. Markus Freitag, Nitika Mathur, Chi-kiu Lo, Eleftherios Avramidis, Ricardo Rei, Brian Thompson, Tom Kocmi, Frederic Blain, Daniel Deutsch, Craig Stewart, Chrysoula Zerva, Sheila Castilho, Alon Lavie, and George Foster. 2023. Results of WMT23 metrics shared task: Metrics might be guilty but references…
  15. Marzena Karpinska and Mohit Iyyer. 2023. Large language models effectively leverage document-level context for literary translation, but critical errors persist. In Proceedings of the Eighth Conference on Machine Translation, pages 419–451, Singapore. Association for Computational Linguistics. https…
  16. Marzena Karpinska, Katherine Thai, Kyle Lo, Tanya Goyal, and Mohit Iyyer. 2024. One thousand and one pairs: A ‘‘novel’’ challenge for long-context language models. CoRR, abs/2406.16264. https://doi.org/10 .18653/v1/2024.emnlp-main.948 find
  17. Albert Gu and Tri Dao. 2023. Mamba: Linear-time sequence modeling with selective state spaces. CoRR, abs/2312.00752. find
  18. Jiatao Gu, Changhan Wang, and Junbo Zhao. 2019. Levenshtein transformer. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8–14, 2019, Vancouver, BC, Canada, pages 11179–11189. find
  19. Jeremy Klemin. 2024. The last frontier of machine translation. The Atlantic. find
  20. Tom Kocmi and Christian Federmann. 2023a. GEMBA-MQM: Detecting translation quality error spans with GPT-4. In Proceedings of the Eighth Conference on Machine Translation, pages 768–775, Singapore. Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.wmt-1.64 ↗
  21. Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. 2024. Large language model based multi-agents: A survey of progress and challenges.
  22. Tom Kocmi and Christian Federmann. 2023b. Large language models are state-of-the-art evaluators of translation quality. In Proceedings of the 24th Annual Conference of the European Association for Machine Translation, pages 193–203, Tampere, Finland. European Association for Machine Translation. .or… find
  23. PubMed: Gabriel Mukobi, Hannah Erlebach, Niklas Lauffer, Lewis Hammond, Alan Chan, and Jesse Clifton. 2023. Welfare diplomacy: Benchmarking language model cooperation. CoRR, abs/2310.08901. — [arXiv:2310.08901](https://arxiv.org/abs/2310.08901) find
  24. Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pages 388–395, Barcelona, Spain. Association for Computational Linguistics.
  25. OpenAI. 2023. GPT-4 technical report. CoRR, abs/2303.08774.
  26. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
  27. Clifford E. Landers. 2001. Literary translation: Apractical guide. MultilingualMatters. https://doi.org/10.21832/9781853595639 — [doi:10.21832/9781853595639](https://doi.org/10.21832/9781853595639) ↗
  28. Nian Li, Chen Gao, Yong Li, and Qingmin Liao. 2023. Large language model-empowered agents for simulating macroeconomic activities. CoRR, abs/2310.10436. — [arXiv:2310.10436](https://arxiv.org/abs/2310.10436) find
  29. Joon Sung Park, Joseph C. O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST 2023, San Francisco, CA,… find
  30. Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu, and Shuming Shi. 2023. Encouraging divergent thinking in large language models through multi-agent debate. CoRR, abs/2305.19118.
  31. Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020. Multilingual denoising pre-training for neural machine translation. Transactions of the Association for Computational Linguistics, 8:726–742. https://doi.org/10.1162 /tacl_a_0034… find
  32. Joon Sung Park, Lindsay Popowski, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2022. Social simulacra: Creating populated prototypes for social computing systems. In The 35th Annual ACM Symposium on User Interface Software and Technology, UIST 2022, Bend, OR, USA, 29…
  33. Arle Richard Lommel, Aljoscha Burchardt, and Hans Uszkoreit. 2013. Multidimensional quality metrics: A flexible system for assessing translation quality. In Proceedings of Translating and the Computer 35, London, UK. Aslib.
  34. Celia Rico Pérez. 2002. Translation and project management. Translation Journal, 6(4):38–52. find
  35. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186–191, Brussels, Belgium. Association for Computational Linguistics. https://doi.org/10 .18653/v1/W18-6319
  36. Zhao Mandi, Shreeya Jain, and Shuran Song. 2023. Roco: Dialectic multi-robot collaboration with large language models. CoRR, abs/2307.04738. — [arXiv:2307.04738](https://arxiv.org/abs/2307.04738) find
  37. Philip M. McCarthy and Scott Jarvis. 2010. Mtld, vocd-d, and hd-d: A validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods, 42(2):381–392. https://doi find
  38. Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. 2023. Communicative agents for software development. CoRR, abs/2307.07924.
  39. Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4–9, 2017, Long Beach, CA, USA, pages 5998–6008.
  40. Ricardo Rei, Craig Stewart, Ana C. Farinha, and Alon Lavie. 2020. COMET: A neural framework for MT evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685–2702, Online. Association for Computational Linguistics. https://doi.org/10.186…
  41. Pat Verga, Sebastian Hofstätter, Sophia Althammer, Yixuan Su, Aleksandra Piktus, Arkady Arkhangorodsky, Minjie Xu, Naomi White, and Patrick S. H. Lewis. 2024. Replacing judges with juries: Evaluating LLM generations with a panel of diverse models. CoRR, abs/2404.18796.
  42. Nathaniel Robinson, Perez Ogayo, David R. Mortensen, and Graham Neubig. 2023. Chat-GPT MT: Competitive for high- (but not low-) resource languages. In Proceedings of the Eighth Conference on Machine Translation, pages 392–418, Singapore. Association for Computational Linguistics. https://doi .org/10…
  43. Giorgos Vernikos, Brian Thompson, Prashant Mathur, and Marcello Federico. 2022. Embarrassingly easy document-level MT metrics: How to convert any pretrained metric into a document-level metric. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 118–128, Abu Dhabi, United Ar…
  44. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik R. Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. In Thirty-seventh Conference on Neural Information Processing Systems.
  45. Rob Voigt and Dan Jurafsky. 2012. Towards a literary machine translation: The role of referential cohesion. In Proceedings of the NAACL-HLT 2012 Workshop on Computational Linguistics for Literature, pages 18–25, Montréal, Canada. Association for Computational Linguistics. find
  46. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8–13 2014, Montreal, Quebec, Canada, pages 3104–3112.
  47. Callum Walker. 2022. Translation Project Management. Routledge. https://doi.org/10 .4324/9781003132813-1, https://doi .org/10.4324/9781003132813 — [doi:10.4324/9781003132813](https://doi.org/10.4324/9781003132813) find
  48. Mary F. Terhaar, Rachael Crickman, and Deborah S. Finnell. 2019. Project management for translation. Translation of Evidence Into Nursing and Healthcare, page 199. https://doi.org/10.1891/9780826147370.0009 — [doi:10.1891/9780826147370.0009](https://doi.org/10.1891/9780826147370.0009) ↗
  49. Longyue Wang, Zefeng Du, Wenxiang Jiao, Chenyang Lyu, Jianhui Pang, Leyang Cui, Kaiqiang Song, Derek Wong, Shuming Shi, and Zhaopeng Tu. 2024a. Benchmarking and improving long-text translation with large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 7… find
  50. Katherine Thai, Marzena Karpinska, Kalpesh Krishna, Bill Ray, Moira Inghilleri, John Wieting, and Mohit Iyyer. 2022. Exploring document-level literary machine translation with parallel paragraphs from world literature. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Pr… find
  51. Longyue Wang, Siyou Liu, Chenyang Lyu, Wenxiang Jiao, Xing Wang, Jiahao Xu, Zhaopeng Tu, Yan Gu, Weiyu Chen, Minghao Wu, Liting Zhou, Philipp Koehn, Andy Way, and Yulin Yuan. 2024b. Findings of the WMT 2024 shared task on discourse-level literary Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszk… find
  52. Minghao Wu, George Foster, Lizhen Qu, and Gholamreza Haffari. 2023. Document flattening: Beyond concatenating context for document-level neural machine translation. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 448–462, Dubrovni… find
  53. Longyue Wang, Chenyang Lyu, Tianbo Ji, Zhirui Zhang, Dian Yu, Shuming Shi, and Zhaopeng Tu. 2023a. Document-level machine translation with large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 16646–16661, Singapore. Association for C…
  54. Minghao Wu, Thuy-Trang Vu, Lizhen Qu, George F. Foster, and Gholamreza Haffari. 2024a. Adapting large language models for document-level machine translation. CoRR, abs/2401.06468. — [arXiv:2401.06468](https://arxiv.org/abs/2401.06468) find
  55. Longyue Wang, Zhaopeng Tu, Yan Gu, Siyou Liu, Dian Yu, Qingsong Ma, Chenyang Lyu, Liting Zhou, Chao-Hong Liu, Yufeng Ma, Weiyu Chen, Yvette Graham, Bonnie Webber, Philipp Koehn, Andy Way, Yulin Yuan, and Shuming Shi. 2023b. Findings of the WMT 2023 shared task on discourse-level literary translation… find
  56. Minghao Wu, Yufei Wang, George Foster, Lizhen Qu, and Gholamreza Haffari. 2024b. Importance-aware data augmentation for document-level neural machine translation. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), p… find
  57. Longyue Wang, Zhaopeng Tu, Andy Way, and Qun Liu. 2017. Exploiting cross-sentence context for neural machine translation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2826–2831, Copenhagen, Denmark. Association for Computational Linguistics. https:…
  58. Minghao Wu, Jiahao Xu, and Longyue Wang. 2024c. TransAgents: Build your translation company with language agents. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 131–141, Miami, Florida, USA. Association for Computational Lingui… find
  59. Yutong Wang, Jiali Zeng, Xuebo Liu, Derek F. Wong, Fandong Meng, Jie Zhou, and Min Zhang. 2024c. Delta: An online document-level translation agent based on multi-level memory. CoRR, abs/2410.08143. — [arXiv:2410.08143](https://arxiv.org/abs/2410.08143) find
  60. Zihao Wang, Shaofei Cai, Anji Liu, Xiaojian Ma, and Yitao Liang. 2023c. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. CoRR, abs/2302.01560.
  61. Yuhao Xie, Zongyao Li, Zhanglin Wu, Daimeng Wei, Xiaoyu Chen, Zhiqiang Rao, Shaojun Li, Hengchao Shang, Jiaxin Guo, Lizhi Lei, Hao Yang, and Yanfei Jiang. 2023. HW-TSC’s submissions to the WMT23 discourse-level literary translation shared task. In Proceedings of the Eighth Conference on Machine Tran… find
  62. Michael J. Wooldridge and Nicholas R. Jennings. 1995. Intelligent agents: Theory Wenhan Xiong, Jingyu Liu, Igor Molybog, Hejia Zhang, Prajjwal Bhargava, Rui Hou, Louis Martin, Rashi Rungta, Karthik Abinav Sankararaman, Barlas Oguz, Madian Khabsa, Han Fang, Yashar Mehdad, Sharan Narang, Kshitiz Malik…
  63. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1–5, 2023. OpenReview.net.
  64. Haoran Xu, Young Jin Kim, Amr Sharaf, and Hany Hassan Awadalla. 2023a. A paradigm shift in machine translation: Boosting translation performance of large language models. CoRR, abs/2309.11674. find
  65. Hongxin Zhang, Weihua Du, Jiaming Shan, Qinhong Zhou, Yilun Du, Joshua B. Tenenbaum, Tianmin Shu, and Chuang Gan. 2023. Building cooperative embodied agents modularly with large language models. CoRR, abs/2307.02485.
  66. Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. 2024. Contrastive preference optimization: Pushing the boundaries of LLM performance in machine translation. CoRR, abs/2401.08417.
  67. Ran Zhang, Wei Zhao, and Steffen Eger. 2024. How good are LLMs for literary translation, really? Literary translation evaluation with humans and LLMs. CoRR, abs/2410.18697. find
  68. Yuzhuang Xu, Shuo Wang, Peng Li, Fuwen Luo, Xiaolong Wang, Weidong Liu, and Yang Liu. 2023b. Exploring large language models for communication games: An empirical study on werewolf. CoRR, abs/2309.04658.
  69. Anqi Zhao, Kaiyu Huang, Hao Yu, and Degen Huang. 2023. DUTNLP system for the WMT2023 discourse-level literary translation. In Proceedings of the Eighth Conference on Machine Translation, pages 296–301, Singapore. Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.wmt-1.31 ↗
  70. Jianhao Yan, Pingchuan Yan, Yulong Chen, Judy Li, Xianchao Zhu, and Yue Zhang. 2024. GPT-4 vs. human translators: A comprehen- find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2025-perhaps-beyond-human-translation-harnessing/