Elamur Research⌕
☰
home / papers / 2026 / PEGRL: Improving Machine Translation by Post-Editing Guided Reinforcement Learning

PEGRL: Improving Machine Translation by Post-Editing Guided Reinforcement Learning

Paper · 2026 · Yunzhi Shen, Hao Zhou, Xin Huang et al.

Summary

LLM translation trained with reinforcement learning (e.g., GRPO) suffers from two problems. Its Monte Carlo group baselines are noisy, and its huge output space rewards broad exploration but gives little signal for small local fixes. PEGRL addresses this by treating translation as a two-step workflow: a draft translation followed by post-editing, both produced by the same model. At each iteration, freshly sampled drafts become on-policy inputs for the post-editing task. The rewards of the post-edited outputs serve two purposes: they train the editing step, and their average scores the draft that produced them. The authors show that conditioning on a fixed draft removes between-draft variability, which makes the post-editing baseline much easier to estimate. They also introduce a weighting scheme that favours the post-editing gradient over the translation gradient; this is biased but more sample-efficient. Experiments cover EN→FI, EN→TR and EN↔ZH on WMT24 and FLORES. PEGRL consistently beats the RL baseline MT-R1-Zero, with the largest gains in directions the base model covers poorly. The work suggests that variance in return estimation deserves explicit attention when designing RL objectives.

Key points

  • On baseline variance, post-editing with a group size of 8 roughly matches translation with a group size of 64.
  • The best gradient weighting is λ_pe = M (the number of post-edits per draft) with λ_mt = 1. This beats both equal weighting and dropping the translation term.
  • Evaluation uses chrF++, COMET-KIWI and XCOMET; XCOMET was not optimized during training. On EN→TR, COMET-KIWI is on par with DeepSeek-V3.2.
  • In pairwise judging by GPT-5.2 and Gemini-3-Pro-Preview on 998 WMT24 samples, PEGRL is preferred over MT-R1-Zero in every direction. For example, on EN→FI the GPT judge chose PEGRL 627 times versus 361 for the baseline. Models compared were 8B (EN→FI, EN→TR) and 0.6B (EN↔ZH).
  • PEGRL was also run on Ascend (Atlas 800I A3) hardware with comparable results. Code is at github.com/NJUNLP/peg-rl and models at huggingface.co/collections/DGME/pegrl.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Nanjing UniversityNanjing · China74 works · 7,814 citations🏢China Mobile Research InstituteBeijing · China2 works · 140 citations🏢China MobileBeijing · China9 works · 21 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Machine translation · 1,087 works

Cite

Cite as Yunzhi Shen, Hao Zhou, Xin Huang, Xue Han, Junlan Feng and Shujian Huang. 2026. PEGRL: Improving Machine Translation by Post-Editing Guided Reinforcement Learning. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.851

@inproceedings{shen2026pegrl,
  title = {{PEGRL: Improving Machine Translation by Post-Editing Guided Reinforcement Learning}},
  author = {Yunzhi Shen, Hao Zhou, Xin Huang, Xue Han, Junlan Feng and Shujian Huang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.851},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.851}
}

References

29 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Duarte M. Alves, José Pombal, Nuno M. Guerreiro, Pedro H. Martins, João Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, Pierre Colombo, José G. C. de Souza, and André F. T. Martins. 2024. Tower: An open multilingual large language model for translation-related tasks.…
  2. Ond rej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Shujian Huang, Matthias Huck, Philipp Koehn, Qun Liu, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Raphael Rubino, Lucia Specia, and Marco Turchi. 2017. Findings of the 2017 conference on machine transla…
  3. Ond rej Bojar, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Koehn, and Christof Monz. 2018. Findings of the 2018 conference on machine translation (wmt18). In Proceedings of the Third Conference on Machine Translation, Volume 2: Shared Task Papers, pages 272–…
  4. Shanbo Cheng, Yu Bao, Qian Cao, Luyang Huang, Liyan Kang, Zhicheng Liu, Yu Lu, Wenhao Zhu, Jingwen Chen, Zhichao Huang, Tao Li, Yifu Li, Huiying Lin, Sitong Liu, Ningxin Peng, Shuaijie She, Lu Xu, Nuo Xu, Sen Yang, and 7 others. 2025. Seed-x: Building strong multilingual translation llm with 7b para… — [arXiv:2507.13618](https://arxiv.org/abs/2507.13618) find
  5. Marta R Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, and 1 others. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672. find
  6. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025a. Deepseek-r1: Incentivizing reasoning capability in l…
  7. DeepSeek-AI, Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenhao Xu, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, and 245 others. 2025b. Deepseek-v3.2: Pushing the frontier of ope…
  8. Daniel Deutsch, Eleftheria Briakou, Isaac Caswell, Mara Finkelstein, Rebecca Galor, Juraj Juraska, Geza Kovacs, Alison Lui, Ricardo Rei, Jason Riesa, Shruti Rijhwani, Parker Riley, Elizabeth Salesky, Firas Trabelsi, Stephanie Winkler, Biao Zhang, and Markus Freitag. 2025. WMT24++: Expanding the Lang…
  9. Félix Do Carmo, Dimitar Shterionov, Joss Moorkens, Joachim Wagner, Murhaf Hossari, Eric Paquin, Dag Schmidtke, Declan Groves, and Andy Way. 2021. A review of the state-of-the-art in automatic post-editing. Machine Translation, 35(2):101–143. find
  10. Zhaopeng Feng, Shaosheng Cao, Jiahan Ren, Jiayuan Su, Ruizhe Chen, Yan Zhang, Zhe Xu, Yao Hu, Jian Wu, and Zuozhu Liu. 2025. Mt-r1-zero: Advancing llm-based machine translation via r1-zero-like reinforcement learning. Preprint, arXiv:2504.10160. find
  11. Wikimedia Foundation. Acl 2019 fourth conference on machine translation (wmt19), shared task: Machine translation of news. find
  12. Nuno M. Guerreiro, Ricardo Rei, Daan van Stigt, Luisa Coheur, Pierre Colombo, and André F. T. Martins. 2023. xcomet: Transparent machine translation evaluation through fine-grained error detection. Preprint, arXiv:2310.10482.
  13. Minggui He, Yilun Liu, Shimin Tao, Yuanchang Luo, Hongyong Zeng, Chang Su, Li Zhang, Hongxia Ma, Daimeng Wei, Weibin Meng, Hao Yang, Boxing Chen, and Osamu Yoshie. 2025. R1-t1: Fully incentivizing translation capability in llms via reasoning learning. Preprint, arXiv:2502.19735.
  14. Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. 2023. Decomposed prompting: A modular approach for solving complex tasks. Preprint, arXiv:2210.02406. find
  15. Zheng Wei Lim, Nitish Gupta, Honglin Yu, and Trevor Cohn. 2025. Mufu: Multilingual fused learning for low-resource translation with llm. Preprint, arXiv:2409.13949. — [arXiv:2409.13949](https://arxiv.org/abs/2409.13949) find
  16. Alan K. Melby. 1984. Machine translation with post editing versus a three-level integrated translator aid system. In Proceedings of the International Conference on Methodology and Techniques of Machine Translation: Processing from words to language, Cranfield University, UK. find
  17. Nicholas Metropolis, Arianna W Rosenbluth, Marshall N Rosenbluth, Augusta H Teller, and Edward Teller. 1953. Equation of state calculations by fast computing machines. The journal of chemical physics, 21(6):1087–1092. find
  18. Maja Popović. 2017. chrf++: words helping character n-grams. In Proceedings of the second conference on machine translation, pages 612–618.
  19. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Brussels, Belgium. Association for Computational Linguistics.
  20. Vikas Raunak, Amr Sharaf, Yiren Wang, Hany Hassan Awadallah, and Arul Menezes. 2023. Leveraging gpt-4 for automatic translation post-editing. Preprint, arXiv:2305.14878.
  21. Ricardo Rei, José G. C. de Souza, Duarte Alves, Chrysoula Zerva, Ana C Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and André F. T. Martins. 2022. COMET-22: Unbabel-IST 2022 submission for the metrics shared task. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages…
  22. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. Preprint, arXiv:1707.06347. find
  23. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. Preprint, arXiv:2402.03300.
  24. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2024. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256.
  25. Jiaan Wang, Fandong Meng, and Jie Zhou. 2025. Deeptrans: Deep reasoning translation via reinforcement learning. Preprint, arXiv:2504.10187.
  26. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-thought prompting elicits reasoning in large language models. Preprint, arXiv:2201.11903.
  27. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025a. Qwen3 technical report. Preprint, arXiv:2505.09388.
  28. Wenjie Yang, Mao Zheng, Mingyang Song, Zheng Li, and Sitong Wang. 2025b. Ssr-zero: Simple self-rewarding reinforcement learning for machine translation. Preprint, arXiv:2505.16637.
  29. Guanning Zeng, Zhaoyi Zhou, Daman Arora, and Andrea Zanette. 2025. Shrinking the variance: Shrinkage baselines for reinforcement learning with verifiable rewards. Preprint, arXiv:2511.03710. — [arXiv:2511.03710](https://arxiv.org/abs/2511.03710) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shen-2026-pegrl-improving-machine-translation-post/