Elamur Research⌕
☰
home / papers / 2026 / Language-Coupled Reinforcement Learning for Multilingual Retrieval-Augmented Generation

Language-Coupled Reinforcement Learning for Multilingual Retrieval-Augmented Generation

Paper · 2026 · Rui Qi, Fengran Mo, Yufeng Chen et al.

Summary

The paper addresses two problems in multilingual retrieval-augmented generation (MRAG). The first is knowledge bias: an LLM answers the same question differently depending on its language. The second is knowledge conflict: passages retrieved from collections in many languages contradict each other. The authors propose LcRL, a search-augmented reinforcement learning framework built on Group Relative Policy Optimization (GRPO). During rollout, a group mixes semantically equivalent queries in several languages, so each language can learn from how the others behave. Retrieval runs over several turns. The model first searches its native-language collection, then the other languages' collections if it lacks knowledge, and finally falls back to English. The reward combines character n-gram recall against the gold answer with an anti-consistency penalty that keeps training stable as more languages are added. LcRL performs competitively against baselines and stays robust with little training data. It also scales to retrieval collections with many languages and transfers to unseen languages. This shows that RL post-training can teach models when and where to search across languages.

Key points

  • The language-coupled rollout puts parallel queries in different languages into one GRPO group. The anti-consistency penalty in the reward targets training collapse.
  • The outcome reward uses character 3-gram recall, which makes it less sensitive to answer length across languages.
  • Compared with mSearch-R1, LcRL avoids the reward collapse that appears around step 160. In late training, its responses are about 12% shorter.
  • GRPO outperforms PPO in the multilingual setting. RL methods hold up better than SFT when training data is scarce.
  • The setup uses multilingual-e5-base as a fixed retriever, Wikipedia dumps with existing QA datasets, and three multilingual LLMs. Code is available at github.com/Cherry-qwq/LcRL-Open.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China333 works · 87,019 citations🎓Université de MontréalMontreal · Canada68 works · 79,382 citations🎓University of Notre DameNotre Dame · United States51 works · 36,717 citations🎓Beijing Jiaotong UniversityBeijing · China22 works · 1,754 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

🧭 Topics 2Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 828 worksInformation retrieval & search · 473 works

Cite

Cite as Rui Qi, Fengran Mo, Yufeng Chen, Xue Zhang, Shuo Wang, Hongliang Li, Xu Jinan, Meng Jiang, Jian-Yun Nie and Kaiyu Huang. 2026. Language-Coupled Reinforcement Learning for Multilingual Retrieval-Augmented Generation. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.700

@inproceedings{qi2026language,
  title = {{Language-Coupled Reinforcement Learning for Multilingual Retrieval-Augmented Generation}},
  author = {Rui Qi, Fengran Mo, Yufeng Chen, Xue Zhang, Shuo Wang, Hongliang Li, Xu Jinan, Meng Jiang, Jian-Yun Nie and Kaiyu Huang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.700},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.700}
}

References

44 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Akari Asai, Jungo Kasai, Jonathan Clark, Kenton Lee, Eunsol Choi, and Hannaneh Hajishirzi. 2021. XOR QA: Cross-lingual open-retrieval question answering. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  2. Mingyang Chen, Linzhuang Sun, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Haofen Wang, Jeff Z. Pan, Wen Zhang, Huajun Chen, Fan Yang, Zenan Zhou, and Weipeng Chen. 2025. Research: Learning to reason with search for llms via reinforcement learning. Preprint, arXiv:2503.19470. find
  3. Nadezhda Chirkova, David Rau, Hervé Déjean, Thibault Formal, Stéphane Clinchant, and Vassilina Nikoulina. 2024. Retrieval-augmented generation in multilingual settings. In Proceedings of the 1st Workshop on Towards Knowledgeable Language Models (KnowLLM 2024), pages 177–188. find
  4. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, and 1 others. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53. find
  5. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. Deepseek-r1: Incentivizing reasoning capability in ll…
  6. Wenlong Deng, Yushu Li, Boying Gong, Yi Ren, Christos Thrampoulidis, and Xiaoxiao Li. 2025. On grpo collapse in search-r1: The lazy likelihood-displacement death spiral. Preprint, arXiv:2512.04220. — [arXiv:2512.04220](https://arxiv.org/abs/2512.04220) find
  7. Hanxing Ding, Liang Pang, Zihao Wei, Huawei Shen, and Xueqi Cheng. 2025. Rowen: Adaptive retrieval-augmented generation for hallucination mitigation in llms. Preprint, arXiv:2402.10612. — [arXiv:2402.10612](https://arxiv.org/abs/2402.10612) find
  8. Kaiyu Huang, Fengran Mo, Xinyu Zhang, Hongliang Li, You Li, Yuanchi Zhang, Weijian Yi, Yulong Mao, Jinchen Liu, Yuzhuang Xu, and 1 others. 2024. A survey on large language models with multilingualism: Recent advances and new frontiers. arXiv preprint arXiv:2405.10936. — [arXiv:2405.10936](https://arxiv.org/abs/2405.10936) find
  9. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. Preprint, arXiv:2503.09516.
  10. Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, and 1 others. 2024. Training language models to self-correct via reinforcement learning. arXiv preprint arXiv:2409.12917.
  11. Somnath Kumar, Vaibhav Balloli, Mercy Ranjit, Kabir Ahuja, Sunayana Sitaram, Kalika Bali, Tanuja Ganu, and Akshay Nambi. 2025. Bridging the language gap: Dynamic learning strategies for improving multilingual performance in LLMs. In Proceedings of the 31st International Conference on Computational L… find
  12. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural…
  13. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, and 1 others. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33:9459–…
  14. Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. 2025. Search-o1: Agentic search-enhanced large reasoning models. arXiv preprint arXiv:2501.05366.
  15. Chenyu Lin, Yilin Wen, Du Su, Hexiang Tan, Fei Sun, Muhan Chen, Chenfu Bao, and Zhonghou Lyu. 2025. Resisting contextual interference in rag via parametric-knowledge reinforcement. Preprint, arXiv:2506.05154. — [arXiv:2506.05154](https://arxiv.org/abs/2506.05154) find
  16. Shayne Longpre, Yi Lu, and Joachim Daiber. 2021. MKQA: A linguistically diverse benchmark for multilingual open domain question answering. Transactions of the Association for Computational Linguistics, 9:1389–1406.
  17. Qi Luo, Xiaonan Li, Yuxin Wang, Tingshuo Fan, Yuan Li, Xinchi Chen, and Xipeng Qiu. 2025. Marag-r1: Beyond single retriever via reinforcement-learned multi-tool agentic retrieval. Preprint, arXiv:2510.27569. — [arXiv:2510.27569](https://arxiv.org/abs/2510.27569) find
  18. Simon Lupart, Mohammad Aliannejadi, and Evangelos Kanoulas. 2025. Chatr1: Reinforcement learning for conversational reasoning and retrieval augmented question answering. arXiv preprint arXiv:2510.13312.
  19. Zhibo Man, Yuanmeng Chen, Yujie Zhang, and Jinan Xu. 2025. DMDTEval: An evaluation and analysis of LLMs on disambiguation in multi-domain translation. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 6065–6082, Suzhou, China. Association for Computatio… find
  20. Zhibo Man, Kaiyu Huang, Yujie Zhang, Yuanmeng Chen, Yufeng Chen, and Jinan Xu. 2024. ICL: Iterative continual learning for multi-domain neural machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 7732–7743, Miami, Florida, USA. Association for Computat… find
  21. Fengran Mo, Yifan Gao, Sha Li, Hansi Zeng, Xin Liu, Zhaoxuan Tan, Xian Li, Jianshu Chen, Dakuo Wang, and Meng Jiang. 2026a. Agentic conversational search with contextualized reasoning via reinforcement learning. Preprint, arXiv:2601.13115.
  22. Fengran Mo, Zhan Su, Yuchen Hui, Jinghan Zhang, Jia Ao Sun, Zheyuan Liu, Chao Zhang, Tetsuya Sakai, and Jian-Yun Nie. 2026b. Opendecoder: Open large language model decoding to incorporate document quality in rag. arXiv preprint arXiv:2601.09028. — [arXiv:2601.09028](https://arxiv.org/abs/2601.09028) find
  23. Shiyu Ni, Keping Bi, Jiafeng Guo, and Xueqi Cheng. 2024. When do LLMs need retrieval augmentation? mitigating LLMs’ overconfidence helps retrieval augmentation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 11375–11388, Bangkok, Thailand. Association for Computational… find
  24. Jeonghyun Park and Hwanhee Lee. 2025. Investigating language preference of multilingual RAG systems. In Findings of the Association for Computational Linguistics: ACL 2025, pages 5647–5675, Vienna, Austria. Association for Computational Linguistics. find
  25. Rui Qi, Zhibo Man, Yufeng Chen, Fengran Mo, Jinan Xu, and Kaiyu Huang. 2025. Sot: Structured-of-thought prompting guides multilingual reasoning in large language models. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 11024–11039. find
  26. Leonardo Ranaldi, Barry Haddow, and Alexandra Birch. 2025a. Multilingual retrieval-augmented generation for knowledge-intensive task. Preprint, arXiv:2504.03616.
  27. Leonardo Ranaldi, Federico Ranaldi, Fabio Massimo Zanzotto, Barry Haddow, and Alexandra Birch. 2025b. Improving multilingual retrieval-augmented language models through dialectic reasoning argumentations. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, page…
  28. Manon Reusens, Philipp Borchert, Margot Mieskes, Jochen De Weerdt, and Bart Baesens. 2023. Investigating bias in multilingual language models: Cross-lingual transfer of debiasing techniques. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2887–2896. find
  29. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36:68539–68551.
  30. Huatong Song, Jinhao Jiang, Wenqing Tian, Zhipeng Chen, Yuhuan Wu, Jiahao Zhao, Yingqian Min, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. 2025. R1-searcher++: Incentivizing the dynamic knowledge acquisition of llms via reinforcement learning. Preprint, arXiv:2505.17005. — [arXiv:2505.17005](https://arxiv.org/abs/2505.17005) find
  31. Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2023. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers), p…
  32. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024a. Multilingual e5 text embeddings: A technical report. arXiv preprint arXiv:2402.05672.
  33. Wenmin Wang, Peilin Zhang, Ge Liu, Ruihua Wu, and Guixiang Song. 2024b. Investigating on the external knowledge in rag for zero-shot cross-language transfer. In 2024 IEEE 4th International Conference on Electronic Technology, Communication and Information (ICETCI), pages 1479–1484. find
  34. Junlin Wu, Xianrui Zhong, Jiashuo Sun, Bolian Li, Bowen Jin, Jiawei Han, and Qingkai Zeng. 2025a. Structure-r1: Dynamically leveraging structural knowledge in llm reasoning through reinforcement learning. Preprint, arXiv:2510.15191. — [arXiv:2510.15191](https://arxiv.org/abs/2510.15191) find
  35. Suhang Wu, Jialong Tang, Baosong Yang, Ante Wang, Kaidi Jia, Jiawei Yu, Junfeng Yao, and Jinsong Su. 2024. Not all languages are equal: Insights into multilingual retrieval-augmented generation. Preprint, arXiv:2410.21970. — [arXiv:2410.21970](https://arxiv.org/abs/2410.21970) find
  36. Yicong Wu, Guangyue Lu, Yuan Zuo, Huarong Zhang, and Junjie Wu. 2025b. Graph-r1: Incentivizing the zero-shot graph learning capability in LLMs via explicit reasoning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 23920–23938, Suzhou, China. Associat… find
  37. Zhichao Xu, Minheng Wang, Yawei Wang, Wenqian Ye, Yuntao Du, Yunpu Ma, and Yijun Tian. 2025. Recon: Reasoning with condensation for efficient retrieval-augmented generation. Preprint, arXiv:2510.10448. find
  38. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  39. Jinghan Zhang, Fengran Mo, Xiting Wang, and Kunpeng Liu. 2024. Blind spot navigation in llm reasoning with thought space explorer. arXiv preprint arXiv:2410.24155. — [arXiv:2410.24155](https://arxiv.org/abs/2410.24155) find
  40. Jinghan Zhang, Xiting Wang, Weijieying Ren, Lu Jiang, Dongjie Wang, and Kunpeng Liu. 2025a. Ratt: A thought structure for coherent and correct llm reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 26733–26741. find
  41. Xue Zhang, Yunlong Liang, Fandong Meng, Songming Zhang, Yufeng Chen, Jinan Xu, and Jie Zhou. 2025b. CM-align: Consistency-based multilingual alignment for large language models. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 25689–25702, Suzhou, China. Association fo… find
  42. Xue Zhang, Yunlong Liang, Fandong Meng, Songming Zhang, Yufeng Chen, Jinan Xu, and Jie Zhou. 2025c. Less, but better: Efficient multilingual expansion for LLMs via layer-wise mixture-of-experts. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Lon…
  43. Xue Zhang, Yunlong Liang, Fandong Meng, Songming Zhang, Yufeng Chen, Jinan Xu, and Jie Zhou. 2025d. Multilingual knowledge editing with language-agnostic factual neurons. In Proceedings of the 31st International Conference on Computational Linguistics, pages 5775–5788, Abu Dhabi, UAE. Association fo…
  44. Xue Zhang, Yunlong Liang, Fandong Meng, Songming Zhang, Kaiyu Huang, Yufeng Chen, Jinan Xu, and Jie Zhou. 2026. Think natively: Unlocking multilingual reasoning with consistency-enhanced reinforcement learning. Preprint, arXiv:2510.07300.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/qi-2026-language-coupled-reinforcement-learning-multilingual/