Elamur Research⌕
☰
home / papers / 2026 / Rethinking Retrieval-Augmented Generation as a Cooperative Decision-Making Problem

Rethinking Retrieval-Augmented Generation as a Cooperative Decision-Making Problem

Paper · 2026 · Lichang Song, Ting Long and Yi Chang

Summary

Most RAG pipelines treat reranking and generation as a one-way chain. The reranker fixes a document order, and the generator depends heavily on that order. A single misranked passage can therefore derail the answer even when the right evidence sits among the top candidates. The authors recast RAG as a cooperative multi-agent problem. The reranker and the generator act as peer agents trained toward one shared reward: whether the produced answer contains the gold answer. Their framework, CoRAG, uses BGE-Reranker as the reranker and an LLM generator tuned with GRPO. Standard MARL optimizers fit ranking poorly because of an MDP-ranking mismatch. For example, separate agents may pick the same document for several positions. CoRAG avoids this with a GRPO-style reranker objective. The task-level reward is converted into smoothed, sampled preference labels for each document, and the reranker learns from them through a margin-based pairwise loss. Training uses only about 10K PopQA examples. The model still beats baselines and transfers to other datasets, including code and table QA. This suggests that joint, outcome-driven training makes RAG output more stable.

Key points

  • Credit assignment: each document gets a binary success signal, averaged over past iterations. This average is smoothed into a Bernoulli parameter (α ∈ (0, 0.5)) and sampled to produce preference labels. The GRPO objective is then reduced to a deterministic pairwise learning-to-rank loss.
  • Training data: about 10K PopQA samples, with a reward of 1 if the response contains the ground-truth answer and 0 otherwise.
  • LLM-as-a-judge on PopQA: CoRAG scores 71.35 on average, against 64.60 for InstructRAG and 41.38 for RetRobust (recomputed; the paper's table prints 45.69). Judges were Llama-3-Instruct 8B, gpt-4o, deepseek-v3.2 and qwen3-vl-235b-a22b. The pattern-based score is 71.26 versus 66.19 for InstructRAG.
  • Code and table QA: the generator gets the best pass@10 on HumanEval/HumanEval+ and the best pass@1 on HumanEval+. On WikiTable Questions it nearly matches the top baseline.
  • Limitation: making the generator less sensitive to ranking may also reduce the payoff from future reranker improvements. Code is available at github.com/CoderrrSong/CoRAG.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Jilin UniversityChangchun · China25 works · 5,252 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksInformation Systems · 1,437 works

🧭 Topics 1Research subjects and languages the work is about.Information retrieval & search · 473 works

Cite

Cite as Lichang Song, Ting Long and Yi Chang. 2026. Rethinking Retrieval-Augmented Generation as a Cooperative Decision-Making Problem. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.2157

@inproceedings{song2026rethinking,
  title = {{Rethinking Retrieval-Augmented Generation as a Cooperative Decision-Making Problem}},
  author = {Lichang Song, Ting Long and Yi Chang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.2157},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.2157}
}

References

27 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024. Self-rag: Learning to retrieve, generate, and critique through self-reflection.
  2. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  3. Francesco Casalegno. 2022. Learning to rank: A complete guide to ranking using machine learning. Towards Data Science. find
  4. Mark Chen. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
  5. Yiqun Chen, Lingyong Yan, Weiwei Sun, Xinyu Ma, Yi Zhang, Shuaiqiang Wang, Dawei Yin, Yiming Yang, and Jiaxin Mao. 2025. Improving retrieval-augmented generation through multi-agent reinforcement learning. arXiv preprint arXiv:2501.15228. — [arXiv:2501.15228](https://arxiv.org/abs/2501.15228) find
  6. Yuqin Dai, Guoqing Wang, Yuan Wang, Kairan Dou, Kaichen Zhou, Zhanwei Zhang, Shuo Yang, Fei Tang, Jun Yin, Pengyu Zeng, and 1 others. 2025. Evinoterag: Enhancing rag models via answer-supportive evidence notes. arXiv preprint arXiv:2509.00877. — [arXiv:2509.00877](https://arxiv.org/abs/2509.00877) find
  7. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  8. Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551.
  9. Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. 2022. Decomposed prompting: A modular approach for solving complex tasks. arXiv preprint arXiv:2210.02406. find
  10. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, and 1 others. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Lin…
  11. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, and 1 others. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33:9459–…
  12. Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V… find
  13. Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. 2023a. Enabling large language models to generate text with citations. arXiv preprint arXiv:2305.14627.
  14. Multi-Linguality Multi-Functionality Multi-Granularity. 2024. M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation.
  15. Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yixin Dai, Jiawei Sun, Haofen Wang, and Haofen Wang. 2023b. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997, 2(1).
  16. Thang Nguyen, Peter Chin, and Yu-Wing Tai. 2025. Ma-rag: Multi-agent retrieval-augmented generation via collaborative chain-of-thought reasoning. arXiv preprint arXiv:2505.20096.
  17. Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps. arXiv preprint arXiv:2011.01060.
  18. AJ Oche, AG Folashade, T Ghosal, and A Biswas. A systematic review of key retrieval-augmented generation (rag) systems: Progress, gaps, and future directions. arxiv 2024. arXiv preprint arXiv:2409.15730. — [arXiv:2409.15730](https://arxiv.org/abs/2409.15730) find
  19. Pengyue Jia, Derong Xu, Xiaopeng Li, Zhaocheng Du, Xiangyang Li, Yichao Wang, Yuhao Wang, Qidong Liu, Maolin Wang, Huifeng Guo, and 1 others. 2025. Bridging relevance and reasoning: Rationale distillation in retrieval-augmented generation. In Findings of the Association for Computational Linguistics… find
  20. Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Active retrieval augmented generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7969–7992.
  21. Panupong Pasupat and Percy Liang. 2015. Compositional semantic parsing on semi-structured tables. arXiv preprint arXiv:1508.00305.
  22. Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, and 1 others. 2018. Improving language understanding by generative pre-training. find
  23. Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented language models. Transactions of the Association for Computational Linguistics, 11:1316–1331.
  24. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  25. Chaitanya Sharma. 2025. Retrieval-augmented generation: A comprehensive survey of architectures, enhancements, and robustness frontiers. arXiv preprint arXiv:2506.00054. — [arXiv:2506.00054](https://arxiv.org/abs/2506.00054) find
  26. Weizhou Shen, Yeyun Gong, Yelong Shen, Song Wang, Xiaojun Quan, Nan Duan, and Weizhu Chen. 2023. Joint generator-ranker learning for natural language generation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 7681–7699. find
  27. Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. 2022. Asqa: Factoid questions meet long-form answers. arXiv preprint arXiv:2204.06092.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/song-2026-rethinking-retrieval-augmented-generation-as/