Elamur Research⌕
☰
home / papers / 2024 / How Much Can RAG Help the Reasoning of LLM?

How Much Can RAG Help the Reasoning of LLM?

Paper · 2024 · Jingyu Liu, Jiaen Lin and Yong Liu

Summary

The paper asks whether retrieval-augmented generation can strengthen an LLM's reasoning itself, beyond supplying facts. The authors draw on the result that fixed-depth transformers can only reason to a bounded depth. They model inference as a layered reasoning tree, where retrieved documents can stand in for some intermediate nodes. Their analysis shows that RAG does help, but only modestly. A model limited to depth l reaches roughly l + c, where c depends on retrieval quality, because reading the documents still uses some layers. Real retrieved text is also noisy. The authors argue that fine-tuning such as LoRA has trouble learning to ignore irrelevant tokens without disturbing the attention patterns needed for reasoning. They further argue that relevance judgments involve three tokens at once, while attention works on pairs, so filtering could require many extra layers. Their fix, DPrompt tuning, recasts this as a pair-wise problem. Experiments on NQ with three 7–8B models show it handles distracting documents better than vanilla, LoRA or prompt-tuned baselines. The work clarifies the theoretical limits of RAG and why noise removal is costly.

Key points

  • Theoretical claim: RAG extends reasoning depth only by a constant. The authors assume extracting information from a document costs λ·l layers with λ < 1, so some depth is always spent on processing the documents.
  • Why filtering is hard: Deciding whether a token is relevant can depend on three or more tokens. Pair-wise attention handles this poorly in few layers. DPrompt tuning converts the problem to a pair-wise form.
  • Experimental setup:
    • Models: Meta-Llama-3.1-8B-Instruct, vicuna-7b-v1.5 and Mistral-7B-Instruct-v0.3.
    • Data: 1,000 sampled NQ queries, with gold and distracting documents extracted using the code of Cuconasu et al. (2024).
    • Hardware and tuning: a single RTX 3090, LoRA rank 64, and prompt tuning with 1, 8 or 15 virtual tokens.
  • Main result: In the appendix table for regular document order, DPrompt is best or tied in most settings. With a gold plus distracting document, Llama-3 rises from 0.41 (vanilla) to about 0.60. The authors report larger gains when the query follows the documents.
  • Sparsity: A Wanda pruning experiment suggests RAG makes LLMs more tolerant of sparsity. With a gold document, performance holds up longer as sparsity increases than with the query alone.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China333 works · 87,019 citations🎓Renmin University of ChinaBeijing · China114 works · 16,118 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksInformation Systems · 1,437 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksReasoning · 1,426 works

Cite

Cite as Jingyu Liu, Jiaen Lin and Yong Liu. 2024. How Much Can RAG Help the Reasoning of LLM?. arXiv:2410.02338.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2410.02338 · arXiv 2410.02338

@misc{liu2024how,
  title = {{How Much Can RAG Help the Reasoning of LLM?}},
  author = {Jingyu Liu, Jiaen Lin and Yong Liu},
  year = {2024},
  doi = {10.48550/arxiv.2410.02338},
  eprint = {2410.02338},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2410.02338}
}

References

40 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-rag: Learning to retrieve, generate, and critique through self-reflection. arXiv preprint arXiv:2310.11511, 2023.
  2. Tom B Brown. Language models are few-shot learners. arXiv preprint ArXiv:2005.14165, 2020.
  3. Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Tao He, Haotian Wang, Weihua Peng, Ming Liu, Bing Qin, and Ting Liu. A survey of chain of thought reasoning: Advances, frontiers and future. arXiv preprint arXiv:2309.15402, 2023. find
  4. Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, and Fabrizio Silvestri. The power of noise: Redefining retrieval for rag systems. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in…
  5. Sunhao Dai, Chen Xu, Shicheng Xu, Liang Pang, Zhenhua Dong, and Jun Xu. Bias and unfairness in information retrieval systems: New challenges in the llm era. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 6437–6447, 2024. find
  6. Grégoire Delétang, Anian Ruoss, Paul-Ambroise Duquenne, Elliot Catt, Tim Genewein, Christopher Mattern, Jordi Grau-Moya, Li Kevin Wenliang, Matthew Aitchison, Laurent Orseau, et al. Language modeling is compression. arXiv preprint arXiv:2309.10668, 2023.
  7. Guhao Feng, Bohang Zhang, Yuntian Gu, Haotian Ye, Di He, and Liwei Wang. Towards revealing the mystery behind chain of thought: a theoretical perspective. Advances in Neural Information Processing Systems, 36, 2024. find
  8. Qiuhan Gu. Llm-based code generation method for golang compiler testing. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 2201–2203, 2023. find
  9. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. arXiv preprint arXiv:2311.05232, 2023.
  10. Yuzhen Huang, Jinghan Zhang, Zifei Shan, and Junxian He. Compression represents intelligence linearly. arXiv preprint arXiv:2404.09937, 2024. — [arXiv:2404.09937](https://arxiv.org/abs/2404.09937) find
  11. Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. Atlas: Few-shot learning with retrieval augmented language models. Journal of Machine Learning Research, 24(251): 1–43, 2023.
  12. Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. Llmlingua: Compressing prompts for accelerated inference of large language models. arXiv preprint arXiv:2310.05736, 2023a.
  13. Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. Longllmlingua: Accelerating and enhancing llms in long context scenarios via prompt compression. arXiv preprint arXiv:2310.06839, 2023b.
  14. Kenji Kawaguchi, Zhun Deng, Xu Ji, and Jiaoyang Huang. How does information bottleneck help deep learning? In International Conference on Machine Learning, pp. 16049–16096. PMLR, 2023. find
  15. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:4…
  16. Robert Lakatos, Peter Pollner, Andras Hajdu, and Tamas Joo. Investigating the performance of retrieval-augmented generation and fine-tuning for the development of ai-driven knowledge-based systems. arXiv preprint arXiv:2403.09727, 2024. — [arXiv:2403.09727](https://arxiv.org/abs/2403.09727) find
  17. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33: 9459–9474, 2020…
  18. Zhiyuan Li, Hong Liu, Denny Zhou, and Tengyu Ma. Chain of thought empowers transformers to solve inherently serial problems. arXiv preprint arXiv:2402.12875, 2024. find
  19. William Merrill and Ashish Sabharwal. The parallelism tradeoff: Limitations of log-precision transformers. Transactions of the Association for Computational Linguistics, 11:531–545, 2023. find
  20. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 27730–27744, 2022.
  21. Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Rühle, Yuqing Yang, Chin-Yew Lin, et al. Llmlingua-2: Data distillation for efficient and faithful task-agnostic prompt compression. arXiv preprint arXiv:2403.12968, 2024.
  22. Aleksandar Petrov, Philip HS Torr, and Adel Bibi. When do prompting and prefix-tuning work? a theory of capabilities and limitations. arXiv preprint arXiv:2310.19698, 2023. — [arXiv:2310.19698](https://arxiv.org/abs/2310.19698) find
  23. Aleksandar Petrov, Philip HS Torr, and Adel Bibi. Prompting a pretrained transformer can be a universal approximator. arXiv preprint arXiv:2402.14753, 2024. — [arXiv:2402.14753](https://arxiv.org/abs/2402.14753) find
  24. Clayton Sanford, Daniel J Hsu, and Matus Telgarsky. Representational strengths and limitations of transformers. Advances in Neural Information Processing Systems, 36, 2024. find
  25. Jonathan Scarlett and Volkan Cevher. An introductory guide to fano’s inequality with applications in statistical estimation. arXiv preprint arXiv:1901.00555, 2019. — [arXiv:1901.00555](https://arxiv.org/abs/1901.00555) find
  26. Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed Chi, Nathanael Schärli, and Denny Zhou. Large language models can be easily distracted by irrelevant context, 2023a. URL https://arxiv.org/abs/2302.00093. ↗
  27. Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. Replug: Retrieval-augmented black-box language models. arXiv preprint arXiv:2301.12652, 2023b.
  28. Heydar Soudani, Evangelos Kanoulas, and Faegheh Hasibi. Fine tuning vs. retrieval augmented generation for less popular knowledge. arXiv preprint arXiv:2403.01432, 2024. — [arXiv:2403.01432](https://arxiv.org/abs/2403.01432) find
  29. Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. A simple and effective pruning approach for large language models. arXiv preprint arXiv:2306.11695, 2023. find
  30. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022.
  31. Siye Wu, Jian Xie, Jiangjie Chen, Tinghui Zhu, Kai Zhang, and Yanghua Xiao. How easily do irrelevant inputs skew the responses of large language models?, 2024. URL https://arxiv.org/abs/2404.03302. ↗
  32. Shicheng Xu, Liang Pang, Huawei Shen, and Xueqi Cheng. Unveil the duality of retrieval-augmented generation: Theoretical analysis and practical solution. arXiv preprint arXiv:2406.00944, 2024. — [arXiv:2406.00944](https://arxiv.org/abs/2406.00944) find
  33. Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, and Zhen-Hua Ling. Corrective retrieval augmented generation. arXiv preprint arXiv:2401.15884, 2024.
  34. Hongkang Yang, Zehao Lin, Wenjin Wang, Hao Wu, Zhiyu Li, Bo Tang, Wenqiang Wei, Jinbo Wang, Zeyun Tang, Shichao Song, et al. Memory3: Language modeling with explicit memory. arXiv preprint arXiv:2407.01178, 2024. — [arXiv:2407.01178](https://arxiv.org/abs/2407.01178) find
  35. Andrew Chi-Chih Yao. Some complexity questions related to distributive computing(preliminary report). In Proceedings of the Eleventh Annual ACM Symposium on Theory of Computing, STOC ’79, pp. 209–213, New York, NY, USA, 1979. Association for Computing Machinery. ISBN 9781450374385. doi: 10.1145/8001… — [doi:10.1145/800135.804414](https://doi.org/10.1145/800135.804414) find
  36. Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Berant. Making retrieval-augmented language models robust to irrelevant context. arXiv preprint arXiv:2310.01558, 2023.
  37. Yaodong Yu, Sam Buchanan, Druv Pai, Tianzhe Chu, Ziyang Wu, Shengbang Tong, Benjamin Ha-effele, and Yi Ma. White-box transformers via sparse rate reduction. Advances in Neural Information Processing Systems, 36:9422–9457, 2023. find
  38. Yuchen Zeng and Kangwook Lee. The expressive power of low-rank adaptation. arXiv preprint arXiv:2310.17513, 2023. — [arXiv:2310.17513](https://arxiv.org/abs/2310.17513) find
  39. Tianjun Zhang, Shishir G Patil, Naman Jain, Sheng Shen, Matei Zaharia, Ion Stoica, and Joseph E Gonzalez. Raft: Adapting language model to domain specific rag. arXiv preprint arXiv:2403.10131, 2024.
  40. Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, and Bin Cui. Retrieval-augmented generation for ai-generated content: A survey. arXiv preprint arXiv:2402.19473, 2024.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2024-much-rag-help-reasoning-llm/