Elamur Research⌕
☰
home / papers / 2025 / You Don't Need Pre-built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structures

You Don't Need Pre-built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structures

Paper · 2025 · Shengyuan Chen, Chuang Zhou, Zheng Yuan et al.

Summary

Graph-based RAG systems such as Microsoft GraphRAG, HippoRAG, RAPTOR, LightRAG and KGP first convert the entire corpus into a graph. This step consumes many LLM tokens and is slow to redo when documents change. The resulting graph may also not fit the reasoning pattern a given question needs. The authors propose LogicRAG, which skips offline graph building and instead derives a structure for each query at inference time.

An LLM uses few-shot prompting to split the question into subproblems and links them into a directed acyclic graph of dependencies. A topological sort then gives an order for answering the subproblems one at a time, with each retrieval conditioned on earlier answers. If the evidence for a node is insufficient, new subproblems are added on the fly. Two pruning mechanisms limit cost. Context pruning keeps a rolling LLM-written summary instead of accumulating raw passages, and graph pruning merges similar subproblems.

On multi-hop QA benchmarks (2WikiMQA, HotpotQA, MuSiQue), the authors report higher accuracy and better efficiency than strong baselines. This is useful wherever knowledge bases are large or change often.

Key points

  • On 2WikiMQA, query-time cost of LogicRAG is about 9.83 s and ~1,778 tokens per question. This compares with GraphRAG (13.05 s / ~4,700 tokens), LightRAG (35.14 s / ~5,731), KGP (70.72 s / ~11,098) and HippoRAG2 (5.89 s / ~2,809). These figures exclude the graph-construction cost that only the baselines incur.
  • Accuracy on 2WikiMQA varies by question type:
    • comparison: 89%
    • bridge-comparison: 67%
    • compositional: 50% (this type makes up 44.4% of the questions)
    • inference: 39%
  • On MuSiQue, accuracy drops steadily as the number of hops increases.
  • Retrieval top-k shows diminishing returns: k = 3 or 5 gives the best balance of quality and cost. On MuSiQue, top-20 pushes the average cost above 6,000 tokens.
  • Graph pruning reduces retrieval rounds across all question types, with the largest savings on compositional and multi-hop questions. 4-hop questions used fewer rounds than 3-hop ones, which the authors attribute to the LLM answering prematurely with partial evidence.
  • Code is available at github.com/chensyCN/LogicRAG.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksInformation Systems · 1,437 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksInformation retrieval & search · 473 works

Cite

Cite as Shengyuan Chen, Chuang Zhou, Zheng Yuan, Qinggang Zhang, Zeyang Cui, Hao Chen, Yilin Xiao, Jiannong Cao and Xiao Huang. 2025. You Don't Need Pre-built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structures. arXiv:2508.06105.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2508.06105 · arXiv 2508.06105

@misc{chen2025you,
  title = {{You Don't Need Pre-built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structures}},
  author = {Shengyuan Chen, Chuang Zhou, Zheng Yuan, Qinggang Zhang, Zeyang Cui, Hao Chen, Yilin Xiao, Jiannong Cao and Xiao Huang},
  year = {2025},
  doi = {10.48550/arxiv.2508.06105},
  eprint = {2508.06105},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2508.06105}
}

References

53 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Anthropic. 2024. The Claude 3 Model Family: Opus, Sonnet, Haiku. Claude-3 Model Card. find
  3. Bei, Y.; Chen, H.; Chen, S.; Huang, X.; Zhou, S.; and Huang, F. 2023. Non-recursive cluster-scale graph interacted model for click-through rate prediction. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, 3748–3752. find
  4. Borgeaud, S.; Mensch, A.; Hoffmann, J.; Cai, T.; Rutherford, E.; Millican, K.; Van Den Driessche, G. B.; Lespiau, J.-B.; Damoc, B.; Clark, A.; et al. 2022. Improving language models by retrieving from trillions of tokens. In International Conference on Machine Learning (ICML). find
  5. Chen, H.; Bei, Y.; Huang, W.; Chen, S.; Huang, F.; and Huang, X. 2024a. Graph cross-correlated network for recommendation. IEEE Transactions on Knowledge and Data Engineering. find
  6. Chen, S.; Zhang, Q.; Dong, J.; Hua, W.; Cao, J.; and Huang, X. 2024b. Neuro-Symbolic Entity Alignment via Variational Inference. arXiv preprint arXiv:2410.04153. — [arXiv:2410.04153](https://arxiv.org/abs/2410.04153) find
  7. Chen, S.; Zhang, Q.; Dong, J.; Hua, W.; Li, Q.; and Huang, X. 2024c. Entity Alignment with Noisy Annotations from Large Language Models. arXiv preprint arXiv:2405.16806. — [arXiv:2405.16806](https://arxiv.org/abs/2405.16806) find
  8. Edge, D.; Trinh, H.; Cheng, N.; Bradley, J.; Chao, A.; Mody, A.; Truitt, S.; and Larson, J. 2024. From local to global: A graph rag approach to query-focused summarization. arXiv preprint arXiv:2404.16130. find
  9. Fang, J.; Jiang, H.; Wang, K.; Ma, Y.; Jie, S.; Wang, X.; He, X.; and Chua, T.-S. 2024. Alphaedit: Null-space constrained knowledge editing for language models. arXiv preprint arXiv:2410.02355. find
  10. Fang, J.; Wang, Y.; Wang, R.; Yao, Z.; Wang, K.; Zhang, A.; Wang, X.; and Chua, T.-S. 2025. Safemlrm: Demystifying safety in multi-modal large reasoning models. arXiv preprint arXiv:2504.08813. — [arXiv:2504.08813](https://arxiv.org/abs/2504.08813) find
  11. Fu, Y.; Peng, H.; Ou, L.; Sabharwal, A.; and Khot, T. 2023. Specializing smaller language models towards multi-step reasoning. In International Conference on Machine Learning, 10421–10430.
  12. PMLR. Gao, Y.; Xiong, Y.; Gao, X.; Jia, K.; Pan, J.; Bi, Y.; Dai, Y.; Sun, J.; and Wang, H. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997.
  13. Guo, K.; Shomer, H.; Zeng, S.; Han, H.; Wang, Y.; and Tang, J. 2025. Empowering GraphRAG with Knowledge Filtering and Integration. arXiv preprint arXiv:2503.13804.
  14. Guo, Z.; Xia, L.; Yu, Y.; Ao, T.; and Huang, C. 2024. LightRAG: Simple and Fast Retrieval-Augmented Generation. arXiv preprint arXiv:2410.05779. Gutiérrez, B. J.; Shu, Y.; Gu, Y.; Yasunaga, M.; and Su, Y. 2024. HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models. In Adva…
  15. He, X.; Tian, Y.; Sun, Y.; Chawla, N. V.; Laurent, T.; LeCun, Y.; Bresson, X.; and Hooi, B. 2024. G-retriever: Retrieval-augmented generation for textual graph understanding and question answering. arXiv preprint arXiv:2402.07630.
  16. Ho, X.; Nguyen, A.-K. D.; Sugawara, S.; and Aizawa, A. 2020. Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps. arXiv preprint arXiv:2011.01060.
  17. Hong, Z.; Yuan, Z.; Chen, H.; Zhang, Q.; Huang, F.; and Huang, X. 2024. Knowledge-to-SQL: Enhancing SQL Generation with Data Expert LLM. In Findings of the Association for Computational Linguistics: ACL 2024. find
  18. Hong, Z.; Yuan, Z.; Zhang, Q.; Chen, H.; Dong, J.; Huang, F.; and Huang, X. 2025. Next-generation database interfaces: A survey of llm-based text-to-sql. IEEE Transactions on Knowledge and Data Engineering. find
  19. Huang, L.; Yu, W.; Ma, W.; Zhong, W.; Feng, Z.; Wang, H.; Chen, Q.; Peng, W.; Feng, X.; Qin, B.; et al. 2023. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems (TOIS).
  20. Izacard, G.; Lewis, P.; Lomeli, M.; Hosseini, L.; Petroni, F.; Schick, T.; Dwivedi-Yu, J.; Joulin, A.; Riedel, S.; and Grave, E. 2023. Atlas: Few-shot learning with retrieval augmented language models. The Journal of Machine Learning Research (JMLR).
  21. Jiang, Z.; Xu, F. F.; Gao, L.; Sun, Z.; Liu, Q.; Dwivedi-Yu, J.; Yang, Y.; Callan, J.; and Neubig, G. 2023. Active retrieval augmented generation. In Empirical Methods in Natural Language Processing (EMNLP).
  22. Khashabi, D.; Min, S.; Khot, T.; Sabharwal, A.; Tafjord, O.; Clark, P.; and Hajishirzi, H. 2020. Unifiedqa: Crossing format boundaries with a single qa system. In Findings of Empirical Methods in Natural Language Processing (EMNLP). Lewis, P.; Perez, E.; Piktus, A.; Petroni, F.; Karpukhin, V.; Goyal…
  23. Li, Y.; Li, W.; and Nie, L. 2021. A graph-guided multi-round retrieval method for conversational open-domain question answering. arXiv preprint arXiv:2104.08443. — [arXiv:2104.08443](https://arxiv.org/abs/2104.08443)
  24. Li, Y.; Li, Z.; Wang, P.; Li, J.; Sun, X.; Cheng, H.; and Yu, J. X. 2023a. A survey of graph meets large language model: Progress and future directions. arXiv preprint arXiv:2311.12399. — [arXiv:2311.12399](https://arxiv.org/abs/2311.12399) find
  25. Li, Z.; Wang, L.; Sun, X.; Luo, Y.; Zhu, Y.; Chen, D.; Luo, Y.; Zhou, X.; Liu, Q.; Wu, S.; et al. 2023b. GSLB: the graph structure learning benchmark. Advances in Neural Information Processing Systems, 36: 30306–30318. find
  26. Liu, Z.; Shengyuan, C.; Zhou, K.; Zha, D.; Huang, X.; and Hu, X. 2023. Rsc: accelerate graph neural networks training via randomized sparse computations. In International Conference on Machine Learning, 21951–21968. find
  27. PMLR. LlamaIndex. 2024. LlamaIndex - Build Knowledge Assistants over your Enterprise Data. find
  28. LlamaIndex Blog. Lu, Z.; Ding, K.; Zhang, Y.; Li, J.; Peng, B.; and Liu, L. 2021. Engage the public: Poll question generation for social media posts. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lang… find
  29. Ma, S.; Xu, C.; Jiang, X.; Li, M.; Qu, H.; and Guo, J. 2024. Think-on-graph 2.0: Deep and interpretable large language model reasoning with knowledge graph-guided retrieval. In International Conference on Learning Representations (ICLR). find
  30. OpenAI. 2023. GPT-4 Technical Report.
  31. OpenAI Blog. Peng, B.; Zhu, Y.; Liu, Y.; Bo, X.; Shi, H.; Hong, C.; Zhang, Y.; and Tang, S. 2024. Graph retrieval-augmented generation: A survey. arXiv preprint arXiv:2408.08921.
  32. Sarthi, P.; Abdullah, S.; Tuli, A.; Khanna, S.; Goldie, A.; and Manning, C. D. 2024. RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval. In International Conference on Learning Representations (ICLR).
  33. Shengyuan, C.; Cai, Y.; Fang, H.; Huang, X.; and Sun, M. 2024. Differentiable neuro-symbolic reasoning on large-scale knowledge graphs. Advances in Neural Information Processing Systems, 36. find
  34. Sun, J.; Xu, C.; Tang, L.; Wang, S.; Lin, C.; Gong, Y.; Ni, L.; Shum, H.-Y.; and Guo, J. 2024. Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph. In International Conference on Learning Representations (ICLR). find
  35. Trivedi, H.; Balasubramanian, N.; Khot, T.; and Sabharwal, A. 2022. ♪ MuSiQue: Multihop Questions via Single-hop Question Composition. Transactions of the Association for Computational Linguistics, 10: 539–554.
  36. Trivedi, H.; Balasubramanian, N.; Khot, T.; and Sabharwal, A. 2023. Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions. In The 61st Annual Meeting Of The Association For Computational Linguistics.
  37. Wang, Y.; Lipka, N.; Rossi, R. A.; Siu, A.; Zhang, R.; and Derr, T. 2024. Knowledge graph prompting for multi-document question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 19206– 19214. find
  38. Wu, K.; Chen, H.; Wang, C.; Karray, F.; Li, Z.; Wang, Y.; and Zhong, F. 2025. Hierarchical Instruction-aware Embodied Visual Tracking. arXiv preprint arXiv:2505.20710. — [arXiv:2505.20710](https://arxiv.org/abs/2505.20710) find
  39. Xiang, Z.; Wu, C.; Zhang, Q.; Chen, S.; Hong, Z.; Huang, X.; and Su, J. 2025. When to use graphs in rag: A comprehensive analysis for graph retrieval-augmented generation. arXiv preprint arXiv:2506.05690. — [arXiv:2506.05690](https://arxiv.org/abs/2506.05690) find
  40. Xiao, Y.; Zhou, C.; Zhang, Q.; Dong, S.; Chen, S.; and Huang, X. 2025a. LAG: Logic-Augmented Generation from a Cartesian Perspective. arXiv:2508.05509. — [arXiv:2508.05509](https://arxiv.org/abs/2508.05509) find
  41. Xiao, Y.; Zhou, C.; Zhang, Q.; Li, B.; Li, Q.; and Huang, X. 2025b. Reliable Reasoning Path: Distilling Effective Guidance for LLM Reasoning with Knowledge Graphs. arXiv:2506.10508. — [arXiv:2506.10508](https://arxiv.org/abs/2506.10508) find
  42. Yang, D.; Rao, J.; Chen, K.; Guo, X.; Zhang, Y.; Yang, J.; and Zhang, Y. 2024a. Im-rag: Multi-round retrieval-augmented generation through learning inner monologues. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 730–740. find
  43. Yang, D.; Rao, J.; Chen, K.; Guo, X.; Zhang, Y.; Yang, J.; and Zhang, Y. 2024b. Im-rag: Multi-round retrieval-augmented generation through learning inner monologues. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 730–740. find
  44. Yang, Z.; Qi, P.; Zhang, S.; Bengio, Y.; Cohen, W. W.; Salakhutdinov, R.; and Manning, C. D. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Empirical Methods in Natural Language Processing (EMNLP).
  45. Zhang, F.; Huang, Z.; Zhou, Y.; Guo, Q.; Li, Z.; Luo, W.; Jiang, D.; Fang, Y.; and Zhou, X. 2025a. EraRAG: Efficient and Incremental Retrieval Augmented Generation for Growing Corpora. arXiv preprint arXiv:2506.20963. — [arXiv:2506.20963](https://arxiv.org/abs/2506.20963) find
  46. Zhang, Q.; Dong, J.; Chen, H.; Zha, D.; Yu, Z.; and Huang, X. 2024. Knowgpt: Knowledge graph based prompting for large language models. Advances in Neural Information Processing Systems, 37: 6052–6080. find
  47. Zhang, Q.; Xiang, Z.; Xiao, Y.; Wang, L.; Li, J.; Wang, X.; and Su, J. 2025b. FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generation. arXiv preprint arXiv:2506.08938.
  48. Zhong, F.; Wu, K.; Wang, C.; Chen, H.; Ci, H.; Li, Z.; and Wang, Y. 2024. Unrealzoo: Enriching photo-realistic virtual worlds for embodied ai. arXiv preprint arXiv:2412.20977. — [arXiv:2412.20977](https://arxiv.org/abs/2412.20977) find
  49. Zhou, C.; Wang, Z.; Chen, S.; Du, J.; Zheng, Q.; Xu, Z.; and Huang, X. 2025a. Taming Language Models for Text-attributed Graph Learning with Decoupled Aggregation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 3463–3474.
  50. Zhou, H.; Du, J.; Zhou, C.; Yang, C.; Xiao, Y.; Xie, Y.; and Huang, X. 2025b. Each Graph is a New Language: Graph Learning with LLMs. arXiv preprint arXiv:2501.11478. find
  51. Zhou, H.; Yu, K.; Zhang, Q.; Chen, H.; Zha, D.; Pei, W.; Kong, A.; and Huang, X. 2025c. Self-Monitoring Large Language Models for Click-Through Rate Prediction. ACM Transactions on Information Systems, 44(1): 1–25. find
  52. Zhou, Y.; Su, Y.; Sun, Y.; Wang, S.; Wang, T.; He, R.; Zhang, Y.; Liang, S.; Liu, X.; Ma, Y.; et al. 2025d. In-depth Analysis of Graph-based RAG in a Unified Framework. arXiv preprint arXiv:2503.04338. — [arXiv:2503.04338](https://arxiv.org/abs/2503.04338) find
  53. Zhuang, L.; Chen, S.; Xiao, Y.; Zhou, H.; Zhang, Y.; Chen, H.; Zhang, Q.; and Huang, X. 2025. LinearRAG: Linear Graph Retrieval Augmented Generation on Large-scale Corpora. arXiv preprint arXiv:2510.10114. — [arXiv:2510.10114](https://arxiv.org/abs/2510.10114) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/chen-2025-don-t-need-pre-built/