Elamur Research⌕
☰
home / papers / 2025 / OG-RAG: Ontology-grounded retrieval-augmented generation for large language models

OG-RAG: Ontology-grounded retrieval-augmented generation for large language models

Summary

General-purpose LLMs often underperform in specialized fields such as agriculture, law, healthcare, or journalism. Fine-tuning is costly, and standard retrieval pipelines use domain-agnostic embeddings that ignore how experts actually structure knowledge. OG-RAG addresses this by using an existing domain ontology, meaning the entities and relations experts have defined, to guide retrieval.

The method works in three steps:

  • Source documents are mapped onto the ontology, producing self-contained "factual blocks" of subject–attribute–value triples.
  • These blocks are organized into a hypergraph, where each hyperedge groups related facts.
  • For a given query, a greedy optimization picks a small set of hyperedges that covers the relevant information. This set becomes a compact context for the LLM.

The authors evaluated OG-RAG on agriculture and news data with four LLMs, comparing it against RAG, RAPTOR and GraphRAG. It improved fact recall and answer correctness, made answers easier for users to trace back to sources, and supported rule-based deduction better than the baselines. Its main practical requirement is that a well-defined ontology already exists for the domain.

Key points

  • Across four LLMs, OG-RAG raised recall of correct facts by about 55% and answer correctness by about 40% compared with baseline retrievers.
  • In a user study with 16 participants, people attributed responses to the supporting context roughly 30% faster.
  • On deductive-reasoning questions about agricultural data (e.g., the Soybean dataset), it improved correctness by about 27%, scored with RAGAS metrics using GPT-4o and GPT-4o-mini.
  • Performance stayed stable when up to roughly half of the ontology's attributes were randomly removed.
  • The code is released at github.com/microsoft/ograg2.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics Language models & pretraining Knowledge graphs & ontologies Information retrieval & search

Institutions

🏢MicrosoftRedmond · United States193 works · 110,465 citations🎓Georgia Institute of TechnologyAtlanta · United States55 works · 21,012 citations🏢LenovoBeijing · China2 works · 124 citations

Cite

Cite as Kartik Sharma, Peeyush Kumar and Yunqing Li. 2025. OG-RAG: Ontology-grounded retrieval-augmented generation for large language models. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.emnlp-main.1674

@inproceedings{sharma2025og,
  title = {{OG-RAG: Ontology-grounded retrieval-augmented generation for large language models}},
  author = {Kartik Sharma, Peeyush Kumar and Yunqing Li},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.1674},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.1674}
}

References

42 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hamed Babaei Giglou, Jennifer D’Souza, and Sören Auer. 2023. Llms4ol: Large language models for ontology learning. In International Semantic Web Conference, pages 408–427. Springer. find
  2. Angels Balaguer, Vinamra Benara, Renato Luiz de Freitas Cunha, Roberto de M Estevão Filho, Todd Hendry, Daniel Holstein, Jennifer Marsman, Nick Mecklenburg, Sara Malvar, Leonardo O Nunes, and 1 others. 2024. Rag vs fine-tuning: Pipelines, tradeoffs, and a case study on agriculture. arXiv e-prints, p… find
  3. Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, and 1 others. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258. our card
  4. Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, and 1 others. 2022. Improving language models by retrieving from trillions of tokens. In International conference on machin… find
  5. Andres M Bran, Alexandru Oarga, Matthew Hart, Magdalena Lederbauer, and Philippe Schwaller. 2025. find
  6. Ontology-retrieval augmented generation for scientific discovery. find
  7. Fengyu Cai, Xinran Zhao, Tong Chen, Sihao Chen, Hongming Zhang, Iryna Gurevych, and Heinz Koeppl. 2024. Mixgr: Enhancing retriever generalization for scientific domain through complementary granularity. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages… find
  8. Marco Cascella, Jonathan Montomoli, Valentina Bellini, and Elena Bignami. 2023. Evaluating the feasibility of chatgpt in healthcare: an analysis of multiple clinical and research scenarios. Journal of medical systems, 47(1):33. find
  9. ChatGPT. 2024. https://chatgpt.com/. [Accessed 08-10-2024]. ↗
  10. Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. 2024. From local to global: A graph rag approach to query-focused summarization. arXiv preprint arXiv:2404.16130. find
  11. Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. 2023. Enabling large language models to generate text with citations. arXiv preprint arXiv:2305.14627. our card
  12. Kaisi Guan, Qian Cao, Yuchong Sun, Xiting Wang, and Ruihua Song. 2024. Bsharedrag: Backbone shared retrieval-augmented generation for the e-commerce domain. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 1137–1158. find
  13. Nicola Guarino, Daniel Oberle, and Steffen Staab. 2009. What is an ontology? Handbook on ontologies, pages 1–17. find
  14. Neel Guha, Julian Nyarko, Daniel Ho, Christopher Ré, Adam Chilton, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel Rockmore, Diego Zambrano, and 1 others. 2024. Legalbench: A collaboratively built benchmark for measuring legal reasoning in large language models. Advances in Neural Informati… our card
  15. Xiaoxin He, Yijun Tian, Yifei Sun, Nitesh Chawla, Thomas Laurent, Yann LeCun, Xavier Bresson, and Bryan Hooi. 2024. G-retriever: Retrieval-augmented generation for textual graph understanding and question answering. Advances in Neural Information Processing Systems, 37:132876–132907. our card
  16. Peter Jackson. 1990. Introduction to expert systems. Addison-Wesley Longman Publishing Co., Inc. find
  17. Ehsan Kamalloo, Aref Jafari, Xinyu Zhang, Nandan Thakur, and Jimmy Lin. 2023. Hagrid: A human-llm collaborative dataset for generative information-seeking with attribution. arXiv preprint arXiv:2307.16883. — [arXiv:2307.16883](https://arxiv.org/abs/2307.16883) find
  18. Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04906. our card
  19. Muhammad Khalifa, David Wadden, Emma Strubell, Honglak Lee, Lu Wang, Iz Beltagy, and Hao Peng. 2024. Source-aware training enables knowledge attribution in language models. arXiv preprint arXiv:2404.01019. find
  20. Bernhard H Korte, Jens Vygen, B Korte, and J Vygen. 2011. Combinatorial optimization, volume 1. Springer. find
  21. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, and 1 others. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459–… find
  22. Mufei Li, Siqi Miao, and Pan Li. 2024. Simple is effective: The roles of graphs and large language models in knowledge-graph-based retrieval-augmented generation. arXiv preprint arXiv:2410.20724. — [arXiv:2410.20724](https://arxiv.org/abs/2410.20724) find
  23. Linhao Luo, Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan. 2023. Reasoning on graphs: Faithful and interpretable large language model reasoning. arXiv preprint arXiv:2310.01061. find
  24. Marjan Mernik, Jan Heering, and Anthony M Sloane. 2005. When and how to develop domain-specific languages. ACM computing surveys (CSUR), 37(4):316– 344. find
  25. Oded Ovadia, Menachem Brief, Moshik Mishaeli, and Oren Elisha. 2023. Fine-tuning or retrieval? comparing knowledge injection in llms. arXiv preprint arXiv:2312.05934. find
  26. Perplexity. 2024. https://www.perplexity.ai/. [Accessed 08-10-2024]. ↗
  27. RAGAS. 2024. https://docs.ragas.io/en/ stable/. [Accessed 08-10-2024]. ↗
  28. Abulhair Saparov, Richard Yuanzhe Pang, Vishakh Padmakumar, Nitish Joshi, Mehran Kazemi, Najoung Kim, and He He. 2023. Testing the general deductive reasoning capacity of large language models using ood examples. Advances in Neural Information Processing Systems, 36:3083–3105. find
  29. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36. Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, and 1 others. 2023. Large language… our card
  30. Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kaluarachchi, Rajib Rana, and Suranga Nanayakkara. 2023. Improving the domain adaptation of retrieval augmented generation (rag) models for open domain question answering. Transactions of the Association for Computational Linguistics,… our card
  31. Aviv Slobodkin, Eran Hirsch, Arie Cattan, Tal Schuster, and Ido Dagan. 2024. Attribute first, then generate: Locally-attributable grounded text generation. arXiv preprint arXiv:2403.17104. find
  32. Yixuan Tang and Yi Yang. 2024. Multihop-rag: Benchmarking retrieval-augmented generation for multi-hop queries. arXiv preprint arXiv:2401.15391. our card
  33. Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. 2023. Large language models in medicine. Nature medicine, 29(8):1930– 1940. find
  34. Siyuan Wang, Zhongyu Wei, Yejin Choi, and Xiang Ren. 2024. Can llms reason with rules? logic scaffolding for stress-testing and improving llms. arXiv preprint arXiv:2402.11442. find
  35. Jinfeng Xiao, Linyi Ding, James Barry, Mohab Elkaref, Geeth De Mel, and Jiawei Han. 2024. Orag: Ontology-guided retrieval-augmented generation for theme-specific entity typing. In First Conference on Language Modeling. find
  36. Ran Xu, Hui Liu, Sreyashi Nag, Zhenwei Dai, Yaochen Xie, Xianfeng Tang, Chen Luo, Yang Li, Joyce C Ho, Carl Yang, and 1 others. 2025. Simrag: Self-improving retrieval-augmented generation for adapting large language models to specialized domains. In Proceedings of the 2025 Conference of the Nations… find
  37. Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D Manning. 2024. Raptor: Recursive abstractive processing for tree-organized retrieval. arXiv preprint arXiv:2401.18059. our card
  38. Ran Xu, Wenqi Shi, Yue Yu, Yuchen Zhuang, Yanqiao Zhu, May Dongmei Wang, Joyce Ho, Chao Zhang, and Carl Yang. 2024. Bmretriever: Tuning large language models as better biomedical text retrievers. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 22234–2… find
  39. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2024. find
  40. Nan Zhang, Prafulla Kumar Choubey, Alexander Fabbri, Gabriel Bernadett-Shapiro, Rui Zhang, Prasenjit Mitra, Caiming Xiong, and Chien-Sheng Wu. 2024a. Sirerag: Indexing similar and related information for multihop reasoning. arXiv preprint arXiv:2412.06206. — [arXiv:2412.06206](https://arxiv.org/abs/2412.06206) find
  41. Tianjun Zhang, Shishir G Patil, Naman Jain, Sheng Shen, Matei Zaharia, Ion Stoica, and Joseph E Gonzalez. 2024b. Raft: Adapting language model to domain specific rag. arXiv preprint arXiv:2403.10131. our card
  42. Chengshuai Zhao, Garima Agrawal, Tharindu Kumarage, Zhen Tan, Yuli Deng, Ying-Chih Chen, and Huan Liu. 2024. Ontology-aware rag for improved question-answering in cybersecurity education. arXiv preprint arXiv:2412.14191. — [arXiv:2412.14191](https://arxiv.org/abs/2412.14191) find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sharma-2025-og-rag-ontology-grounded-retrieval/