Elamur Research⌕
☰
home / papers / 2023 / Crawling the Internal Knowledge-Base of Language Models

Crawling the Internal Knowledge-Base of Language Models

Paper · 2023 · Roi Cohen, Mor Geva, Jonathan Berant et al.

Summary

Large language models absorb many facts during pretraining, but there is no transparent way to see what they store. The authors propose LMCRAWL, which converts a model's implicit knowledge into an explicit knowledge graph built around a single seed entity. Earlier extraction work required relations to be given in advance. Here, the model first proposes which relations apply to an entity and then supplies the objects for each relation. Every step uses few-shot in-context prompting without fine-tuning, and the demonstrations are drawn from WikiData. To protect accuracy, the prompts teach the model to reply "Don't know" when it is unsure. These examples come from WikiData facts the model fails to recall. To raise coverage, the model also paraphrases entity and relation names, and the results are merged. Run on GPT-3 (text-davinci-002) with 140 seeds and two expansion hops, the method produces graphs with roughly 82–92% precision and a reasonable number of facts per entity. Some of these facts fall outside the WikiData schema. The work points toward ways to inspect, query and audit what LMs know.

Key points

  • The crawl is split into prompt-based sub-tasks: relation generation, relation paraphrasing, subject paraphrasing, plain object generation, and object generation with a "Don't know" option.
  • In-context examples are built from WikiData. Relation paraphrasing works with an instruction prompt alone and needs no demonstrations.
  • The experiments use GPT-3 (text-davinci-002) on 140 seed entities with depth-2 graphs (e.g., a graph around Alan Turing). Estimated precision is about 82–92%.
  • The model is not limited to a fixed schema. It produced triplets such as (Boston Celtics, championships, 17) and relations like a team's mascot, which WikiData does not contain.
  • Precision was checked automatically via Google search. A manual audit of 1,000 sampled facts showed that 4.1% of the facts marked correct were actually wrong, while 22% of the facts marked incorrect were true. This suggests the real precision is likely higher. Cost and error propagation kept the graphs at two hops.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Allen Institute for Artificial IntelligenceSeattle · United States275 works · 97,634 citations🎓Tel Aviv UniversityTel Aviv · Israel82 works · 10,655 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksKnowledge graphs & ontologies · 192 works

Cite

Cite as Roi Cohen, Mor Geva, Jonathan Berant and Amir Globerson. 2023. Crawling the Internal Knowledge-Base of Language Models. arXiv:2301.12810.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2301.12810 · arXiv 2301.12810

@misc{cohen2023crawling,
  title = {{Crawling the Internal Knowledge-Base of Language Models}},
  author = {Roi Cohen, Mor Geva, Jonathan Berant and Amir Globerson},
  year = {2023},
  doi = {10.48550/arxiv.2301.12810},
  eprint = {2301.12810},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2301.12810}
}

References

28 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dimitrios Alivanistos, Selene Báez Santamaría, Michael Cochez, Jan-Christoph Kalo, Emile van Krieken, and Thiviyan Thanapalasingam. 2022. Prompting as probing: Using language models for knowledge base construction. arXiv preprint arXiv:2208.11057. — [arXiv:2208.11057](https://arxiv.org/abs/2208.11057) find
  2. Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona Diab, and Marjan Ghazvininejad. 2022. A review on language models as knowledge bases. arXiv preprint arXiv:2204.06031. find
  3. Gabor Angeli, Melvin Jose Johnson Premkumar, and Christopher D Manning. 2015. Leveraging linguistic structure for open domain information extraction. pages 344–354.
  4. Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1247–1250. find
  5. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… — Language Models are Few-Shot Learners
  6. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311.
  7. Shrey Desai and Greg Durrett. 2020. Calibration of pre-trained transformers. arXiv preprint arXiv:2003.07892. find
  8. Andrew Drozdov, Nathanael Schärli, Ekin Akyürek, Nathan Scales, Xinying Song, Xinyun Chen, Olivier Bousquet, and Denny Zhou. 2022. Compositional semantic parsing with large language models. arXiv preprint arXiv:2209.15003. find
  9. Anthony Fader, Stephen Soderland, and Oren Etzioni. 2011. Identifying relations for open information extraction. In Proceedings of the 2011 conference on empirical methods in natural language processing, pages 1535–1545. find
  10. Christiane Fellbaum. 2020. WordNet: An Electronic Lexical Database. MIT Press. find
  11. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. 2020. The Pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027. find
  12. Shibo Hao, Bowen Tan, Kaiwen Tang, Hengzhe Zhang, Eric P Xing, and Zhiting Hu. 2022. Bertnet: Harvesting knowledge graphs from pretrained language models. arXiv preprint arXiv:2206.14268. — [arXiv:2206.14268](https://arxiv.org/abs/2206.14268) find
  13. Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. 2021. How can we know when language models know? on the calibration of language models for question answering. Transactions of the Association for Computational Linguistics, 9:962–977. find
  14. Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield Dodds, Nova DasSarma, Eli Tran-Johnson, et al. 2022. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221. find
  15. Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. 2022. Decomposed prompting: A modular approach for solving complex tasks. arXiv preprint arXiv:2210.02406. find
  16. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Teaching models to express their uncertainty in words. arXiv preprint arXiv:2205.14334. find
  17. Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Sebastian Miller, Alexander H. Riedel. 2019. Language models as knowledge bases? EMNLP.
  18. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67.
  19. Simon Razniewski, Andrew Yates, Nora Kassner, and Gerhard Weikum. 2021. Language models as or for knowledge bases. arXiv preprint arXiv:2110.04888. find
  20. Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the parameters of a language model? EMNLP. find
  21. Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. 31(1). find
  22. Yi Tay, Vinh Q Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, et al. 2022. Transformer memory as a differentiable search index. arXiv preprint arXiv:2202.06991.
  23. Neeraj Varshney, Swaroop Mishra, and Chitta Baral. 2022. Investigating selective prediction approaches across several tasks in iid, ood, and adversarial settings. arXiv preprint arXiv:2203.00211. find
  24. Duc-Thuan Vo and Ebrahim Bagheri. 2017. Open information extraction. Encyclopedia with semantic computing and Robotic intelligence, 1(01):1630003. find
  25. Denny Vrandečić and Markus Krötzsch. 2014. Wikidata: a free collaborative knowledgebase. Communications of the ACM, 57(10):78–85.
  26. Denny Vrandečić and Markus Krötzsch. 2014. Wikidata: A free collaborative knowledge base. Communications of the ACM, 57:78–85. find
  27. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903.
  28. Alexander Yates, Michele Banko, Matthew Broadhead, Michael J Cafarella, Oren Etzioni, and Stephen Soderland. 2007. Textrunner: open information extraction on the web. In Proceedings of Human Language Technologies: The Annual Conference of the North American Chapter of the Association for Computation…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/cohen-2023-crawling-internal-knowledge-base-language/