Elamur Research⌕
☰
home / papers / 2023 / Methods for Measuring, Updating, and Visualizing Factual Beliefs in Language Models

Methods for Measuring, Updating, and Visualizing Factual Beliefs in Language Models

Paper · 2023 · Peter Hase, Mona Diab, Asli Celikyilmaz et al.

Summary

The paper investigates how well language models of roughly 100M parameters hold coherent factual "beliefs" and how those beliefs can be edited after training without retraining. The authors use models finetuned for fact verification and question answering, with data from zsRE, FEVER, LeapOfThought and Wikidata5m. They test whether answers stay stable across paraphrases and whether they agree with logically entailed facts. They also add two evaluation metrics: success on entailed data, and preservation of similar but logically unrelated ("local neutral") beliefs. Their editing method is SLAG, a training objective for hypernetwork-style learned optimizers such as KnowledgeEditor. It uses harder training data, many small edits instead of one large edit, and explicit training on sequences of updates. The authors also propose belief graphs, which link two beliefs when editing one flips the other. They find that the models' beliefs are only weakly consistent. Plain off-the-shelf optimizers are strong when fixing a single fact, but SLAG wins when many facts are edited in a row. Belief graphs reveal largely arbitrary dependencies and a few highly influential "core" beliefs.

Key points

  • On the authors' measurements, the ~100M-parameter models stay below 70% paraphrase consistency and show mixed consistency under entailment. Entailment tests use LeapOfThought, and local neutral pairs come from Wikidata5m.
  • KnowledgeEditor nearly breaks down when several beliefs are updated one after another. Training with SLAG on sequential updates recovers much of that loss.
  • On Wikidata5m, Retain Rate for all data is 61.51±1.33, while retention of local neutral beliefs is 15.66 points lower. On FEVER, wrong beliefs flip about 4% of the time when other facts are edited, compared with 2.5% for correct ones.
  • The FEVER belief graph has 10,444 nodes, 1.88m edges and 66.64% update-transitivity. The LeapOfThought graph has 8,642 nodes, 9.71m edges and 24.38% update-transitivity, computed on 4,000 points. In both graphs, no node is edgeless.
  • Code is released at github.com/peterbhase/SLAG-Belief-Updating.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of North Carolina at Chapel HillChapel Hill · United States67 works · 16,474 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Peter Hase, Mona Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal and Srinivasan Iyer. 2023. Methods for Measuring, Updating, and Visualizing Factual Beliefs in Language Models. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.eacl-main.199

@inproceedings{hase2023methods,
  title = {{Methods for Measuring, Updating, and Visualizing Factual Beliefs in Language Models}},
  author = {Peter Hase, Mona Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal and Srinivasan Iyer},
  booktitle = {Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics},
  year = {2023},
  doi = {10.18653/v1/2023.eacl-main.199},
  url = {https://doi.org/10.18653/v1/2023.eacl-main.199}
}

References

29 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 610–623. find
  2. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — Language Models are Few-Shot Learners
  3. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, and Furu Wei. 2021. Knowledge neurons in pretrained transformers. arXiv preprint arXiv:2104.08696. find
  4. Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021. Editing factual knowledge in language models. In EMNLP, pages 6491–6506. Association for Computational Linguistics. find
  5. Bhuwan Dhingra, Jeremy R Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein, and William W Cohen. 2021. Time-aware language models as temporal knowledge bases. arXiv preprint arXiv:2106.15110.
  6. Bradley Efron and Robert J Tibshirani. 1994. An Introduction to the Bootstrap. CRC press. find
  7. Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. 2021. Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics, 9:1012–1031. find
  8. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. 2020. Realtoxicityprompts: Evaluating neural toxic degeneration in language models. In Findings of EMNLP.
  9. Anastasia Giannakidou and Alda Mari. 2020. A linguistic framework for knowledge, belief, and veridicality judgement. HAL. find
  10. Benjamin Heinzerling and Kentaro Inui. 2021. Language models as knowledge bases: On entity representations, storage capacity, and paraphrased queries. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1772–1791, Online.… find
  11. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual knowledge in gpt. arXiv preprint arXiv:2202.05262. find
  12. Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. 2021. Fast model editing at scale. arXiv preprint arXiv:2110.11309. find
  13. Albert Newen and Tobias Starzak. 2020. How to ascribe beliefs to animals. Mind & Language. find
  14. J. Pearl. 2009. Causality. Cambridge University Press. find
  15. Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. 2021. KILT: a benchmark for knowledge intensive language tasks. In Proceedings of th…
  16. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural…
  17. Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the parameters of a language model? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5418–5426, Online. Association for Computational Linguistics. find
  18. Nora Kassner, Oyvind Tafjord, Hinrich Schütze, and Peter Clark. 2021. Beliefbank: Adding memory to a pre-trained language model for a systematic notion of belief. arXiv preprint arXiv:2109.14723. find
  19. Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020. A primer in BERTology: What we know about how BERT works. Transactions of the Association for Computational Linguistics, 8:842–866.
  20. Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d’Autume, Sebastian Ruder, Dani Yogatama, et al. 2021. Mind the gap: Assessing temporal generalization in neural language models. In NeurIPS. find
  21. Eric Schwitzgebel. 2019. Belief. In Edward N. Zalta, editor, The Stanford Encyclopedia of Philosophy, Fall 2019 edition. Metaphysics Research Lab, Stanford University. find
  22. Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017. Zero-shot relation extraction via reading comprehension. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 333–342, Vancouver, Canada. Association for Computational Linguistics.
  23. Stephanie Lin, Jacob Hilton, and Owain Evans. 2021. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958.
  24. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In ICLR. find
  25. Anton Sinitsin, Vsevolod Plokhotnyuk, Dmitriy Pyrkin, Sergei Popov, and Artem Babenko. 2020. Editable neural networks. In ICLR. find
  26. Alon Talmor, Oyvind Tafjord, Peter Clark, Yoav Goldberg, and Jonathan Berant. 2020. Leap-of-thought: Teaching pre-trained models to systematically reason over implicit knowledge. In NeurIPS. find
  27. James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vo…
  28. Symbol Glossary f θ Language Model g ϕ Learned optimizer x i Main Input ŷ i LM output on x i y i ∗ Desired output ∇ θ L(x i , y i ∗ ) Gradient of LM Update(x i , ŷ i , y i ∗ , θ) Update one LM belief L(ϕ; x i , ŷ i , y i ∗ , θ) Belief update objective for x i L Sequential (ϕ; D, θ t ) Sequential… find
  29. Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix Yu, and Sanjiv Kumar. 2020. Modifying memories in transformer models. arXiv preprint arXiv:2012.00363. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hase-2023-methods-measuring-updating-visualizing-factual/