Elamur Research⌕
☰
home / papers / 2021 / BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief

BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief

Paper · 2021 · Nora Kassner, Oyvind Tafjord, Hinrich Schütze et al.

Summary

Pretrained language models (PTLMs) store a lot of world knowledge, but their answers often contradict each other. This makes it unclear what such a model actually "believes." The authors keep the model frozen and place it inside a larger system with a persistent symbolic memory called the BeliefBank. The BeliefBank stores the model's raw true/false answers as weighted beliefs.

Two mechanisms then refine this store:

  • Constraint reasoning: a weighted MaxSAT solver uses weighted logical constraints between facts to overturn beliefs that conflict strongly with the rest.
  • Feedback: relevant stored beliefs are supplied as context when the model answers new questions.

The authors test this in a controlled setting where both candidate facts (e.g., about animals and their properties) and constraints are given, and they release a dataset built for this purpose. Both mechanisms make the system's beliefs more consistent and more accurate as more questions arrive. Earlier fine-tuning methods mostly improved consistency without improving accuracy. The work matters because it lets a system gain a coherent, inspectable picture of the world and keep improving without any retraining or training data.

Key points

  • A belief is a triple of sentence, True/False label and strength in [0, 1]. A constraint is a weighted implication between labelled sentences, and mutual exclusivity is expressed as a pair of rules.
  • Consistency is measured as the complement of the conditional constraint violation (τ) metric from Li et al. (2019).
  • In a small human-in-the-loop test, a person fixed about 6 wrong beliefs after the first 10% of questions. This raised final F1 and Consistency by roughly 2 points (absolute) each, beyond the effect of the corrections themselves.
  • The dataset is available at allenai.org/data/beliefbank.
  • Proposed next steps for real QA deployment:
    • use retrieval over the BeliefBank to select feedback;
    • use textual entailment, such as CA-MTL, to match beliefs to constraints;
    • query the model itself for constraints instead of writing them by hand.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Allen Institute for Artificial IntelligenceSeattle · United States317 works · 106,633 citations🎓Ludwig-Maximilians-Universität MünchenMunich · Germany116 works · 57,070 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 4,145 works

Cite

Cite as Nora Kassner, Oyvind Tafjord, Hinrich Schütze and Peter Clark. 2021. BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief. arXiv:2109.14723.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2109.14723 · arXiv 2109.14723

@misc{kassner2021beliefbank,
  title = {{BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief}},
  author = {Nora Kassner, Oyvind Tafjord, Hinrich Schütze and Peter Clark},
  year = {2021},
  doi = {10.48550/arxiv.2109.14723},
  eprint = {2109.14723},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2109.14723}
}

References

44 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Akari Asai and Hannaneh Hajishirzi. 2020. Logic-guided data augmentation and regularization for consistent question answering. ArXiv, abs/2004.10157.
  2. Matthias Broecheler, Lilyana Mihalkova, and L. Getoor. 2010. Probabilistic similarity logic. In UAI. find
  3. Andrew Carlson, J. Betteridge, Bryan Kisiel, Burr Settles, Estevam R. Hruschka, and Tom Michael Mitchell. 2010. Toward an architecture for never-ending language learning. In AAAI. find
  4. P. Clark, Oyvind Tafjord, and Kyle Richardson. 2020. Transformers as soft reasoners over language. In IJ-CAI.
  5. Joe Davison, Joshua Feldman, and Alexander Rush. 2019. Commonsense knowledge mining from pre-trained models. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1173–…
  6. Johan De Kleer. 1986. An assumption-based tms. Artificial intelligence, 28(2):127–162. find
  7. Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An efficient SMT solver. In International conference on Tools and Algorithms for the Construction and Analysis of Systems, pages 337–340. Springer. find
  8. Rina Dechter and Avi Dechter. 1988. Belief maintenance in dynamic constraint networks. University of California, Computer Science Department. find
  9. Sayna Ebrahimi, Suzanne Petryk, Akash Gokul, William Gan, J. Gonzalez, Marcus Rohrbach, and Trevor Darrell. 2021. Remembering for the right reasons: Explanations reduce catastrophic forgetting. ICLR. find
  10. Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, E. Hovy, Hinrich Schütze, and Yoav Goldberg. 2021. Measuring and improving consistency in pretrained language models. ArXiv, abs/2102.01017. find
  11. Allyson Ettinger. 2020. What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models. Transactions of the Association for Computational Linguistics, 8:34–48.
  12. Christiane Fellbaum. 2005. Wordnet and wordnets. In Keith et al. Brown, editor, Encyclopedia of Language and Linguistics, Second Edition. Elsevier, Oxford. find
  13. M. Genesereth and N. Nilsson. 1987. Logical foundations of artificial intelligence. Kaufmann. find
  14. Konstantin Genin and Franz Huber. 2021. Formal Representations of Belief. In Edward N. Zalta, editor, The Stanford Encyclopedia of Philosophy, spring 2021 edition. Metaphysics Research Lab, Stanford University. find
  15. Dedre Gentner and Albert L. Stevens. 1983. Mental Models. Lawrence Erlbaum Associates. find
  16. A. Graves, Greg Wayne, M. Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwinska, Sergio Gomez Colmenarejo, Edward Grefenstette, Tiago Ramalho, J. Agapiou, Adrià Puigdomènech Badia, K. Hermann, Yori Zwols, Georg Ostrovski, Adam Cain, Helen King, C. Summerfield, P. Blunsom, K. Kavukcuoglu,… find
  17. Kelvin Guu, Kenton Lee, Z. Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. Realm: Retrieval-augmented language model pre-training. ArXiv, abs/2002.08909.
  18. Mikael Henaff, J. Weston, Arthur D. Szlam, Antoine Bordes, and Y. LeCun. 2016. Tracking the world state with recurrent entity networks. In ICLR.
  19. D. Hilton. 1996. Mental models and causal explanation: Judgements of probable cause and explanatory relevance. Thinking & Reasoning, 2:273–308. find
  20. P. Johnson-Laird. 1983. Mental Models : Towards a Cognitive Science of Language. Harvard University Press. find
  21. D. Kahneman. 2011. Thinking, fast and slow. Farrar, Straus, and Giroux. find
  22. Nora Kassner and H. Schütze. 2020. Negated and misprimed probes for pretrained language models: Birds can talk, but cannot fly. In ACL.
  23. Patrick Lewis, Ethan Perez, Aleksandara Piktus, Fabio Petroni, V. Karpukhin, Naman Goyal, Heinrich Kuttler, M. Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In NeurIPS.
  24. Tao Li, Vivek Gupta, Maitrey Mehta, and Vivek Srikumar. 2019. A logic-driven framework for consistency of neural models. In EMNLP.
  25. Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, L. Carin, and Weizhu Chen. 2021. What makes good in-context examples for GPT-3? ArXiv, abs/2101.06804.
  26. John W. McCarthy. 1959. Programs with common sense. In Proc. Tedding Conf. on the Mechanization of Thought Processes, pages 75–91. find
  27. Pasquale Minervini and S. Riedel. 2018. Adversarially regularising neural NLI models to integrate logical background knowledge. In CoNLL. find
  28. R. Moore. 1983. Semantical considerations on non-monotonic logic. In IJCAI. find
  29. G. I. Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and S. Wermter. 2019. Continual lifelong learning with neural networks: A review. Neural networks : the official journal of the International Neural Network Society, 113:54–71.
  30. J. Pearl. 1986. Fusion, propagation, and structuring in belief networks. Artif. Intell., 29:241–288. find
  31. F. Petroni, Tim Rocktäschel, Patrick Lewis, A. Bakhtin, Yuxiang Wu, Alexander H. Miller, and S. Riedel. 2019. Language models as knowledge bases? In EMNLP.
  32. Jonathan Pilault, Amine Elhattami, and C. Pal. 2021. Conditionally adaptive multi-task learning: Improving transfer learning in nlp using fewer parameters & less data. In ICLR. find
  33. J. Pujara, H. Miao, L. Getoor, and William W. Cohen. 2013. Knowledge graph identification. In International Semantic Web Conference. find
  34. Abhilasha Ravichander, Eduard Hovy, Kaheer Suleman, Adam Trischler, and Jackie Chi Kit Cheung. 2020. On the systematicity of probing contextualized word representations: The case of hypernymy in BERT. In Proceedings of the Ninth Joint Conference on Lexical and Computational Semantics, pages 88–102,…
  35. Marco Tulio Ribeiro, Carlos Guestrin, and Sameer Singh. 2019. Are red roses red? Evaluating consistency of question-answering models. In ACL. find
  36. Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the parameters of a language model? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5418–5426, Online. Association for Computational Linguistics. find
  37. Vered Shwartz, Peter West, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020. Unsupervised commonsense question answering with self-talk. In EMNLP. find
  38. Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. In AAAI. find
  39. Sanjay Subramanian, Ben Bogin, Nitish Gupta, Tomer Wolfson, Sameer Singh, Jonathan Berant, and Matt Gardner. 2020. Obtaining faithful interpretations from compositional neural networks. In ACL. find
  40. Sainbayar Sukhbaatar, Arthur D. Szlam, J. Weston, and R. Fergus. 2015. End-to-end memory networks. In NeurIPS. find
  41. Oyvind Tafjord and Peter Clark. 2021. General-purpose question-answering with Macaw. arXiv, abs/2109.02593.
  42. Alon Talmor, Oyvind Tafjord, Peter Clark, Yoav Goldberg, and Jonathan Berant. 2020. Leap-of-thought: Teaching pre-trained models to systematically reason over implicit knowledge. In NeurIPS.
  43. Xiaojin Zhu. 2015. Machine teaching: An inverse problem to machine learning and an approach toward optimal education. In AAAI. find
  44. Appendix: BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kassner-2021-beliefbank-adding-memory-pre/