Elamur Research⌕
☰
home / papers / 2020 / Are Pretrained Language Models Symbolic Reasoners over Knowledge?

Are Pretrained Language Models Symbolic Reasoners over Knowledge?

Summary

Probing studies show that pretrained language models hold many facts, but they cannot reveal how that knowledge was acquired. Kassner, Krojer and Schütze ask whether BERT gains facts by inferring them through symbolic rules or by memorizing them. They pretrain BERT from scratch on synthetic corpora of subject–relation–object triples built from artificial tokens. This gives them complete control over which facts the model sees. For reasoning, each corpus embeds one of six rules. Some conclusions are withheld and later queried as masked-object cloze questions, with no premises supplied at test time. For memorization, they use a smaller BERT that cannot store every training fact and vary how often facts occur and whether they fit an entity-group pattern. BERT acquires equivalence and implication. Its strong scores on symmetry, inversion and negation turn out to mask flawed rule use. Composition (two-hop) is hard unless richer context is provided. Frequent and schema-conformant facts are memorized more reliably than rare facts or exceptions. The study separates inference from recall in a way earlier probing could not.

Key points

  • Six rules tested: equivalence, symmetry, inversion, negation, implication and composition.
    • Each reasoning corpus draws on a vocabulary of 5000 entities, 1000 attributes and 500 relations.
    • Rule-following relations make up 50% of the set, while the rest produce random facts.
    • Training uses all premises plus 90% of the conclusions. The remaining 10% form the test set, scored by precision at m.
  • Reasoning results:
    • Equivalence and implication are learned.
    • Symmetry, inversion and negation are not truly learned, despite high test precision.
    • Two-hop composition fails without additional semantic context.
    • Off-the-shelf BERT-large shows only partial rule knowledge. For example, it treats "shares borders with" as symmetric in some cases.
  • Memorization frequency experiment:
    • The vocabulary has 125,000 entities, 20 relations and 2250 attributes.
    • 800,000 facts are split into groups of 8000, each repeated between 1 and 100 times.
    • Rare facts are not retained.
  • Schema experiment: entity groups of 250 members share sets of 10 attributes. Facts that fit the group pattern are easier to memorize than exceptions.
  • Code release: code and data generators are available at github.com/BennoKrojer/reasoning-over-facts.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

44 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 42 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.conll-1.45

@inproceedings{kassner2020are,
  title = {{Are Pretrained Language Models Symbolic Reasoners over Knowledge?}},
  author = {Kassner, Nora and Krojer, Benno and Schütze, Hinrich},
  booktitle = {Proceedings of the 24th Conference on Computational Natural Language Learning},
  year = {2020},
  doi = {10.18653/v1/2020.conll-1.45},
  url = {https://doi.org/10.18653/v1/2020.conll-1.45}
}

References

32 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Devansh Arpit, Stanisław Jastrzundefinedbski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S. Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, and Simon Lacoste-Julien. 2017. A closer look at memorization in deep networks. In Proceedings of the 34th International Conf… find
  2. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… find
  3. Peter Clark, Oyvind Tafjord, and Kyle Richardson. 2020. Transformers as soft reasoners over language. IJCAI. find our card
  4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  5. Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019. Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs. In NAACL-HLT. find our card
  6. Allyson Ettinger. 2020. What bert is not: Lessons from a new suite of psycholinguistic diagnostics for language models. Transactions of the Association for Computational Linguistics, 8:34–48. find our card
  7. Emily Goodwin, Koustuv Sinha, and Timothy J. O’Donnell. 2020. Probing linguistic systematicity. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1958–1969, Online. Association for Computational Linguistics. find
  8. Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018. Annotation artifacts in natural language inference data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techno… find
  9. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. REALM: Retrieval-augmented language model pre-training. ArXiv, abs/2002.08909. find
  10. Dieuwke Hupkes, Verna Dankers, Mathijs Mul, and Elia Bruni. 2020. Compositionality decomposed: How do neural networks generalise? J. Artif. Intell. Res., 67:757–795. find
  11. Nora Kassner and Hinrich Schütze. 2020. Negated and misprimed probes for pretrained language models: Birds can talk, but cannot fly. ACL. find our card
  12. Divyansh Kaushik and Zachary C. Lipton. 2018. How much reading does reading comprehension require? a critical investigation of popular benchmarks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 5010–5015, Brussels, Belgium. Association for Computatio… find our card
  13. Beth Levin. 1993. English Verb Classes and Alternations. The University of Chicago Press, Chicago. find
  14. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692. find
  15. Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3428–3448, Florence, Italy. Association for Computational Ling… find our card
  16. Mojtaba Nayyeri, Chengjin Xu, Jens Lehmann, and Hamed Shariat Yazdi. 2019. Logicenn: A neural based knowledge graphs embedding model with logical rules. ArXiv, abs/1908.07141. — [arXiv:1908.07141](https://arxiv.org/abs/1908.07141) find
  17. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural… find our card
  18. Guy Politzer. 2007. Reasoning with conditionals. Topoi, 26(1):79–95. find
  19. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. find
  20. Kyle Richardson, Hai Hu, Lawrence S. Moss, and Ashish Sabharwal. 2020. Probing natural language inference models through semantic fragments. In Thirty-Fourth AAAI Conference on Artificial Intelligence. find our card
  21. Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the parameters of a language model? ArXiv, abs/2002.08910. find
  22. Koustuv Sinha, Shagun Sodhani, Jin Dong, Joelle Pineau, and William L. Hamilton. 2019. CLUTRR: A diagnostic benchmark for inductive reasoning from text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan… find
  23. Steven A. Sloman. 1996. The empirical case for two systems of reasoning. Psychological Bulletin, 119(1):3–22. find
  24. Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. Rotate: Knowledge graph embedding by relational rotation in complex space. In International Conference on Learning Representations. find
  25. Alon Talmor, Yanai Elazar, Yoav Goldberg, and Jonathan Berant. 2019. oLMpics - on what language model pre-training captures. ArXiv, abs/1912.13283. find our card
  26. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Proces… find
  27. Jason Weston, Antoine Bordes, Sumit Chopra, and Tomas Mikolov. 2016. Towards ai-complete question answering: A set of prerequisite toy tasks. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings. find our card
  28. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface’s transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771. find our card
  29. Hitomi Yanaka, Koji Mineshima, Daisuke Bekki, and Kentaro Inui. 2020. Do neural models learn systematicity of monotonicity inference in natural language? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6105–6117, Online. Association for Computational… find
  30. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369… find our card
  31. Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. 2017. Understanding deep learning requires rethinking generalization. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24 -26, 2017, Conference Track Proceedings. find
  32. Zhanqiu Zhang, Jianyu Cai, Yongdong Zhang, and Jie Wang. 2020. Learning hierarchy-aware knowledge graph embeddings for link prediction. In Thirty-Fourth AAAI Conference on Artificial Intelligence. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kassner-2020-are-pretrained-language-models-symbolic/