Elamur Research
☰
home / papers / 2020 / Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly

Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2020.acl-main.698.pdf

Summary

Kassner and Schütze test how robust the factual knowledge in pretrained language models (PLMs) really is. They build on the LAMA probe of Petroni et al. (2019) and add two new probing datasets. Negated LAMA puts a negation word such as "not" into every cloze template or question by hand. Misprimed LAMA adds a misleading word and a question mark in front of a query, e.g. "Talk? Birds can [MASK]". They run both on Transformer-XL, two ELMo variants and two BERT sizes. The models give nearly the same answers to a statement and its negation. BERT-large is also easily led to the wrong answer by one distracting word that a person would simply ignore. A small BERT trained from scratch on a synthetic corpus memorizes negated facts it has seen but does not generalize to new ones. Once finetuned with true/false labels, it learns negation perfectly. The authors conclude that PLMs answer these queries mostly by surface pattern matching rather than by recalling and reasoning over facts. They argue that models need better handling of negation, better confidence estimates and more robustness to distractors.

Key points

  • Resources: negated LAMA (about 42,867 negated sentences) and four misprimed LAMA versions (A–D), built from Google-RE, T-REx, ConceptNet and SQuAD. Data and code are on GitHub (norakassner/LAMA_primed_negated).
  • Negation: in most settings, the Spearman correlation between predictions for positive and negated queries is above 85%, and the top-1 answers often overlap. BERT copes slightly better than Transformer-XL and ELMo.
  • Mispriming (BERT-large): misprimes chosen at random do relatively little harm. Misprimes taken from highly ranked wrong fillers (version C) cut precision by more than 60% in most subsets. Adding 20 neutral sentences between the misprime and the query (version D) still leaves large drops.
  • Synthetic corpus: BERT-base was trained on balanced positive and negative sentences about 200 subjects and 20 adjectives (10 antonym pairs). It memorizes the training facts but generalizes poorly to unseen ones. Supervised finetuning to classify sentences as true/false reaches 100% test accuracy.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

208 papers of our library cite it (of the 140673 whose bibliographies we read); 16 of them have a page here, 192 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.acl-main.698 · arXiv 1911.03343

@inproceedings{kassner2020negated,
  title = {{Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly}},
  author = {Kassner, Nora and Schütze, Hinrich},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.698},
  eprint = {1911.03343},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2020.acl-main.698}
}

References

28 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1195–1205, New Orleans, Louisiana. Association for Computational Linguistics. find
  2. Chris Alberti, Kenton Lee, and Michael Collins. 2019. A BERT baseline for the natural questions. ArXiv, abs/1901.08634. — [arXiv:1901.08634](https://arxiv.org/abs/1901.08634) find
  3. Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019. COMET: Commonsense transformers for automatic knowledge graph construction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4762–4779, Florence… find our card
  4. Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc Le, and Ruslan Salakhutdinov. 2019. Transformer-XL: Attentive language models beyond a fixed-length context. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2978–2988, Florence, Italy. Assoc… find
  5. Ishita Dasgupta, Demi Guo, Andreas Stuhlmüller, Samuel J Gershman, and Noah D Goodman. 2018. Evaluating compositionality in sentence embeddings. arXiv preprint arXiv:1802.04302. — [arXiv:1802.04302](https://arxiv.org/abs/1802.04302) find
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  7. Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Laforest, and Elena Simperl. 2018. T-REx: A large scale alignment of natural language with knowledge base triples. In Proceedings of the Eleventh International Conference on Language Resources and Evaluati… find
  8. Allyson Ettinger. 2019. What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models. Transactions of the Association for Computational Linguistics, 8:34–48. find our card
  9. Mario Giulianelli, Jack Harding, Florian Mohnert, Dieuwke Hupkes, and Willem Zuidema. 2018. Under the hood: Using diagnostic classifiers to investigate and improve how language models track agreement information. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neura… find
  10. Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni. 2018. Colorless green recurrent networks dream hierarchically. In Proceedings of the 2018 Conference of the North Jaap Jumelet and Dieuwke Hupkes. 2018. Do language models understand anything? on the ability of LSTMs… find our card
  11. Katharina Kann, Alex Warstadt, Adina Williams, and Samuel R. Bowman. 2019. Verb argument structure alternations in word and sentence embeddings. In Proceedings of the Society for Computation in Linguistics (SCiL) 2019, pages 287–297. find
  12. Najoung Kim, Roma Patel, Adam Poliak, Patrick Xia, Alex Wang, Tom McCoy, Ian Tenney, Alexis Ross, Tal Linzen, Benjamin Van Durme, Samuel R. Bowman, and Ellie Pavlick. 2019. Probing what different NLP tasks teach machines about function word comprehension. In Proceedings of the Eighth Joint Conferenc… find our card
  13. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red-field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural… find our card
  14. Xiang Li, Aynaz Taheri, Lifu Tu, and Kevin Gimpel. 2016. Commonsense knowledge base completion. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1445–1455, Berlin, Germany. Association for Computational Linguistics. find
  15. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692. find
  16. Rebecca Marvin and Tal Linzen. 2018. Targeted syntactic evaluation of language models. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1192–1202, Brussels, Belgium. Association for Computational Linguistics. find
  17. Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3428–3448, Florence, Italy. Association for Computational Ling… find our card
  18. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… find
  19. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural… find our card
  20. Nina Poerner, Ulli Waltinger, and Hinrich Schütze. 2019. BERT is not a knowledge base (yet): Factual knowledge vs. name-based reasoning in unsupervised qa. ArXiv, abs/1911.03681. find
  21. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. find
  22. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas. Association for Computational Linguistics. find our card
  23. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Semantically equivalent adversarial rules for debugging NLP models. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 856–865, Melbourne, Australia. Association for C… find
  24. Endel Tulving and Daniel Schacter. 1990. Priming and human memory systems. Science, 247(4940):301– 306. find
  25. Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. Universal adversarial triggers for attacking and analyzing NLP. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Proce… find
  26. Alex Warstadt and Samuel R. Bowman. 2019. Grammatical analysis of pretrained sentence encoders with acceptability judgments. ArXiv, abs/1901.03438. — [arXiv:1901.03438](https://arxiv.org/abs/1901.03438) find
  27. Alex Warstadt, Yu Cao, Ioana Grosu, Wei Peng, Hagen Blix, Yining Nie, Anna Alsop, Shikha Bordia, Haokun Liu, Alicia Parrish, Sheng-Fu Wang, Jason Phang, Anhad Mohananey, Phu Mon Htut, Paloma Jeretic, and Samuel R. Bowman. 2019. Investigating BERT’s knowledge of language: Five analysis methods with N… find
  28. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, R’emi Louf, Morgan Funtow-icz, and Jamie Brew. 2019. Huggingface’s transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771. find our card