Elamur Research
☰
home / papers / 2020 / How Context Affects Language Models' Factual Predictions

How Context Affects Language Models' Factual Predictions

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/2005.04611

Summary

Language models such as BERT hold some factual knowledge in their weights, but a fixed number of parameters cannot reliably capture everything in sources like Wikipedia. This paper asks whether adding text passages to a cloze-style query can improve factual predictions without any supervised training. The authors use the LAMA probe (Google-RE, T-REx and SQuAD subsets) and test BERT-large and RoBERTa-large with four kinds of context: oracle Wikipedia snippets, paragraphs retrieved by DrQA's TF-IDF retriever, passages written by an autoregressive LM, and adversarial, unrelated text. Query and context are kept apart by segment embeddings and a [SEP] token for BERT, or an eos token for RoBERTa. Retrieved context greatly raises BERT's precision, and the resulting system matches or beats the supervised DrQA reader. With proper segment separation, BERT's Next Sentence Prediction head appears to detect irrelevant passages, so noisy context does little harm. This suggests that off-the-shelf retrieval combined with a pre-trained LM can serve as a capable unsupervised QA system.

Key points

  • Data: LAMA probe with Google-RE (3 relations, 5527 facts), T-REx (41 relations, 34039 facts) and SQuAD (305 cloze-converted questions). The RoBERTa experiments use a version trimmed by about 15% so that every answer is in its vocabulary.
  • Weighted-average P@1 for BERT-large rises from 30.5 without context to 42.8 with retrieved paragraphs, compared with 27.2 for the supervised DrQA baseline. Oracle snippets push it to 63.6, an upper bound.
  • Generated contexts come from a 1.4B-parameter autoregressive LM trained on CC-NEWS, using delayed beam search. They help much less than retrieved text.
  • The authors attribute BERT's robustness to adversarial contexts to NSP. RoBERTa, trained with MLM only, lacks this mechanism.
  • The approach is limited by LAMA's single-token answers. Producing multi-token spans from bidirectional LMs remains difficult, whereas extractive readers handle such spans naturally.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

129 papers of our library cite it (of the 140673 whose bibliographies we read); 12 of them have a page here, 117 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2005.04611 · arXiv 2005.04611

@misc{petroni2020how,
  title = {{How Context Affects Language Models' Factual Predictions}},
  author = {Petroni, Fabio and Lewis, Patrick and Piktus, Aleksandra and Rocktäschel, Tim and Wu, Yuxiang and Miller, Alexander H. and Riedel, Sebastian},
  year = {2020},
  doi = {10.48550/arxiv.2005.04611},
  eprint = {2005.04611},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2005.04611}
}

References

52 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. PIQA: reasoning about physical commonsense in natural language. CoRR, abs/1911.11641, 2019. find our card
  2. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. Reading wikipedia to answer open-domain questions. ACL, 2017. find our card
  3. Eunsol Choi, Daniel Hewlett, Jakob Uszkoreit, Illia Polosukhin, Alexandre Lacoste, and Jonathan Berant. Coarse-to-fine question answering for long documents. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 209–220, Vancouver,… find
  4. Christopher Clark and Matt Gardner. Simple and effective multi-paragraph reading comprehension. ACL, 2018. find our card
  5. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the AI2 reasoning challenge. CoRR, abs/1803.05457, 2018. URL http://arxiv.org/abs/1803.05457. ↗
  6. Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, and Andrew McCallum. Multi-step retriever-reader interaction for scalable open-domain question answering. In ICLR (Poster). OpenReview.net, 2019. find
  7. Nicola De Cao, Michael Schlichtkrull, Wilker Aziz, and Ivan Titov. How do decisions emerge across layers in neural models? interpretation with differentiable masking. arXiv preprint arXiv:2004.14992, 2020. find
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. NAACL-HLT, 2019. find
  9. Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Elena Simperl, and Frederique Laforest. T-rex: A large scale alignment of natural language with knowledge base triples. LREC, 2018. find
  10. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. Realm: Retrieval-augmented language model pre-training. arXiv:2002.08909, 2020. find
  11. John Hewitt and Christopher D. Manning. A structural probe for finding syntax in word representations. In NAACL-HLT (1), pages 4129–4138. Association for Computational Linguistics, 2019. find our card
  12. Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. How can we know what language models know? CoRR, abs/1911.12543, 2019. find our card
  13. Nora Kassner and Hinrich Schütze. arXiv:1911.03343, 2019. find our card
  14. Najoung Kim, Roma Patel, Adam Poliak, Patrick Xia, Alex Wang, Tom McCoy, Ian Tenney, Alexis Ross, Tal Linzen, Benjamin Van Durme, Samuel R. Bowman, and Ellie Pavlick. Probing what different NLP tasks teach machines about function word comprehension. In *SEM@NAACL-HLT, pages 235–249. Association for… find our card
  15. Zhen-Zhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. Albert: A lite bert for self-supervised learning of language representations. ArXiv, abs/1909.11942, 2019. find
  16. Jinhyuk Lee, Seongjun Yun, Hyunjae Kim, Miyoung Ko, and Jaewoo Kang. Ranking paragraphs for improving answer recall in open-domain question answering. In EMNLP, pages 565–569. Association for Computational Linguistics, 2018. find
  17. Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. Latent retrieval for weakly supervised open domain question answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6086–6096, Florence, Italy, July 2019. Association for Computational Linguistic… find our card
  18. Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. Zero-shot relation extraction via reading comprehension. CoNLL, 2017. find
  19. Negated lama: Birds cannot fly. arXiv preprint Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv prepr… find our card
  20. Patrick Lewis, Ludovic Denoyer, and Sebastian Riedel. Unsupervised question answering by cloze translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4896–4910, Florence, Italy, July 2019b. Association for Computational Linguistics. doi: 10.186… find
  21. Yankai Lin, Haozhe Ji, Zhiyuan Liu, and Maosong Sun. Denoising distantly supervised open-domain question answering. In ACL (1), pages 1736–1745. Association for Computational Linguistics, 2018. find
  22. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019. find
  23. Rebecca Marvin and Tal Linzen. Targeted syntactic evaluation of language models. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pages 1192–1202, 2018. URL https://aclanthology.info/papers/D18-1151/d18-1151. ↗
  24. Luca Massarelli, Fabio Petroni, Aleksandra Piktus, Myle Ott, Tim Rocktäschel, Vassilis Plachouras, Fabrizio Silvestri, and Sebastian Riedel. How decoding strategies affect the verifiability of generated text. arXiv preprint arXiv:1911.03587, 2019. find
  25. Sewon Min, Victor Zhong, Richard Socher, and Caiming Xiong. Efficient and robust question answering from minimal context over documents. In ACL (1), pages 1725–1735. Association for Computational Linguistics, 2018. find
  26. Liang Pang, Yanyan Lan, Jiafeng Guo, Jun Xu, Lixin Su, and Xueqi Cheng. HAS-QA: hierarchical answer spans model for open-domain question answering. In AAAI, pages 6875–6882. AAAI Press, 2019. find
  27. Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. Language models as knowledge bases? EMNLP, 2019. find our card
  28. Nina Poerner, Ulli Waltinger, and Hinrich Schütze. Bert is not a knowledge base (yet): Factual knowledge vs. name-based reasoning in unsupervised qa. arXiv preprint arXiv:1911.03681, 2019. find
  29. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAI Blog, 2019. find
  30. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. CoRR, abs/1910.10683, 2019. find
  31. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. EMNLP, 2016. find our card
  32. Siva Reddy, Danqi Chen, and Christopher D. Manning. CoQA: A conversational question answering challenge. Transactions of the Association for Computational Linguistics, 7:249–266, March 2019. doi: 10.1162/tacl a 00266. URL https://www.aclweb.org/anthology/Q19-1016. ↗ our card
  33. Kyle Richardson and Ashish Sabharwal. What does my QA model know? devising controlled probes using expert knowledge. CoRR, abs/1912.13337, 2019. find
  34. Kyle Richardson, Hai Hu, Lawrence S. Moss, and Ashish Sabharwal. Probing natural language inference models through semantic fragments. CoRR, abs/1909.07521, 2019. find our card
  35. Min Joon Seo, Jinhyuk Lee, Tom Kwiatkowski, Ankur P. Parikh, Ali Farhadi, and Hannaneh Hajishirzi. Real-time open-domain question answering with dense-sparse phrase index. In ACL (1), pages 4430–4441. Association for Computational Linguistics, 2019. find
  36. Alon Talmor, Yanai Elazar, Yoav Goldberg, and Jonathan Berant. olmpics - on what language model pre-training captures. CoRR, abs/1912.13283, 2019. find our card
  37. Ian Tenney, Dipanjan Das, and Ellie Pavlick. BERT rediscovers the classical NLP pipeline. In ACL (1), pages 4593–4601. Association for Computational Linguistics, 2019a. find our card
  38. Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R. Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R. Bowman, Dipanjan Das, and Ellie Pavlick. What do you learn from context? probing for sentence structure in contextualized word representations. In ICLR (Poster). OpenReview.net,… find our card
  39. Jesse Vig and Yonatan Belinkov. Analyzing the structure of attention in a transformer language model. Second BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, 2019. find our card
  40. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353–355, Bruss… find our card
  41. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. Superglue: A stickier benchmark for general-purpose language understanding systems. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d’ Alché-Buc, E. Fox, and R. Garnett, editors… find our card
  42. Shuohang Wang, Mo Yu, Xiaoxiao Guo, Zhiguo Wang, Tim Klinger, Wei Zhang, Shiyu Chang, Gerry Tesauro, Bowen Zhou, and Jing Jiang. R3 : Reinforced ranker-reader for open-domain question answering. In AAAI, pages 5981–5988. AAAI Press, 2018b. find
  43. Shuohang Wang, Mo Yu, Jing Jiang, Wei Zhang, Xiaoxiao Guo, Shiyu Chang, Zhiguo Wang, Tim Klinger, Gerald Tesauro, and Murray Campbell. Evidence aggregation for answer reranking in open-domain question answering. In ICLR (Poster). OpenReview.net, 2018c. find
  44. Alex Warstadt and Samuel R Bowman. Linguistic analysis of pretrained sentence encoders with acceptability judgments. arXiv preprint arXiv:1901.03438, 2019. — [arXiv:1901.03438](https://arxiv.org/abs/1901.03438) find
  45. Alex Warstadt, Yu Cao, Ioana Grosu, Wei Peng, Hagen Blix, Yining Nie, Anna Alsop, Shikha Bordia, Haokun Liu, Alicia Parrish, Sheng-Fu Wang, Jason Phang, Anhad Mohananey, Phu Mon Htut, Paloma Jeretic, and Samuel R. Bowman. Investigating bert’s knowledge of language: Five analysis methods with npis. I… find
  46. Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, and Luke Zettlemoyer. Zero-shot entity linking with dense entity retrieval. arXiv preprint arXiv:1911.03814, 2019. find our card
  47. Wenhan Xiong, Mo Yu, Xiaoxiao Guo, Hong Wang, Shiyu Chang, Murray Campbell, and William Yang Wang. Simple yet effective bridge reasoning for open-domain multi-hop question answering. Second Workshop on Machine Reading for Question Answering, 2019. find
  48. Vikas Yadav, Steven Bethard, and Mihai Surdeanu. Alignment over heterogeneous embeddings for question answering. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages… find
  49. Wei Yang, Yuqing Xie, Aileen Lin, Xingyu Li, Luchen Tan, Kun Xiong, Ming Li, and Jimmy Lin. End-to-end open-domain question answering with BERTserini. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), pages 72–77, M… find our card
  50. Wei Yang, Haotian Zhang, and Jimmy Lin. Simple applications of BERT for ad hoc document retrieval. CoRR, abs/1903.10972, 2019b. — [arXiv:1903.10972](https://arxiv.org/abs/1903.10972) find
  51. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems, pages 5754–5764, 2019c. find our card
  52. Zeynep Akkalyoncu Yilmaz, Shengjin Wang, Wei Yang, Haotian Zhang, and Jimmy Lin. Applying BERT to document retrieval with birch. In EMNLP/IJCNLP (3), pages 19–24. Association for Computational Linguistics, 2019. find