Elamur Research⌕
☰
home / papers / 2021 / Probing Across Time: What Does RoBERTa Know and When?

Probing Across Time: What Does RoBERTa Know and When?

Paper · 2021 · Leo Z. Liu, Yizhong Wang, Jungo Kasai et al.

Summary

The paper asks when, during pretraining, a masked language model acquires different kinds of knowledge, rather than only what a finished model knows. The authors reproduce RoBERTa-base pretraining from scratch, save checkpoints across roughly 1M training steps, and evaluate each checkpoint with five public probe packages. Structural probes (linear classifiers on frozen representations) and behavioral probes (zero-training masked-LM or sentence-scoring tests) cover linguistic, factual, commonsense and reasoning knowledge. They also pretrain variants on corpora that differ in domain and size, and fine-tune checkpoints on standard benchmarks. Linguistic knowledge appears early, stays stable and is robust to corpus domain. Factual and commonsense knowledge grow more slowly and depend more on the domain. Reasoning is largely not learned in a stable way. Corpus diversity seems to matter more than raw corpus size. These results can guide more efficient pretraining schedules and help explain how different capabilities emerge. The authors release their code and checkpoints.

Key points

  • Probe suite: LKT (POS tagging, chunking, NER, arc prediction/classification), BLiMP (minimal pairs), LAMA (T-REx, Google RE, ConceptNet, SQuAD), CAT (commonsense) and oLMpics (reasoning).
  • The order in which knowledge types are learned holds across data variations. Corpus domain and size mainly change learning speed and final scores.
  • Most benchmarks, such as SST-2 and SQuAD, reach high fine-tuning performance in under half of the pretraining steps. Tasks that need complex knowledge, such as ReCoRD and WSC, benefit from longer pretraining.
  • Unlike Chiang et al. (2020), the authors find that factual and commonsense knowledge keep improving during pretraining. They attribute this to their broader probe selection, since a T-REx subset alone plateaus early.
  • MLP probes scored highly even on randomly initialized RoBERTa, so the authors use linear probes instead. Checkpoints are available at github.com/leo-liuzy/probe-across-time.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of WashingtonSeattle · United States263 works · 180,581 citations🔬Allen Institute for Artificial IntelligenceSeattle · United States219 works · 89,712 citations

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works

🧭 Topics 2 Language models & pretraining · 1,343 works Interpretability · 58 works

Cite

Cite as Leo Z. Liu, Yizhong Wang, Jungo Kasai, Hannaneh Hajishirzi and Noah A. Smith. 2021. Probing Across Time: What Does RoBERTa Know and When?. Findings of the Association for Computational Linguistics: EMNLP 2021.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.findings-emnlp.71 · arXiv 2104.07885

@inproceedings{liu2021probing,
  title = {{Probing Across Time: What Does RoBERTa Know and When?}},
  author = {Leo Z. Liu, Yizhong Wang, Jungo Kasai, Hannaneh Hajishirzi and Noah A. Smith},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2021},
  year = {2021},
  doi = {10.18653/v1/2021.findings-emnlp.71},
  eprint = {2104.07885},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2021.findings-emnlp.71}
}

References

37 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yonatan Belinkov, Sebastian Gehrmann, and Ellie Pavlick. 2020. Interpretability and analysis in neural NLP. In Proc. of ACL: Tutorial Abstracts. find
  2. Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Wen-tau Yih, and Yejin Choi. 2020. Abductive commonsense reasoning. In Proc. of ICLR. our card
  3. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… our card
  4. Cheng-Han Chiang, Sung-Feng Huang, and Hung-yi Lee. 2020. Pretrained language model embryology: The birth of ALBERT. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6813–6828, Online. Association for Computational Linguistics. find
  5. Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D Manning. 2019. What does BERT look at? an analysis of BERT’s attention. In Proc. of BlackboxNLP. find
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proc. of NAACL. our card
  7. William B. Dolan and Chris Brockett. 2005. Automatically constructing a corpus of sentential paraphrases. In Proc. of IWP. Asian Federation of Natural Language Processing. our card
  8. Aaron Gokaslan and Vanya Cohen. 2019. OpenWeb-Text corpus. http://Skylion007.github.io/ OpenWebTextCorpus . \#\# Acknowledgments Suchin Gururangan, Ana Marasović, Swabha In random order, we appreciate valuable discus-Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, sion and feedback from Noah’s ARK gro… ↗
  9. John Hewitt and Percy Liang. 2019. Designing and interpreting probes with control tasks. In Proc. of EMNLP. Association for Computational Linguistics. find
  10. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2018. Language models are unsupervised multitask learners. find
  11. Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. 2019. Revealing the dark secrets of bert. In Proc. of EMNLP. our card
  12. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, W. Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. JMLR. our card
  13. Loïc Lannelongue, Jason Grealey, and Michael Inouye. 2020. Green algorithms: Quantifying the carbon emissions of computation. CoRR, abs/2007.07610. find
  14. Hector J. Levesque, Ernest Davis, and Leora Morgenstern. 2012. The Winograd schema challenge. In Proc. of KR&R. find
  15. Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, and Noah A. Smith. 2019a. Linguistic knowledge and transferability of contextual representations. In Proc. of NAACL. our card
  16. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. RoBERTa: A Robustly Optimized BERT Pretraining Approach. find
  17. James L. McClelland and David E. Rumelhart. 1986. Parallel Distributed Processing, volume 2. find
  18. Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James F. Allen. 2016. A corpus and cloze evaluation for deeper understanding of commonsense stories. In Proc. of NAACL. find
  19. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100, 000+ questions for machine comprehension of text. In Proc. of EMNLP. our card
  20. Julian Salazar, Davis Liang, Toan Q. Nguyen, and Katrin Kirchhoff. 2020. Masked language model scoring. In Proc. of ACL. our card
  21. Naomi Saphra and Adam Lopez. 2019. Understanding learning dynamics of language models with SVCCA. In Proc. of NAACL. our card
  22. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proc. of EMNLP. find
  23. Alon Talmor, Yanai Elazar, Yoav Goldberg, and Jonathan Berant. 2019. oLMpics - on what language model pre-training captures. TACL. our card
  24. Trieu H. Trinh and Quoc V. Le. 2018. A simple method for commonsense reasoning. http://web.archive.org/save/http://commoncrawl. org/2016/10/news-dataset-availabl . ↗
  25. Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. 2020. BLiMP: The benchmark of linguistic minimal pairs for english. TACL. our card
  26. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proc. of EMNLP. our card
  27. Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. 2019. Neural network acceptability judgments. TACL. find
  28. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proc. of NAACL. our card
  29. Adina Williams, Nikita Nangia, and Samuel R. Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proc. of NAACL. find
  30. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. In Proc. of NeurIPS. our card
  31. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. 2019. Language models as knowledge bases? In Proc. of EMNLP. our card
  32. Tiago Pimentel, Josef Valvoda, Rowan Hall Maudslay, Ran Zmigrod, Adina Williams, and Ryan Cotterell. 2020. Information-theoretic probing for linguistic structure. In Proc. of ACL. our card
  33. Lianhui Qin, Vered Shwartz, Peter West, Chandra Bhagavatula, Jena D. Hwang, Ronan Le Bras, Antoine Bosselut, and Yejin Choi. 2020. Back to the future: Unsupervised backprop-based decoding for counterfactual and abductive commonsense reasoning. In Proc. of EMNLP. our card
  34. Sheng Zhang, Xiaodong Liu, Jingjing Liu, Jianfeng Gao, Kevin Duh, and Benjamin Van Durme. 2018. ReCoRD: Bridging the gap between human and machine commonsense reading comprehension. our card
  35. Yian Zhang, Alex Warstadt, Haau-Sing Li, and Samuel R. Bowman. 2020. When do you need billions of words of pretraining data? find
  36. Xuhui Zhou, Yue Zhang, Leyang Cui, and Dandan Huang. 2020. Evaluating commonsense in pre-trained language models. In Proc. of AAAI. find
  37. Yukun Zhu, Ryan Kiros, Richard S. Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In Proc. of ICCV. our card

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2021-probing-across-time-what-does/