Elamur Research
☰
home / papers / 2019 / What Does BERT Learn about the Structure of Language?

What Does BERT Learn about the Structure of Language?

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/P19-1356.pdf

Summary

This paper asks which parts of English linguistic structure BERT actually encodes, and at which depth of the network. The authors probe each of the 12 layers of the pretrained bert-base-uncased model in four ways. First, they cluster span representations of CoNLL 2000 chunks. Second, they run the ten SentEval probing tasks of Conneau et al. (2018) and compare against an untrained BERT baseline. Third, they test subject-verb agreement on the Linzen et al. (2016) stimuli while controlling for the number of attractor nouns. Fourth, they fit Tensor Product Decomposition Networks (TPDN) with different role schemes to approximate BERT's representations on SNLI. Phrase-level information is strongest in the early layers and fades higher up. Lower layers favour surface features, middle layers syntactic ones, and upper layers semantic ones. Agreement across longer distances needs deeper layers. A tree-based role scheme fits BERT's representations best, so even a purely attention-based model ends up with implicitly hierarchical, compositional structure. This helps explain BERT's success and points to its limits.

Key points

  • Phrasal syntax: k-means clustering (k = 10) of span embeddings gives an NMI of 0.38 at layer 1, which drops to about 0.16–0.19 in layers 8–12.
  • SentEval probing shows a layered hierarchy. SentLen peaks at layer 3 (96.2), and an untrained BERT beats the trained model on it in higher layers. TopConst peaks at layer 7 (84.1), and BShift and CoordInv peak at layer 9, both above earlier published results.
  • Subject-verb agreement: with no attractors, accuracy reaches about 95.7 (layer 8). With 3–4 attractors, the best scores are around 49–50 and appear only at layer 9.
  • TPDN: a tree-based role scheme gives the lowest MSE against BERT at most layers, ahead of left-to-right, right-to-left, bag-of-words, bidirectional and random-tree schemes.
  • A dependency tree built from the self-attention weights of layer 2, head 11 using Chu-Liu-Edmonds recovers determiner–noun and subject–verb links. The code is published at github.com/ganeshjawahar/interpret_bert.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

568 papers of our library cite it (of the 140673 whose bibliographies we read); 21 of them have a page here, 547 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p19-1356

@inproceedings{jawahar2019what,
  title = {{What Does BERT Learn about the Structure of Language?}},
  author = {Jawahar, Ganesh and Sagot, Benoît and Seddah, Djamé},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1356},
  url = {https://doi.org/10.18653/v1/p19-1356}
}

References

24 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2017. Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks. International Conference on Learning Representations. find our card
  2. Yonatan Belinkov and James Glass. 2019. Analysis Methods in Neural Language Processing: A Survey. Transactions of the Association for Computational Linguistics. find our card
  3. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642. Association for Computational Linguistics. find
  4. Yoeng-Jin Chu and Tseng-Hong Liu. 1967. On the shortest arborescence of a directed graph. In Science Sinica, pages 1396–1400. find
  5. Alexis Conneau and Douwe Kiela. 2018. SentEval: An Evaluation Toolkit for Universal Sentence Representations. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018). European Language Resource Association. find our card
  6. Alexis Conneau, Germán Kruszewski, Guillaume Lample, Loı̈c Barrault, and Marco Baroni. 2018. What you can cram into a single \\$&!\#* vector: Probing sentence embeddings for linguistic properties. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Lo… find our card
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Computing Research Repository, arXiv:1810.04805. Version 1. find
  8. Yoav Goldberg. 2019. Assessing BERT’s Syntactic Abilities. Computing Research Repository, arXiv:1901.05287. Version 1. find our card
  9. Networks Process Hierarchical Structure. J. Artif. Int. Res., 61(1):907–926. find
  10. Dan Klein and Christopher D. Manning. 2003. Accurate unlexicalized parsing. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - Volume 1, ACL ’03, pages 423– 430, Stroudsburg, PA, USA. Association for Computational Linguistics. find our card
  11. Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016. Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies. Transactions of the Association for Computational Linguistics, 4:521– 535. find our card
  12. Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew Peters, and Noah A. Smith. 2019. Linguistic Knowledge and Transferability of Contextual Representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technol… find our card
  13. Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research, 9(Nov):2579–2605. find
  14. R. Thomas McCoy, Tal Linzen, Ewan Dunbar, and Paul Smolensky. 2019. RNNs Implicitly Implement Tensor Product Representations. International Conference on Learning Representations. find
  15. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global Vectors for Word Representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532– 1543. find
  16. Matthew Peters, Mark Neumann, Luke Zettlemoyer, and Wen-tau Yih. 2018. Dissecting Contextual Word Embeddings: Architecture and Representation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1499–1509. Association for Computational Linguistics. find our card
  17. John Hewitt and Christopher D. Manning. 2019. A Structural Probe for Finding Syntax in Word Representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics. find our card
  18. Alessandro Raganato and Jörg Tiedemann. 2018. An Analysis of Encoder Representations in Transformer-Based Machine Translation. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 287–297. Association for Computational Linguistics. find
  19. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. find
  20. Erik F. Tjong Kim Sang and Sabine Buchholz. 2000. Introduction to the CoNLL-2000 Shared Task Chunking. In Fourth Conference on Computational Natural Language Learning, CoNLL 2000, and the Second Learning Language in Logic Workshop, LLL 2000, Held in cooperation with ICGI-2000, Lisbon, Portugal, Sept… find
  21. Dieuwke Hupkes, Sara Veldhoen, and Willem Zuidema. 2018. Visualisation and ‘Diagnostic Classifiers’ Reveal How Recurrent and Recursive Neural Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Sam Bowman, Dipanjan Das, and Ellie Pavlick. 20… find our card
  22. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017,… find
  23. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353–355.… find our card
  24. Kelly W. Zhang and Samuel R. Bowman. 2018. Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis. Computing Research Repository, arXiv:1809.10040. Version 2. find