Elamur Research
☰
home / papers / 2019 / Visualizing and Measuring the Geometry of BERT

Visualizing and Measuring the Geometry of BERT

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1906.02715

Summary

The paper asks how BERT internally organizes the linguistic knowledge that makes it so easy to fine-tune. It combines visualizations with quantitative tests. For syntax, the Google Brain authors build "attention probes". For each token pair, they gather the attention values from every head and layer of BERT-base into a single vector. They then train simple linear classifiers on Penn Treebank sentences converted to dependency form.

They also give a mathematical explanation for Hewitt and Manning's observation that squared Euclidean distance between context embeddings tracks parse tree distance. Such "Pythagorean" tree embeddings always exist and are easy to build, even approximately with random vectors. For semantics, they study word senses using SemCor. They train a probe that projects embeddings into a smaller subspace and test how merging unrelated contexts shifts sense representations.

The central finding is that syntactic and semantic information appear to occupy separate, relatively low-dimensional linear subspaces of the same vectors. This matters for interpretability. It also exposes a failure mode of attention, where tokens absorb meaning across sentence boundaries, and it offers an alternative to hyperbolic tree embeddings.

Key points

  • Attention probe data: The probe uses Penn Treebank parses converted with PyStanfordDependencies. Of the 3.1 million dependency relations, only the 30 relation types with more than 5,000 examples were kept.
  • Attention probe results: Linear probes on model-wide attention vectors reach 85.8% accuracy for detecting whether a dependency exists. They reach 71.9% for predicting its type.
  • Tree embedding theorems: Every n-node tree can be given a Pythagorean (power-2) embedding in R^(n−1). For every p < 2, some tree has no power-p embedding. A "random branch embedding" built from random unit vectors approximates the Pythagorean case in high dimensions.
  • Concatenation experiment: The authors joined 25,096 SemCor sentence pairs that use one keyword in different senses. This raised the final-layer sense misclassification rate from 2.43% (single sentences) to 8.23% (concatenated).
  • Semantic subspace projection: Projecting embeddings into the learned semantic subspace raised similarity ratios from 1.43 to 1.88 for single sentences and from 1.28 to 1.58 for concatenated ones. Misclassification fell to 2.27% and 7.31% respectively.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

175 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 167 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1906.02715 · arXiv 1906.02715

@misc{coenen2019visualizing,
  title = {{Visualizing and Measuring the Geometry of BERT}},
  author = {Coenen, Andy and Reif, Emily and Yuan, Ann and Kim, Been and Pearce, Adam and Viegas, Fernanda and Wattenberg, Martin},
  year = {2019},
  doi = {10.48550/arxiv.1906.02715},
  eprint = {1906.02715},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1906.02715}
}

References

28 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Terra Blevins, Omer Levy, and Luke Zettlemoyer. Deep rnns encode soft hierarchical syntax. arXiv preprint arXiv:1805.04218, 2018. find
  2. Shan Carter, Zan Armstrong, Ludwig Schubert, Ian Johnson, and Chris Olah. Activation atlas. Distill, 2019. https://distill.pub/2019/activation-atlas. ↗
  3. Alexis Conneau, German Kruszewski, Guillaume Lample, Loïc Barrault, and Marco Baroni. What you can cram into a single vector: Probing sentence embeddings for linguistic properties. arXiv preprint arXiv:1805.01070, 2018. find our card
  4. Marie-Catherine De Marneffe, Bill MacCartney, Christopher D Manning, et al. Generating typed dependency parses from phrase structure parses. In Lrec, volume 6, pages 449–454, 2006. find our card
  5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. find
  6. John Hewitt and Christopher D Manning. A structural probe for finding syntax in word representations. Association for Computational Linguistics, 2019. find our card
  7. Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory Sayres. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). arXiv preprint arXiv:1711.11279, 2017. — [arXiv:1711.11279](https://arxiv.org/abs/1711.11279) find
  8. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012. find
  9. Yann LeCun, Yoshua Bengio, et al. Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks, 3361(10):1995, 1995. find
  10. Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. Assessing the ability of lstms to learn syntax-sensitive dependencies. Transactions of the Association for Computational Linguistics, 4:521–535, 2016. find our card
  11. Nelson F Liu, Matt Gardner, Yonatan Belinkov, Matthew Peters, and Noah A Smith. Linguistic knowledge and transferability of contextual representations. arXiv preprint arXiv:1903.08855, 2019. find our card
  12. Hiroshi Maehara. Euclidean embeddings of finite metric spaces. Discrete Mathematics, 313(23):2848–2856, 2013. find
  13. Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. Building a large annotated corpus of english: The penn treebank. Comput. Linguist., 19(2):313–330, June 1993. find our card
  14. David McClosky. Pystanforddependencies. PyStanfordDependencies, 2015. find
  15. Leland McInnes, John Healy, and James Melville. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426, 2018. find our card
  16. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119, 2013. find
  17. George A. Miller, Claudia Leacock, Randee Tengi, and Ross Bunker. A semantic concordance. In HLT, 1993. find
  18. Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical representations. In Advances in neural information processing systems, pages 6338–6347, 2017. https://github.com/dmcc/ ↗ our card
  19. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830… find
  20. Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations. arXiv preprint arXiv:1802.05365, 2018. find
  21. Alessandro Raganato, Jose Camacho-Collados, and Roberto Navigli. Word sense disambiguation: A unified evaluation framework and empirical comparison. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 99–110, Va… find our card
  22. Isaac J Schoenberg. On certain metric spaces arising from euclidean spaces by a change of metric and their imbedding in hilbert space. Annals of mathematics, pages 787–793, 1937. find
  23. Ian Tenney, Dipanjan Das, and Ellie Pavlick. Bert rediscovers the classical nlp pipeline. arXiv preprint arXiv:1905.05950, 2019. find our card
  24. Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R Bowman, Dipanjan Das, et al. What do you learn from context? probing for sentence structure in contextualized word representations. 2018. find our card
  25. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008, 2017. find
  26. Jesse Vig. Visualizing attention in transformer-based language models. arXiv preprint arXiv:1904.02679, 2019. — [arXiv:1904.02679](https://arxiv.org/abs/1904.02679) find
  27. Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In European conference on computer vision, pages 818–833. Springer, 2014. find our card
  28. Yury. Minimizing the maximum dot product among k unit vectors in an n-dimensional space. https://cstheory.stackexchange.com/q/14748, 2012. Accessed: 2019-05-09. ↗