Elamur Research⌕
☰
home / papers / 2020 / Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT

Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT

Summary

Standard probing classifiers add trainable parameters on top of a pre-trained model. This makes it unclear whether linguistic knowledge comes from the model or is learned by the probe itself. Wu et al. propose Perturbed Masking, a probing method with no parameters and no supervision. It estimates how much one word influences BERT's prediction of another. First the target token is masked, then a context token is also masked, and the resulting change in the target's representation is measured. Repeating this for every token pair gives an impact matrix. Graph-based parsing algorithms then turn this matrix into dependency trees, constituency trees, and, through span-level masking, document structures. The authors tested the approach on the English PUD treebank and on WSJ10-U, a Penn Treebank subset converted to Universal Dependencies. The induced trees beat linguistically uninformed baselines, though often only modestly. For discourse structure they fell below simple baselines. When the BERT-derived dependency structures were fed into a downstream sentiment classifier, results matched or exceeded those obtained with parser-produced, linguist-designed dependencies. The work offers a lower bound on BERT's unsupervised syntactic ability.

Key points

  • Two-stage masking: the impact score is the distance between the target token's representations with and without the context word masked. Two variants of the distance were tried: Euclidean distance ("Dist") and the change in the target token's predicted probability ("Prob").
  • Model and subwords: experiments used BERT base, uncased (Wolf et al. implementation). For split-up subword tokens, all pieces are masked together and their impacts are averaged.
  • Data: 1,000 sentences from the English PUD treebank (CoNLL 2017 Shared Task) and 7,422 short sentences in WSJ10-U, converted with Stanford CoreNLP.
  • Tree extraction and evaluation: projective trees were built with the Eisner algorithm and non-projective trees with Chu-Liu/Edmonds, using the gold root. Results were scored with UAS against right/left-chain baselines and a randomly initialized BERT.
  • Downstream test: the induced dependency structures were plugged into the sentiment system of Zhang et al. (2019), performing on par with or better than parser-provided trees.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Dependency parsing · Treebanks

Cited by

92 papers of our library cite it (of the 140884 whose bibliographies we read); 2 of them have a page here, 90 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.acl-main.383

@inproceedings{wu2020perturbed,
  title = {{Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT}},
  author = {Wu, Zhiyong and Chen, Yun and Kao, Ben and Liu, Qun},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.383},
  url = {https://doi.org/10.18653/v1/2020.acl-main.383}
}

References

58 references, 28 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2017. Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks. In ICLR ’17. find our card
  2. Geoff Bacon and Terry Regier. 2019. Does BERT agree? Evaluating knowledge of structure dependence through agreement relations. ArXiv. find
  3. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski. 2003. Building a discourse-tagged corpus in the framework of rhetorical structure theory. In Current and new directions in discourse and dialogue, pages 85–112. Springer. find our card
  4. Yoeng-Jin Chu. 1965. On the shortest arborescence of a directed graph. Scientia Sinica, 14:1396–1400. find
  5. Alexis Conneau, Germán Kruszewski, Guillaume Lample, Loı̈c Barrault, and Marco Baroni. 2018. What you can cram into a single vector: Probing sentence embeddings for linguistic properties. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pape… find our card
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. ArXiv, abs/1810.04805. find
  7. Chris Dyer, Gábor Melis, and Phil Blunsom. 2019. A critical analysis of biased parsers in unsupervised parsing. arXiv preprint arXiv:1909.09428. — [arXiv:1909.09428](https://arxiv.org/abs/1909.09428) find
  8. Jack Edmonds. 1967. Optimum branchings. Journal of Research of the national Bureau of Standards B, 71(4):233–240. find
  9. Jason Eisner and Noah A Smith. 2010. Favor short dependencies: Parsing with soft and hard constraints on dependency length. In Trends in Parsing Technology, pages 121–150. Springer. find
  10. Jason M Eisner. 1996. Three new probabilistic models for dependency parsing: An exploration. In Proceedings of the 16th conference on Computational linguistics-Volume 1, pages 340–345. Association for Computational Linguistics. find our card
  11. Mario Giulianelli, Jack Harding, Florian Mohnert, Dieuwke Hupkes, and Willem Zuidema. 2018. Under the hood: Using diagnostic classifiers to investigate and improve how language models track agreement information. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neura… find
  12. Yoav Goldberg. 2019. Assessing BERT’s Syntactic Abilities. ArXiv, pages 2–5. find our card
  13. Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni. 2018. Colorless Green Recurrent Networks Dream Hierarchically. In NAACL-HLT, pages 1195–1205. find our card
  14. John Hewitt and Percy Liang. 2019. Designing and interpreting probes with control tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2733–2743. find
  15. Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019. What Does BERT Look at? An Analysis of BERT’s Attention. In BlackboxNLP@ACL, pages 276–286. find
  16. John Hewitt and Christopher D. Manning. 2019. A Structural Probe for Finding Syntax in Word Representations. In Naacl, pages 4129–4138. find our card
  17. Michael Collins. 1999. HEAD-DRIVEN STATISTICAL MODELS FOR NATURAL LANGUAGE PARSING. Ph.D. thesis, University of Pennsylvania. find
  18. Phu Mon Htut, Jason Phang, Shikha Bordia, and Samuel R. Bowman. 2019. Do attention heads in bert track syntactic dependencies? find
  19. Sarthak Jain and Byron C Wallace. 2019. Attention is not explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3543–3556. find our card
  20. Ganesh Jawahar, Benoı̂t Sagot, and Djamé Seddah. 2019. What Does BERT Learn about the Structure of Language? In ACL, pages 3651–3657. find our card
  21. Dan Klein and Christopher D Manning. 2004. Corpus-based induction of syntactic structure: Models of dependency and constituency. In Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, page 478. Association for Computational Linguistics. find
  22. Sandra Kübler, Ryan McDonald, and Joakim Nivre. 2009. Dependency parsing. Synthesis Lectures on Human Language Technologies, 1(1):1–127. find
  23. Bowen Li, Lili Mou, and Frank Keller. 2019a. An imitation learning approach to unsupervised parsing. arXiv preprint arXiv:1906.02276. find
  24. David Mareček and Rudolf Rosa. 2019. From Balustrades to Pierre Vinken: Looking for Syntax in Transformer Self-Attentions. In BlackboxNLP@ACL, pages 263–275. find
  25. Rebecca Marvin and Tal Linzen. 2018. Targeted syntactic evaluation of language models. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1192–1202. find
  26. Matthew Peters, Mark Neumann, Luke Zettlemoyer, and Wen-tau Yih. 2018a. Dissecting contextual word embeddings: Architecture and representation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1499–1509. find our card
  27. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke S. Zettlemoyer. 2018b. Deep contextualized word representations. ArXiv, abs/1802.05365. find
  28. Livia Polanyi. 1988. A formal model of the structure of discourse. Journal of pragmatics, 12(5-6):601–638. find
  29. Sujian Li, Liang Wang, Ziqiang Cao, and Wenjie Li. 2014. Text-level discourse dependency parsing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 25–35. find
  30. Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. SemEval-2014 task 4: Aspect based sentiment analysis. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 27–35, Dublin, Ireland. find our card
  31. Xintong Li, Guanlin Li, Lemao Liu, Max Meng, and Shuming Shi. 2019b. On the Word Alignment from Neural Machine Translation. In ACL, pages 1293– 1303. find
  32. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind K Joshi, and Bonnie L Webber. The penn discourse treebank 2.0. Citeseer. find our card
  33. Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016. Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies. TACL, 4(1990):521–535. find our card
  34. Alessandro Raganato and Jörg Tiedemann. 2018. An Analysis of Encoder Representations in Transformer-Based Machine Translation. In BlackboxNLP@EMNLP, pages 287–297. find our card
  35. Nelson F Liu, Matt Gardner, Yonatan Belinkov, Matthew E Peters, and Noah A Smith. 2019. Linguistic knowledge and transferability of contextual representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technol… find our card
  36. Rudolf Rosa and David Mareček. 2019. Inducing Syntactic Trees from BERT Representations. In BlackboxNLP@ACL. find
  37. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David Mc-Closky. 2014. The Stanford CoreNLP natural language processing toolkit. In Association for Computational Linguistics (ACL) System Demonstrations, pages 55–60. find our card
  38. Roy Schwartz, Omri Abend, Roi Reichart, and Ari Rappoport. 2011. Neutralizing linguistically problematic annotations in unsupervised dependency parsing evaluation. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages… find
  39. Mitchell Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of english: The penn treebank. find our card
  40. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715– 1725. find
  41. David Mareček and Rudolf Rosa. 2018. Extracting Syntactic Trees from Transformer Encoder Self-Attentions. In BlackboxNLP@EMNLP, pages 347– 349. find
  42. Sofia Serrano and Noah A. Smith. 2019. Is attention interpretable? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy. find our card
  43. Yikang Shen, Zhouhan Lin, Chin wei Huang, and Aaron Courville. 2018. Neural language modeling by jointly learning syntax and lexicon. In International Conference on Learning Representations. find
  44. Yikang Shen, Shawn Tan, Alessandro Sordoni, and Aaron Courville. 2019. Ordered neurons: Integrating tree structures into recurrent neural networks. find
  45. Haoyue Shi, Hao Zhou, Jiaze Chen, and Lei Li. 2018. On tree-based neural sentence modeling. In International Conference on Learning Representations. find
  46. Wei Shi and Vera Demberg. 2019. Next sentence prediction helps implicit discourse relation classification within and across domains. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EM… find our card
  47. Xing Shi, Inkit Padhi, and Kevin Knight. 2016. Does String-Based Neural MT Learn Source Syntax? In EMNLP, Table 2, pages 1526–1534. find our card
  48. Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019a. Bert rediscovers the classical nlp pipeline. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics. find our card
  49. An Yang and Sujian Li. 2018. Scidtb: Discourse dependency treebank for scientific abstracts. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 444–449. find our card
  50. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. ArXiv, abs/1906.08237. find our card
  51. Daniel Zeman, Martin Popel, Milan Straka, Jan Hajic, Joakim Nivre, Filip Ginter, Juhani Luotolahti, Sampo Pyysalo, Slav Petrov, Martin Potthast, et al. 2017. find
  52. Conll 2017 shared task: Multilingual parsing from raw text to universal dependencies. In CoNLL 2017 Shared Task, pages 1–19. Association for Computational Linguistics. find our card
  53. Chen Zhang, Qiuchi Li, and Dawei Song. 2019. Syntax-aware aspect-level sentiment classification with proximity-weighted convolution network. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1145–1148. ACM. find
  54. Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R. Bowman, Dipanjan Das, and Ellie Pavlick. 2019b. What do you learn from context? Probing for sentence structure in contextualized word representations. In ICLR, pages 1–17. find our card
  55. Reut Tsarfaty, Joakim Nivre, and Evelina Ndersson. 2011. Evaluating dependency parsing: robust and heuristics-free cross-nnotation evaluation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 385–396. Association for Computational Linguistics. find
  56. Sarah Wiegreffe and Yuval Pinter. 2019. Attention is not not explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 11–20. find our card
  57. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface’s transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771. find our card
  58. Hiroyasu Yamada and Yuji Matsumoto. 2003. Statistical dependency analysis with support vector machines. In Proceedings of the Eighth International Conference on Parsing Technologies, pages 195– 206. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2020-perturbed-masking-parameter-free-probing/