Elamur Research⌕
☰
home / papers / 2021 / BERT Busters: Outlier Dimensions that Disrupt Transformers

BERT Busters: Outlier Dimensions that Disrupt Transformers

Paper · 2021 · Olga Kovaleva, Saurabh Kulshreshtha, Anna Rogers et al.

Summary

Earlier work found that pre-trained Transformers lose little when large fractions of their weights are pruned. This paper shows the opposite can also hold: such models break down when a tiny, carefully chosen set of parameters is removed. The authors locate "outlier dimensions" in the output LayerNorm of each encoder layer. These are hidden positions where both the scaling factor and the bias have unusually large magnitude, and they recur at the same index across most layers. Setting these parameters to zero in every layer masks the corresponding feature. This sharply raises MLM loss and lowers GLUE scores, even though under 0.0001% of the weights change. The effect appears in six BERT-family models and also in ELECTRA, BART, XLNet and, in its final dense sublayer, GPT-2. To see how the outliers arise, the authors pre-train a BERT-medium from scratch and find that they form gradually and early. The work suggests that the final per-layer transformations matter more than commonly assumed. It also points to risks such as stealthy model-tampering attacks and to new considerations for pruning and initialization.

Key points

  • Detection heuristic: a dimension counts as an outlier when its output LayerNorm scale and bias both lie at least 3σ from their means in at least half of the layers.
    • For BERT-large, the layer requirement was relaxed to one third.
    • For RoBERTa, the threshold was relaxed to 2σ.
  • Size of the effect: zeroing these weights reduces downstream performance by as much as 44 points.
    • In BERT-base-uncased, the outliers are dimensions 308 and 381.
    • Up to 95% of WikiText tokens show abnormally large values at these positions.
  • Other components: masking the same positions in the input embeddings or in the linear layers had no significant effect.
    • Google and HuggingFace checkpoints of BERT-base place the outliers at different positions.
  • Pre-training experiment: the authors trained BERT-medium (8 layers, 512 hidden units) on BookCorpus, saving checkpoints every 2000 steps.
    • The outlier weights began to diverge around step 50k, roughly when the loss started falling.
    • One outlier dimension emerged (417).
  • Practical implications: the authors flag possible attacks that modify very few weights and recommend verifying model checksums.
    • The experiments used about 436.8 kWh of energy, roughly 108 kg of CO2.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of CopenhagenCopenhagen · Denmark78 works · 14,228 citations🎓University of Massachusetts LowellLowell · United States8 works · 3,758 citations

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksComputer Vision and Pattern Recognition · 1,282 works

Cite

Cite as Olga Kovaleva, Saurabh Kulshreshtha, Anna Rogers and Anna Rumshisky. 2021. BERT Busters: Outlier Dimensions that Disrupt Transformers. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.findings-acl.300

@inproceedings{kovaleva2021bert,
  title = {{BERT Busters: Outlier Dimensions that Disrupt Transformers}},
  author = {Olga Kovaleva, Saurabh Kulshreshtha, Anna Rogers and Anna Rumshisky},
  booktitle = {Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021},
  year = {2021},
  doi = {10.18653/v1/2021.findings-acl.300},
  url = {https://doi.org/10.18653/v1/2021.findings-acl.300}
}

References

32 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450.
  2. Tianlong Chen, Jonathan Frankle, Shiyu Chang, Sijia Liu, Yang Zhang, Zhangyang Wang, and Michael Carbin. 2020. The lottery ticket hypothesis for pre-trained bert networks. In Advances in Neural Information Processing Systems, volume 33, pages 15834–15846. Curran Associates, Inc. find
  3. Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. ELECTRA: pre-training text encoders as discriminators rather than generators. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. find
  4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  5. Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah Smith. 2020. Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping. arXiv:2002.06305 [cs]. find
  6. Kawin Ethayarajh. 2019. How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Process… find
  7. Angela Fan, Edouard Grave, and Armand Joulin. 2019. Reducing Transformer Depth on Demand with Structured Dropout. In International Conference on Learning Representations.
  8. Jonathan Frankle and Michael Carbin. 2019. The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. In International Conference on Learning Representations. find
  9. Prakhar Ganesh, Yao Chen, Xin Lou, Mohammad Ali Khan, Yin Yang, Deming Chen, Marianne Winslett, Hassan Sajjad, and Preslav Nakov. 2020. Compressing large-scale transformer-based models: A case study on BERT. arXiv preprint arXiv:2002.11985.
  10. Mitchell Gordon, Kevin Duh, and Nicholas Andrews. 2020. Compressing BERT: Studying the effects of weight pruning on transfer learning. In Proceedings of the 5th Workshop on Representation Learning for NLP, pages 143–155, Online. Association for Computational Linguistics.
  11. Demi Guo, Alexander M. Rush, and Yoon Kim. 2020. Parameter-Efficient Transfer Learning with Diff Pruning. arXiv:2012.07463 [cs].
  12. Song Han, Jeff Pool, John Tran, and William J. Dally. 2015. Learning both weights and connections for efficient neural network. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pag… find
  13. Karl Moritz Hermann, Tomás Kociský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-…
  14. Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. 2020. TinyBERT: Distilling BERT for natural language understanding. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4163–4174, Online. Association for Computational Linguis… find
  15. Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. 2019. Revealing the Dark Secrets of BERT. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4356–436…
  16. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the…
  17. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. find
  18. Ziyang Luo, Artur Kulmizev, and Xiaoxi Mao. 2021. Positional artefacts propagate through masked language model embeddings. find
  19. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. Pointer sentinel mixture models. CoRR, abs/1609.07843. language understanding. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-1…
  20. Paul Michel, Omer Levy, and Graham Neubig. 2019. Are Sixteen Heads Really Better than One? Advances in Neural Information Processing Systems 32 (NIPS 2019).
  21. Toan Q. Nguyen and Julian Salazar. 2019. Transformers without tears: Improving the normalization of self-attention. CoRR, abs/1910.05895. find
  22. Sai Prasanna, Anna Rogers, and Anna Rumshisky. 2020. When BERT Plays the Lottery, All Tickets Are Winning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3208–3229, Online. Association for Computational Linguistics. find
  23. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. find
  24. Ofir Zafrir, Guy Boudoukh, Peter Izsak, and Moshe Wasserblat. 2019. Q8BERT: Quantized 8Bit BERT. In 5th Workshop on Energy Efficient Machine Learning and Cognitive Computing (NeurIPS 2019). find
  25. Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In Proceedings of the IEEE international conference on computer vision, pages 19–…
  26. Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020. A Primer in BERTology: What We Know About How BERT Works. Transactions of the Association for Computational Linguistics, 8:842–866.
  27. Sheng Shen, Zhewei Yao, Amir Gholami, Michael W. Mahoney, and Kurt Keutzer. 2020. Powernorm: Rethinking batch normalization in transformers. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learni… find
  28. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017,…
  29. Alex Wang, Amapreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2018. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353–35…
  30. Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang, and Tieyan Liu. 2020. On layer normalization in the transformer architecture. In International Conference on Machine Learning, pages 10524–10533. PMLR. find
  31. Jingjing Xu, Xu Sun, Zhiyuan Zhang, Guangxiang Zhao, and Junyang Lin. 2019. Understanding and improving layer normalization. In Advances in Neural Information Processing Systems, volume 32, pages 4381–4391. Curran Associates, Inc. find
  32. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019. Xlnet: Generalized autoregressive pretraining for find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kovaleva-2021-bert-busters-outlier-dimensions-that/