A Primer in BERTology: What We Know About How BERT Works

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2020.tacl-1.54.pdf

Summary

BERT quickly became a default baseline across NLP, but it was still unclear why it worked so well, and that made principled improvements to the architecture hard. This TACL 2020 survey brings together findings from more than 150 analysis papers. It briefly recaps the model: stacked Transformer encoder layers, wordpiece input, pre-training with MLM and NSP, then task-specific fine-tuning. It then sorts the evidence on what syntactic, semantic and world knowledge BERT learns, how researchers detect it (cloze-style MLM probes, attention-weight analysis, probing classifiers) and where in the network that knowledge seems to sit. Later sections cover proposed changes to training objectives, architecture and fine-tuning, the evidence that the model is overparameterized, and compression methods. The overall finding is that BERT stores a lot of linguistic and factual information but often fails to use it robustly. It relies on shallow heuristics and struggles with reasoning, numbers and negation. The paper ends with a research agenda, which makes it a useful starting point for anyone analysing or adapting BERT-style models, including for Ukrainian NLP.

Key points

• Syntactic structure can be recovered from BERT's token embeddings: Hewitt and Manning learned linear transformations that reconstructed dependencies on Penn Treebank data. Full parse trees could not be extracted from individual attention heads, though.
• BERT is fragile in ways that undercut its apparent linguistic skill. Its outputs barely change under shuffled word order or truncated input (Ettinger, 2019), and swapping names in a coreference task changes 85% of its predictions.
• Through cloze queries, vanilla BERT matches knowledge-base methods for some relation types (Petroni et al., 2019, LAMA). However, it cannot reason over those facts, and some of its apparent world knowledge comes from stereotypical associations.
• Embeddings become more context-specific in later layers and fill a narrow, anisotropic cone in vector space. Many heads and layers can be disabled with little loss, and "winning" lottery-ticket subnetworks exist.
• The suggested research directions include harder benchmarks that need verbal reasoning, behavioural stress tests such as Checklist (Ribeiro et al., 2020), and caution when using pruning or ablation as analysis tools, because knowledge may be spread across the network or duplicated.