📄 Probing the Attention Representation of Filler-Gap Dependency in Transformers

Summary

Earlier surprisal studies found that language models are sensitive to filler-gap dependencies (FGDs). These effects weaken as more clauses separate filler and gap, and they also weaken, puzzlingly, when the complementizer "that" is overt. This paper looks inside GPT-2 (12 layers × 12 heads) to see what attention does in these cases. The measure is the attention weight from the verb at the gap site back to the wh-filler. The authors test it on controlled item sets that vary:

• clausal boundaries (0C/1C/2C),
• linear distance,
• island type,
• relative clauses,
• whether complementizers are overt.

Two heads, h(5,2) and h(8,9), selectively mark grammatical gaps. Their attention also correlates with filled-gap surprisal. This attention survives long linear distance but drops with clausal embedding and islands.

Head h(5,2) moves its attention toward the nearest overt C. This offers a possible explanation for why overt Cs dampen surprisal effects.

Single-head ablation with TransformerLens supports a role for h(5,2) but not for h(8,9). The strongest ablation effect comes from h(1,10), a head that probing did not flag. The authors conclude that FGD processing is spread across partly specialized heads, and that attention weights tell only part of the story.

Key points

• Probing results: h(5,2) has the highest verb-to-filler attention of all 144 heads in wh- and relative-clause conditions. Both h(5,2) and h(8,9) correlate significantly with filled-gap surprisal (β = 6.33 and 5.61, p < .001).
• Islands: the materials are 50 island item sets from Kobzeva et al. (2025). h(5,2) is graded, dropping more for CNPC (strong) islands than for whether-islands (weak). h(8,9) treats both island types alike.
• Overt complementizers: h(5,2) follows an "attend closest C" pattern, and its filler attention correlates with surprisal (β = 12.59). h(8,9) attends less to the filler when Cs are overt but still focuses mainly on it.
• Ablation: one head at a time was zeroed out using TransformerLens. h(1,10) has the strongest effect and h(5,2) the second strongest in [+wh]. Ablating h(8,9) has no predicted effect. Effects are much weaker in [+th] items, which lack a filler.
• Limitations and future work: the authors stress that neither probing nor ablation shows true causality. They propose activation patching as the next step.