Elamur Research⌕
☰
home / papers / 2026 / Start Making Sense(s): A Developmental Probe of Attention Specialization Using Lexical Ambiguity

Start Making Sense(s): A Developmental Probe of Attention Specialization Using Lexical Ambiguity

Paper · 2026 · Pamela D. Rivière and Sean Trott

Summary

Researchers can describe how self-attention works mathematically, yet it remains unclear which attention heads perform which functions and when they acquire them during training. Rivière and Trott use lexical ambiguity as a test case. The same word, such as "lamb," can take a different sense depending on a preceding modifier, so the context changes while the target token stays fixed. Using the noun items of the RAW-C dataset, they measure how well cosine distances between contextual embeddings predict human relatedness ratings. They track this across Pythia checkpoints from 14M to 12B parameters. In Pythia-14M and Pythia-410M, they look for heads whose attention from the ambiguous noun to its cue rises and falls with disambiguation performance over training. They then test these heads against changes in cue position and part-of-speech, and ablate them to check causal impact. Heads in 410M generalize much better, while ablation hurts 14M more. Replications across 14M random seeds show similar learning curves but different locations for the key heads. Overall, the study suggests that tracking models through training reveals mechanisms a final checkpoint can hide.

Key points

  • Data: RAW-C contains 672 minimal sentence pairs. Only the 504 noun pairs were used (318 polysemous, 186 homonymous), because autoregressive Pythia models cannot see the post-verbal cues that disambiguate the verbs.
  • Checkpoints and metric: Pythia-14M was evaluated at all 154 checkpoints and the other models at 20. Performance is the R² from regressing relatedness on layer-wise embedding distance.
  • Scaling: Each tenfold increase in size added about 0.16 R², with diminishing returns. Pythia-410M reached R² = 0.524, about 92% of 12B with roughly 3% of its parameters. No model matched the human level of 0.64, though 6.9B came closest. Pythia-14M performed best at Layer 3 rather than its final layer.
  • Ablation and robustness: Ablating the candidate heads lowered R² about 30x more in 14M than in 410M, which has 16x as many heads. Heads in 410M were far more robust to perturbations of cue position and part-of-speech.
  • Seeds and code: Across the Pythia-14M random seeds, developmental trajectories were stable, but the layer hosting the key heads varied systematically. The authors also describe a "passing the baton" pattern, in which early-influential heads give way to others during training. Code and data are available at github.com/seantrott/entangled_meanings.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of California San DiegoSan Diego · United States73 works · 51,948 citations🎓Rutgers, The State University of New JerseyNew Brunswick · United States36 works · 12,026 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Word sense disambiguation · 83 works

Cite

Cite as Pamela D. Rivière and Sean Trott. 2026. Start Making Sense(s): A Developmental Probe of Attention Specialization Using Lexical Ambiguity. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl.a.691

@article{riviere2026start,
  title = {{Start Making Sense(s): A Developmental Probe of Attention Specialization Using Lexical Ambiguity}},
  author = {Pamela D. Rivière and Sean Trott},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.1162/tacl.a.691},
  url = {https://doi.org/10.1162/tacl.a.691}
}

References

49 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marta Vázquez Abuín and Marcos Garcia. 2025. Assessing lexical ambiguity resolution in language models with new wic datasets in galician and spanish. Procesamiento del Lenguaje Natural, 74:305–319. find
  2. Gioia Baldissin, Dominik Schlechtweg, and Sabine Schulte im Walde. 2022. DiaWUG: A dataset for diatopic lexical semantic variation in Spanish. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 2601–2609, Marseille, France. European Language Resources Association. find
  3. Yuchen Bian, Jiaji Huang, Xingyu Cai, Jiahong Yuan, and Kenneth Church. 2021. On attention redundancy: A comprehensive study. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 930–945, Online. Asso… find
  4. Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintana Sutawika, and Oskar van der Wal. 2023. Pythia: A suite for analyzing large language models across… find
  5. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the association for computational linguistics, 5:135–146. \#\# Acknowledgments We thank Benjamin K. Bergen, Cameron R. Jones, Catherine Arnett, Tyler A. Chang, Jame…
  6. Angelica Chen, Ravid Shwartz-Ziv, Kyunghyun Cho, Matthew L. Leavitt, and Naomi Saphra. 2024. Sudden drops in the loss: Syntax acquisition, phase transitions, and simplicity bias in mlms. In The Twelfth International Conference on Learning Representations. of Proceedings of Machine Learning Research,… find
  7. Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019. What does bert look at? an analysis of bert’s attention. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, page 276. Association for Computational Linguistics. find
  8. Jumbly Grindrod. 2024. Transformers, contextualism, and polysemy. arXiv preprint arXiv:2404.09577. — [arXiv:2404.09577](https://arxiv.org/abs/2404.09577) find
  9. Janosch Haber and Massimo Poesio. 2020. Word sense distance in human similarity judgements and contextualised word embeddings. In Proceedings of the Probability and Meaning Conference (PaM 2020), pages 128–145. find
  10. Arthur Conmy, Augustine Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso. 2023. Towards automated circuit discovery for mechanistic interpretability. Advances in Neural Information Processing Systems, 36:16318–16352. find
  11. Janosch Haber and Massimo Poesio. 2021. Patterns of lexical ambiguity in contextualised language models. arXiv preprint arXiv:2109.13032. — [arXiv:2109.13032](https://arxiv.org/abs/2109.13032) find
  12. Isabelle Dautriche. 2015. Weaving an ambiguous lexicon. Ph.D. thesis, Université Sorbonne Paris Cité. find
  13. Janosch Haber and Massimo Poesio. 2024. Polysemy—evidence from linguistics, behavioral science, and contextualized language models. Computational Linguistics, 50(1):351–417.
  14. Kaya De Barbaro, Christine M. Johnson, and Gedeon O. Deák. 2013. Twelve-month “social revolution”emerges from mother-infant sensorimotor coordination: A longitudinal investigation. Human Development, 56(4):223–248. find
  15. Tal Haklay, Hadas Orgad, David Bau, Aaron Mueller, and Yonatan Belinkov. 2025. Position-aware automatic circuit discovery. arXiv preprint arXiv:2502.04577. find
  16. Shwai He, Guoheng Sun, Zheyu Shen, and Ang Li. 2024. What matters in transformers? not all attention is needed. arXiv preprint arXiv:2406.15786. — [arXiv:2406.15786](https://arxiv.org/abs/2406.15786) find
  17. Finlay Fehlauer, Kyle Mahowald, and Tiago Pimentel. 2025. Convergence and divergence of language models under different random seeds. arXiv preprint arXiv:2509.26643. find
  18. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, E… — Training Compute-Optimal Large Language Models
  19. Marcos García. 2021. Exploring the representation of word meanings in context: A case study on homonymy and synonymy. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long… find
  20. Aina Garí Soler and Marianna Apidianaki. 2021. Let’s play mono-poly: Bert can reveal words’ polysemy level and partitionability into senses. Transactions of the Association for Computational Linguistics, 9:825–844.
  21. Michael Y Hu, Angelica Chen, Naomi Saphra, and Kyunghyun Cho. 2023. Latent state models of training dynamics. arXiv preprint arXiv:2308.09543. find
  22. Atticus Geiger, Zhengxuan Wu, Hanson Lu, Josh Rozner, Elisa Kreiss, Thomas Icard, Noah Goodman, and Christopher Potts. 2022. Inducing causal structure for interpretable neural networks. In Proceedings of the 39th International Conference on Machine Learning, volume 162 find
  23. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Askell Amanda, Bai Yuntao, Chen Anna, Conerly Tom, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt,… find
  24. Dario Amodei. 2020. Scaling laws for neural language models. CoRR, abs/2001.08361.
  25. Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. 2019. Revealing the dark secrets of BERT. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4365–437…
  26. Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord, Taira Anderson, David Atkinson, Faeze Brahman, Christopher Clark, Pradeep Dasigi, Nouha Dziri, Michal Guerquin, Hamish I… find
  27. Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. 2024. Talking heads: Understanding inter-layer communication in transformer language models. Advances in Neural Information Processing Systems, 37:61372–61418. find
  28. Paul Michel, Omer Levy, and Graham Neubig. 2019. Are sixteen heads really better than one? Advances in neural information processing systems, 32.
  29. Aaron Mueller, Jannik Brinkmann, Millicent Li, Samuel Marks, Koyena Pal, Nikhil Prakash, Can Rager, Aruna Sankaranarayanan, Arnab Sen Sharma, Jiuding Sun, Eric Todd, David Bau, and Yonatan Belinkov. 2025. The quest for the right mediator: Surveying mechanistic interpretability through the lens of ca… find
  30. Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amode… — In-context Learning and Induction Heads
  31. Aakanksha Naik, Abhilasha Ravichander, Norman Sadeh, Carolyn Rose, and Graham Neubig. 2018. Stress test evaluation for natural language inference. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2340–2353, Santa Fe, New Mexico, USA. Association for Computation… find
  32. Yein Park, Chanwoong Yoon, Jungwoo Park, Minbyul Jeong, and Jaewoo Kang. 2025. Does time have its place? temporal heads: Where language models recall time-specific information. arXiv preprint arXiv:2502.14258. find
  33. Sathvik Nair, Mahesh Srinivasan, and Stephan Meylan. 2020. Contextualized word embeddings encode aspects of human-like word sense knowledge. In Proceedings of the Workshop on the Cognitive Aspects of the Lexicon, pages 129–141. find
  34. Astrid A. Prinz, Dirk Bucher, and Eve Marder. 2004. Similar network activity from disparate circuit parameters. Nature neuroscience, 7(12):1345–1352. find
  35. Roberto Navigli. 2009. Word sense disambiguation: A survey. ACM computing surveys (CSUR), 41(2):1–69. find
  36. Pamela D. Rivière, Anne L. Beatty-Martínez, and Sean Trott. 2025. Evaluating contextualized representations of (Spanish) ambiguous words: Elhage Nelson, Nanda Neel, Olsson Catherine, Henighan Tom, Joseph Nicholas, Mann Ben, Murfet, and Tom McGrath. 2025. Open problems in mechanistic interpretability… — [arXiv:2501.16496](https://arxiv.org/abs/2501.16496) find
  37. A new lexical resource and empirical analysis. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8322–8338. find
  38. Curt Tigges, Michael Hanna, Qinan Yu, and Stella Biderman. 2024. LLM circuit analyses are consistent across training and scale. arXiv preprint arXiv:2407.10827. — [arXiv:2407.10827](https://arxiv.org/abs/2407.10827) find
  39. Jennifer M. Rodd, M. Gareth Gaskell, and William D. Marslen-Wilson. 2004. Modelling the effects of semantic ambiguity in word recognition. Cognitive science, 28(1):89–104. find
  40. Sean Trott. 2025. Toward a theory of generalizability in LLM mechanistic interpretability research. In Mechanistic Interpretability Workshop at NeurIPS 2025. find
  41. Dominik Schlechtweg, Barbara McGillivray, Simon Hengchen, Haim Dubossarsky, and Nina Tahmasebi. 2020. Semeval-2020 task 1: Unsupervised lexical semantic change detection. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1–23.
  42. Sean Trott and Benjamin Bergen. 2021. RAW-C: Relatedness of ambiguous words in context (a new lexical resource for English). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1…
  43. Dominik Schlechtweg, Shafqat Mumtaz Virk, Pauline Sander, Emma Sköldberg, Lukas Theuer Linke, Tuo Zhang, Nina Tahmasebi, Jonas Kuhn, and Sabine Schulte Im Walde. 2024. The DURel annotation tool: Human and computational measurement of semantic proximity, sense clusters and semantic change. In Proceed… find
  44. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
  45. Oskar van der Wal, Pietro Lesci, Max Muller-Eberstein, Naomi Saphra, Hailey Schoelkopf, Willem Zuidema, and Stella Biderman. 2025. Polypythias: Stability and outliers across fifty language model pre-training runs. arXiv preprint arXiv:2503.09543. — [arXiv:2503.09543](https://arxiv.org/abs/2503.09543) find
  46. Natalie Shapira, Mosh Levy, Seyed Hossein Alavi, Xuhui Zhou, Yejin Choi, Yoav Goldberg, Maarten Sap, and Vered Shwartz. 2024. Clever hans or neural theory of mind? stress testing social reasoning in large language models. In Proceedings of the 18th Conference of the European Chapter of the Associati… find
  47. Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2023. Interpretability in the wild: a circuit for indirect object identification in GPT-2 Small. In The Eleventh International Conference on Learning Representations. find
  48. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoe… — Transformers: State-of-the-Art Natural Language Processing
  49. System Demonstrations, pages 38–45, Online. Association for Computational Linguistics. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/riviere-2026-start-making-sense-s-developmental/