Elamur Research⌕
☰
home / papers / 2026 / Bridging Linguistic Structure and Mechanistic Interpretability for Conceptual Interpretation in Language Models

Bridging Linguistic Structure and Mechanistic Interpretability for Conceptual Interpretation in Language Models

Summary

The authors test whether the mechanisms that interpretability researchers have identified in transformers, mostly on factual recall tasks, also operate when a model infers an abstract concept from its definition. To this end, they propose the DSRA (Definitional Semantic Role Analysis) methodology. It transfers ROME-style causal tracing to the reverse dictionary task, in which the model receives a dictionary definition and must name the word it describes. First, the embeddings of the definition tokens are corrupted with Gaussian noise. Then individual internal states are restored to their values from the clean run, and the authors measure how far the model recovers the correct answer. The states that restore the answer most strongly are grouped by token position, part of speech, and definitional semantic roles (DSR). The DSR taxonomy comes from Silva et al. (2016) and draws on Argument Structure Theory. Experiments were conducted on GPT-J-6B (English) and BERTIN GPT-J-6B (Spanish) with definitions from WordNet and Spanish WordNet. MLP, MHA, and hidden states have different localization profiles, and these profiles are similar in both languages. The most influential states fall disproportionately often on words that specify the distinguishing qualities of the concept. Thus, formal lexical semantics helps explain the model's internal computations better than analysis of individual tokens.

Key points

  • Among the top-50 MLP states with the highest restoration, over 31% fall on the differentia_quality role, while supertype and differentia_event account for only about 6% each.
  • MLP states have a bimodal distribution: they are concentrated in the early and upper layers. In the early layers, they link content tokens to highly specific DSR categories. The MHA contribution is spread across the middle and upper layers and is most concentrated on the last token. Hidden states aggregate information in the upper layers, also on the last token.
  • Localization in the English (GPT-J-6B, 28 layers) and Spanish (BERTIN GPT-J-6B) models is similar. However, DSR annotations exist only for the English dataset.
  • Limitations acknowledged by the authors: only one model scale and single-token prediction were studied. Each component type was analyzed separately, without interactions between them, and MHA was treated as a whole module, without splitting it into individual heads. WordNet definitions are more formulaic than ordinary text.
  • The code and data have been released. The paper is published in the CoNLL 2026 proceedings, pp. 722–741.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Semantic role labeling

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.conll-main.44

@inproceedings{aljaafari2026bridging,
  title = {{Bridging Linguistic Structure and Mechanistic Interpretability for Conceptual Interpretation in Language Models}},
  author = {Aljaafari, Nura and Carvalho, Danilo and Freitas, Andre},
  booktitle = {Proceedings of the 30th Conference on Computational Natural Language Learning},
  year = {2026},
  doi = {10.18653/v1/2026.conll-main.44},
  url = {https://doi.org/10.18653/v1/2026.conll-main.44}
}

References

50 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Arthur Conmy, Augustine Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso. 2023. Towards automated circuit discovery for mechanistic interpretability. Advances in Neural Information Processing Systems, 36:16318–16352. find
  2. Jeff Da, Ronan Le Bras, Ximing Lu, Yejin Choi, and Antoine Bosselut. 2021. Analyzing commonsense emergence in few-shot knowledge models. In Conference on Automated Knowledge Base Construction. find
  3. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8493– 8502, Dublin, Ireland. Association for Computatio… find
  4. Javier De la Rosa, Eduardo G Ponferrada, Paulo Villegas, Pablo Gonzalez de Prado Salas, Manu Romero, and Marıa Grandury. 2022. BERTIN: Efficient pre-training of a spanish language model using perplexity sampling. Procesamiento del Lenguaje Natural, 68(0):13–23. find
  5. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2024. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36. find
  6. Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, et al. 2021. A mathematical framework for transformer circuits. Transformer Circuits Thread, 1:1. find
  7. Javier Ferrando, Gerard I. Gállego, Ioannis Tsiamas, and Marta R. Costa-jussà. 2023. Explaining how transformers use context to build predictions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5486–5513, Toronto, Canada. Ass… find
  8. Michele Bevilacqua, Marco Maru, and Roberto Navigli. 2020. Generationary or “How we went beyond word sense inventories and learned to gloss”. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 7207–7221. Association for Computational Linguistics. find our card
  9. Javier Ferrando and Elena Voita. 2024. Information flow routes: Automatically interpreting language models at scale. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17432–17445, Miami, Florida, USA. Association for Computational Linguistics. find
  10. Keith Brown. 2006. Encyclopedia of language and linguistics (2nd Edition), volume 1. Elsevier. find
  11. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. 2020. The Pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027. find
  12. Lawrence Chan, Adria Garriga-Alonso, Nicholas Goldowsky-Dill, Ryan Greenblatt, Jenny Nitishin-skaya, Ansh Radhakrishnan, Buck Shlegeris, and Nate Thomas. 2022. Causal scrubbing: A method for rigorously testing interpretability hypotheses. In AI Alignment Forum, pages 1828–1843. find
  13. Stephen Clark, Bob Coecke, and Mehrnoosh Sadrzadeh. 2008. A compositional distributional model of meaning. In Proceedings of the Second Quantum Interaction Symposium (QI-2008), pages 133–140. Oxford. find
  14. Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. 2021. Causal abstractions of neural networks. Advances in neural information processing systems, 34:9574–9586. find
  15. Mor Geva, Avi Caciularu, Kevin Wang, and Yoav Goldberg. 2022. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 30–45, Abu Dhabi, United Arab Emirates. Asso… find our card
  16. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484–5495, Online and Punta Cana, Dominican Republic. Association for Computational… find
  17. Aitor Gonzalez-Agirre, Egoitz Laparra, and German Rigau. 2012. Multilingual central repository version 3.0: upgrading a very large lexical knowledge base. In Proceedings of the 6th Global WordNet Conference (GWC 2012), Matsue. find
  18. Stefan Heimersheim and Neel Nanda. 2024. How to use and interpret activation patching. arXiv preprint arXiv:2404.15255. — [arXiv:2404.15255](https://arxiv.org/abs/2404.15255) find
  19. Graeme Hirst. 1987. Semantic interpretation and the resolution of ambiguity. Cambridge University Press. find
  20. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685. find
  21. Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, and Lee Sharkey. 2023. Sparse autoencoders find highly interpretable features in language models. In The Twelfth International Conference on Learning Representations. find
  22. Ray S Jackendoff. 1992. Semantic structures, volume 18. MIT press. find
  23. Javier De la Rosa and Andres Fernández. 2022. Zero-shot reading comprehension and reasoning for spanish with BERTIN GPT-J-6B. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2022). CEUR Workshop Proceedings. find
  24. Beth Levin. 1993. English verb classes and alternations: A preliminary investigation. University of Chicago press. find
  25. Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun Ma, and Jie Yu. 2024. Pmet: Precise model editing in a transformer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18564–18572. find
  26. Zhen Li, Xiting Wang, Weikai Yang, Jing Wu, Zhengyan Zhang, Zhiyuan Liu, Maosong Sun, Hui Zhang, and Shixia Liu. 2022. A unified understanding of deep nlp models for text classification. IEEE Transactions on Visualization and Computer Graphics, 28(12):4980–4994. find
  27. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems, 35:17359–17372. find
  28. Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. 2023. Mass editing memory in a transformer. The Eleventh International Conference on Learning Representations (ICLR). find
  29. George A. Miller. 1994. WordNet: A lexical database for English. In Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994. find our card
  30. Neel Nanda. 2023. Attribution patching: Activation patching at industrial scale. URL: https://www.neel-nanda. io/mechanistic-interpretability/attribution-patching. ↗
  31. Thanapon Noraset, Chen Liang, Larry Birnbaum, and Doug Downey. 2017. Definition modeling: Learning to define word embeddings in natural language. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, pages 3259–3266. find
  32. Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. 2022. In-context learning and induction heads. arXiv preprint arXiv:2209.11895. find
  33. OpenAI. 2023. ChatGPT: Optimizing language models for dialogue. find
  34. Barbara Partee et al. 1984. Compositionality. Varieties of formal semantics, 3:281–311. find
  35. Malka Rappaport Hovav and Beth Levin. 2008. The english dative alternation: The case for verb sensitivityl. Journal of linguistics, 44(1):129–167. find
  36. Dong Shu, Xuansheng Wu, Haiyan Zhao, Daking Rai, Ziyu Yao, Ninghao Liu, and Mengnan Du. 2025. A survey on sparse autoencoders: Interpreting the internal mechanisms of large language models. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 1690–1712, Suzhou, China. Asso… find
  37. Vivian Silva, Siegfried Handschuh, and André Freitas. 2016. Categorization of semantic roles for dictionary definitions. In Proceedings of the 5th Workshop on Cognitive Aspects of the Lexicon (CogALex - V), pages 176–184, Osaka, Japan. find
  38. The COLING 2016 Organizing Committee. find
  39. Paul Smolensky and Géraldine Legendre. 2006. The harmonic mind: From neural computation to optimality-theoretic grammar (Cognitive architecture), Vol. 1. MIT press. find
  40. Xiangchen Song, Jiaqi Sun, Zijian Li, Yujia Zheng, and Kun Zhang. 2025. LLM interpretability with identifiable temporal-instantaneous representation. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  41. Kriz Tahimic and Charibeth Cheng. 2025. Mechanistic interpretability of code correctness in llms via sparse autoencoders. arXiv preprint arXiv:2510.02917. — [arXiv:2510.02917](https://arxiv.org/abs/2510.02917) find
  42. Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. BERT rediscovers the classical NLP pipeline. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4593– 4601. Association for Computational Linguistics. find our card
  43. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30. find
  44. Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020. Investigating gender bias in language models using causal mediation analysis. In Advances in Neural Information Processing Systems, volume 33, pages 12388–12401. find
  45. Ben Wang. 2021. Mesh-Transformer-JAX: Model-Parallel Implementation of Transformer Language Model with JAX. https://github.com/ kingoflolz/mesh-transformer-jax. ↗
  46. Ben Wang and Aran Komatsuzaki. 2021. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. https://github.com/kingoflolz/ mesh-transformer-jax. ↗
  47. Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2022. Interpretability in the wild: a circuit for indirect object identification in gpt-2 small. arXiv preprint arXiv:2211.00593. find
  48. Guangzhi Xiong, Zhenghao He, Bohan Liu, Sanchit Sinha, and Aidong Zhang. 2026. Toward faithful retrieval-augmented generation with sparse autoencoders. In The Fourteenth International Conference on Learning Representations. find
  49. Ningyu Xu, Qi Zhang, Menghan Zhang, Peng Qian, and Xuanjing Huang. 2024. On the tip of the tongue: Analyzing conceptual representation in large language models with reverse-dictionary probe. arXiv preprint arXiv:2402.14404. — [arXiv:2402.14404](https://arxiv.org/abs/2402.14404) find
  50. A compute budget. This section documents dataset construction, filtering, prompt selection, and trace aggregation. Dataset Creation, Filtering, and Prompt Selection Three eligibility conditions governed dataset and model choices. First, the task must elicit compositional semantic processing with a v… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/aljaafari-2026-bridging-linguistic-structure-mechanistic-interpretability/