Elamur Research⌕
☰
home / papers / 2026 / The Quest for the Right Mediator: Surveying Mechanistic Interpretability for NLP Through the Lens of Causal Mediation Analysis

The Quest for the Right Mediator: Surveying Mechanistic Interpretability for NLP Through the Lens of Causal Mediation Analysis

Paper · 2026 · Aaron Mueller, Jannik Brinkmann, Millicent Li et al.

Summary

Research on mechanistic interpretability (MI) of language models is fragmented. Studies use bespoke evaluations, lack a shared theoretical basis, and rarely define the elementary causal units that make up a "mechanism." This survey, published in Computational Linguistics, treats MI as recovering causal graphs that show how a model's internal computations carry the influence of inputs through to outputs. It grounds this framing in causal mediation analysis and counterfactual interventions. Rather than sorting prior work by technique, the authors classify it by mediator type: single neurons, attention heads, whole layers or submodules, and directions not aligned with the neuron basis. They also classify the strategies used to search over these mediators. For each mediator they discuss granularity, whether it acts as a causal bottleneck, and whether it is native to the model or learned by a separate module. They also trace MI's history back to backpropagation. The outcome is a more unified picture of the field, goal-based guidance for choosing methods, and an agenda calling for new mediators and standardized evaluations.

Key points

  • A mediator's causal contribution is quantified as its indirect effect (IE). The framework builds on Lewis's counterfactual theory of causation and Pearl's mediation analysis.
  • MI and causal interpretability overlap only partly. Sparse autoencoders are correlational yet widely used in MI, while LIME intervenes on inputs rather than model internals.
  • Recommendations depend on the research goal:
    • Explaining behavior without a prior hypothesis: unsupervised search over non-basis-aligned directions.
    • Fixing specific errors: exhaustive search over coarse units (layers or submodules), combined with editors such as ROME or MEMIT.
    • Testing a concrete hypothesis (e.g., "truthfulness"): supervised probing, or counterfactual optimization over subspaces via alignment search with a learned rotation.
  • Basis-aligned units (neurons, attention heads) are often a poor starting point for realistic large models. They remain useful in toy settings or when sparse autoencoders cannot be trained.
  • The authors cite AlphaEdit, which updates the null space of MLP layers, as outperforming ROME and MEMIT on several benchmarks.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Stanford UniversityStanford · United States434 works · 314,419 citations🎓Massachusetts Institute of TechnologyCambridge · United States211 works · 69,000 citations🎓Technion – Israel Institute of TechnologyHaifa · Israel49 works · 5,306 citations🎓Northeastern UniversityBoston · United States57 works · 6,533 citations🎓University of MannheimMannheim · Germany29 works · 8,347 citations🎓Boston UniversityBoston · United States26 works · 4,022 citations🏢AnthropicSan Francisco · United States6 works · 1,536 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Interpretability · 130 works

Cite

Cite as Aaron Mueller, Jannik Brinkmann, Millicent Li, Samuel Marks, Koyena Pal, Nikhil Prakash, Can Rager, Aruna Sankaranarayanan, Arnab Sen Sharma, Jiuding Sun, Eric Todd, David Bau and Yonatan Belinkov. 2026. The Quest for the Right Mediator: Surveying Mechanistic Interpretability for NLP Through the Lens of Causal Mediation Analysis. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli.a.572

@article{mueller2026quest,
  title = {{The Quest for the Right Mediator: Surveying Mechanistic Interpretability for NLP Through the Lens of Causal Mediation Analysis}},
  author = {Aaron Mueller, Jannik Brinkmann, Millicent Li, Samuel Marks, Koyena Pal, Nikhil Prakash, Can Rager, Aruna Sankaranarayanan, Arnab Sen Sharma, Jiuding Sun, Eric Todd, David Bau and Yonatan Belinkov},
  journal = {Computational Linguistics},
  year = {2026},
  doi = {10.1162/coli.a.572},
  url = {https://doi.org/10.1162/coli.a.572}
}

References

125 references, 36 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Amini, Afra, Tiago Pimentel, Clara Meister, and Ryan Cotterell. 2023. Naturalistic causal probing for morpho-syntax. Transactions of the Association for Computational Linguistics, 11:384–403. https://doi.org/10.1162/tacl_a_00554 ↗
  2. Antverg, Omer and Yonatan Belinkov. 2022. On the pitfalls of analyzing individual neurons in language models. In International Conference on Learning Representations. find
  3. Arora, Aryaman, Dan Jurafsky, and Christopher Potts. 2024. CausalGym: Benchmarking causal interpretability methods on linguistic tasks. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14638–14663. https://doi.org/10.18653/v1/2… ↗
  4. Bastings, Jasmijn, Wilker Aziz, and Ivan Titov. 2019. Interpretable neural predictions with differentiable binary variables. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2963–2977. https://doi.org/10.18653/v1/P19-1284 Bau, Anthony, Yonatan Belink… ↗
  5. Bau, David. 2022. Baukit. https://github.com/davidbau/baukit Bau, David, Jun-Yan Zhu, Hendrik Strobelt, Agata Lapedriza, Bolei Zhou, and Antonio Torralba. 2020. Understanding the role of individual units in a deep neural network. Proceedings of the National Academy of Sciences, 117(48):30071–30078.… — [doi:10.1073/pnas.1907375117](https://doi.org/10.1073/pnas.1907375117) ↗
  6. Belinkov, Yonatan. 2022. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):207–219. https://doi.org/10.1162/coli_a_00422 ↗
  7. Belinkov, Yonatan, Nadir Durrani, Fahim Dalvi, Hassan Sajjad, and James Glass. 2017. What do neural machine translation models learn about morphology? In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 861–872. https://doi.org/10…
  8. Belinkov, Yonatan and James Glass. 2019. Analysis methods in neural language processing: A survey. Transactions of the Association for Computational Linguistics, 7:49–72. https://doi.org/10 .1162/tacl a 00254 Belrose, Nora, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and St…
  9. Bengio, Yoshua, Aaron Courville, and Pascal Vincent. 2013. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8):1798–1828. https://doi.org/10.1109/TPAMI.2013.50, PubMed: 23787338 Bereska, Leonard and Stratis Gavves. 2024. Mech… ↗
  10. Bills, Steven, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. 2023. Language models can explain neurons in language models. find
  11. OpenAI Blog. Boz, Olcay. 2002. Extracting decision trees from trained neural networks. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’02, pages 456–461. https://doi.org/10 .1145/775047.775113 Braun, Dan, Jordan Taylor, Nicholas Goldowsky… — [doi:10.52202/079017-3408](https://doi.org/10.52202/079017-3408) find
  12. Bricken, Trenton, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, et al. 2023. Towards monosemanticity: Decomposing language models with dictionary learning. find
  13. Transformer Circuits Thread. Brinkmann, Jannik, Abhay Sheshadri, Victor Levoso, Paul Swoboda, and Christian Bartelt. 2024. A mechanistic analysis of a transformer trained on a symbolic multi-step reasoning task. arXiv preprint arXiv:2402.11917. https://doi.org/10 .18653/v1/2024.findings-acl.242 Brow… find
  14. Cao, Steven, Victor Sanh, and Alexander Rush. 2021. Low-complexity probing via finding subnetworks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 960–966. https://doi.org/10 .18653/v1/2021.naac… find
  15. AI Alignment Forum. Chen, Angelica, Ravid Shwartz-Ziv, Kyunghyun Cho, Matthew L. Leavitt, and Naomi Saphra. 2024a. Sudden drops in the loss: Syntax acquisition, phase transitions, and simplicity bias in MLMs. In The Twelfth International Conference on Learning Representations. find
  16. Chen, Yida, Aoyu Wu, Trevor DePodesta, Catherine Yeh, Kenneth Li, Nicholas Castillo Marin, Oam Patel, Jan Riecke, Shivam Raval, Olivia Seow, et al. 2024b. Designing a dashboard for transparency and control of conversational AI. arXiv preprint arXiv:2406.07882. — [arXiv:2406.07882](https://arxiv.org/abs/2406.07882) find
  17. Chughtai, Bilal, Lawrence Chan, and Neel Nanda. 2023. A toy model of universality: Reverse engineering how networks learn group operations. In Proceedings of the 40th International Conference on Machine Learning. find
  18. Cohen, Roi, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. 2024. Evaluating the ripple effects of knowledge editing in language models. Transactions of the Association for Computational Linguistics, 12:283–298. https://doi.org/10.1162/tacl a 00644 Conmy, Arthur, Augustine Mavor-Parker, Aengus… ↗
  19. Craven, Mark W. and Jude W. Shavlik. 1994. Using sampling and queries to extract rules from trained neural networks. In Machine Learning Proceedings 1994, pages 37–45. https://doi.org/10.1016/B978-1-55860-335-6.50013-1 Csordás, Róbert, Christopher Potts, Christopher D. Manning, and Atticus Geiger… — [doi:10.18653/v1/2024](https://doi.org/10.18653/v1/2024) ↗
  20. Dabkowski, Piotr and Yarin Gal. 2017. Real time image saliency for black box classifiers. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pages 6970–6979. find
  21. Dalvi, Fahim, Hassan Sajjad, and Nadir Durrani. 2023. NeuroX library for neuron analysis of deep NLP models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 226–234. https://doi.org/10 .18653/v1/2023.acl-demo.21 Dalv… find
  22. De Cao, Nicola, Michael Sejr Schlichtkrull, Wilker Aziz, and Ivan Titov. 2020. How do decisions emerge across layers in neural models? Interpretation with differentiable masking. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3243–3255. https…
  23. Elazar, Yanai, Shauli Ravfogel, Alon Jacovi, and Yoav Goldberg. 2021. Amnesic Probing: Behavioral explanation with amnesic counterfactuals. Transactions of the Association for Computational Linguistics, 9:160–175. https://doi.org/10.1162/tacl a 00359 Elhage, Nelson, Tristan Hume, Catherine Olsson,… ↗
  24. Transformer Circuits Thread. Elhage, Nelson, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, et al. 2022b. Toy models of superposition. find
  25. Transformer Circuits Thread. Elhage, Nelson, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, et al. 2021. A mathematical framework for transformer circuits. find
  26. Transformer Circuits Thread. Elman, Jeffrey L. 1989. Representation and structure in connectionist models. University of California, San Diego, Center for Research in Language. https://doi.org/10.21236/ADA259504 Elman, Jeffrey L. 1990. Finding structure in time. Cognitive Science, 14:179–211. https… — [doi:10.21236/ada259504](https://doi.org/10.21236/ada259504) ↗
  27. Erhan, Dumitru, Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2009. Visualizing higher-layer features of a deep network. Technical Report 1341, University of Montreal. find
  28. Fang, Junfeng, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi, Xiang Wang, Xiangnan He, and Tat-Seng Chua. 2025. find
  29. AlphaEdit: Null-space constrained model editing for language models. In The Thirteenth International Conference on Learning Representations. Feder, Amir, Katherine A. Keith, Emaad Manzoor, Reid Pryzant, Dhanya Sridhar, Zach Wood-Doughty, Jacob Eisenstein, Justin Grimmer, Roi Reichart, Margaret E. Ro… find
  30. Fel, Thomas, Ekdeep Singh Lubana, Jacob S. Prince, Matthew Kowal, Victor Boutin, Isabel Papadimitriou, Binxu Wang, Martin Wattenberg, Demba Ba, and Talia Konkle. 2025. Archetypal SAE: Adaptive and stable dictionary learning for concept extraction in large vision models. arXiv:2502.12892. Ferrando, J… find
  31. Feucht, Sheridan, Eric Todd, Byron Wallace, and David Bau. 2025. The dual-route model of induction. In Second Conference on Language Modeling. find
  32. Finlayson, Matthew, Aaron Mueller, Sebastian Gehrmann, Stuart Shieber, Tal Linzen, and Yonatan Belinkov. 2021. Causal analysis of syntactic agreement mechanisms in neural language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Internat… find
  33. Geiger, Atticus, Hanson Lu, Thomas F. Icard, and Christopher Potts. 2021. Causal abstractions of neural networks. In Advances in Neural Information Processing Systems, volume 34, pages 9574–9586. find
  34. Geiger, Atticus, Zhengxuan Wu, Christopher Potts, Thomas Icard, and Noah Goodman. 2024. Finding alignments between interpretable causal variables and distributed neural representations. In Causal Learning and Reasoning, pages 160–187. find
  35. Geva, Mor, Jasmijn Bastings, Katja Filippova, and Amir Globerson. 2023. Dissecting recall of factual associations in auto-regressive language models. In The 2023 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10.18653/v1/2023.emnlp-main .751 Ghorbani, Amirata and Ja… ↗
  36. Gilpin, Leilani H., David Bau, Ben Z. Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. 2018. Explaining explanations: An overview of interpretability of machine learning. In 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), pages 80–89. https://doi.org/10.1… find
  37. Goodfellow, Ian, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems, volume 27, pages 1–9. find
  38. Goyal, Yash, Amir Feder, Uri Shalit, and Been Kim. 2020. Explaining classifiers with causal concept effect (CaCE). arXiv preprint arXiv:1907.07165. — [arXiv:1907.07165](https://arxiv.org/abs/1907.07165) find
  39. Gu, Albert and Tri Dao. 2024. Mamba: Linear-time sequence modeling with selective state spaces. In First Conference on Language Modeling. find
  40. Gu, Albert, Karan Goel, and Christopher Re. 2022. Efficiently modeling long sequences with structured state spaces. In International Conference on Learning Representations. find
  41. Guidotti, Riccardo, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A survey of methods for explaining black box models. ACM Computing Surveys, 51(5). https://doi.org/10.1145/3236009 Gupta, Abhijeet, Gemma Boleda, Marco Baroni, and Sebastian Padó. 2015.… ↗
  42. Hanna, Michael, Ollie Liu, and Alexandre Variengien. 2023. How does GPT-2 compute greater-than? Interpreting mathematical abilities in a pre-trained language model. In Advances in Neural Information Processing Systems, volume 36, pages 76033–76060.
  43. Hanna, Michael, Sandro Pezzelle, and Yonatan Belinkov. 2024. Have faith in faithfulness: Going beyond circuit overlap when finding model mechanisms. In ICML 2024 Workshop on Mechanistic Interpretability. find
  44. Hase, Peter, Mohit Bansal, Been Kim, and Asma Ghandeharioun. 2023. Does localization inform editing? Surprising differences in causality-based localization vs. knowledge editing in language models. In Thirty-seventh Conference on Neural Information Processing Systems. find
  45. Hayashi, Yoichi. 1990. A neural expert system with automated extraction of fuzzy if-then rules and its application to medical diagnosis. In Advances in Neural Information Processing Systems, volume 3. find
  46. He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778. https://doi.org/10.1109/CVPR.2016.90 ↗
  47. Hewitt, John, Kawin Ethayarajh, Percy Liang, and Christopher Manning. 2021. Conditional probing: Measuring usable information beyond a baseline. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1626–1639. https://doi.org/10.18653/v1/2021.emnlp-main .1… ↗
  48. Hewitt, John, John Thickstun, Christopher Manning, and Percy Liang. 2023. Backpack language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9103–9125. https://doi.org/10.18653/v1/2023.acl-long.506 Hinton, G. E., J. L.… ↗
  49. Hu, Xinyan, Kayo Yin, Michael I. Jordan, Jacob Steinhardt, and Lijie Chen. 2025. Understanding in-context learning of addition via activation subspaces. arXiv:2505.05145. find
  50. Huang, Jing, Atticus Geiger, Karel D’Oosterlinck, Zhengxuan Wu, and Christopher Potts. 2023. Rigorously assessing natural language explanations of neurons. In Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 317–331. https://doi.org/10 .18653/v1/…
  51. Huang, Jing, Zhengxuan Wu, Christopher Potts, Mor Geva, and Atticus Geiger. 2024. RAVEL: Evaluating interpretability methods on disentangling language model representations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8669… find
  52. Iskander, Shadi, Kira Radinsky, and Yonatan Belinkov. 2023. Shielded representations: Protecting sensitive attributes through iterative gradient-based projection. In Findings of the Association for Computational Linguistics: ACL 2023, pages 5961–5977. https://doi.org/10.18653/v1/2023 .findings-acl.3… ↗
  53. Jermyn, Adam, Chris Olah, and Tom Henighan. 2023. Attention head superposition. find
  54. Transformer Circuits Thread. Joseph, Sonia. 2023. ViT Prisma: A mechanistic interpretability library for vision transformers. https://github.com/soniajoseph/vit-prisma Kaplan, Jared, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and D… ↗
  55. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361. Karnin, E. D. 1990. A simple procedure for pruning back-propagation trained neural networks. IEEE Transactions on Neural Networks, 1(2):239–242. https://doi.org/10.1109/72.80236, PubMed: 18282841 Karpathy, Andrej, Justin John… ↗
  56. Karvonen, Adam, Can Rager, Johnny Lin, Curt Tigges, Joseph Bloom, David Chanin, Yeu-Tong Lau, Eoin Farrell, Callum McDougall, Kola Ayonrinde, et al. 2025. SAEBench: A comprehensive benchmark for sparse autoencoders in language model interpretability. arXiv:2503.09532. — [arXiv:2503.09532](https://arxiv.org/abs/2503.09532) find
  57. Karvonen, Adam, Benjamin Wright, Can Rager, Rico Angell, Jannik Brinkmann, Logan Riggs Smith, Claudio Mayrink Verdun, David Bau, and Samuel Marks. 2024. Measuring progress in dictionary learning for language model interpretability with board game models. In ICML 2024 Workshop on Mechanistic Interpre… find
  58. Koh, Pang Wei and Percy Liang. 2017. Understanding black-box predictions via influence functions. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1885–1894.
  59. Koh, Pang Wei, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. 2020. Concept bottleneck models. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 5338–5348. Köhn, Arne. 2015.… find
  60. Krishnan, R., G. Sivakumar, and P. Bhattacharya. 1999. Extracting decision trees from trained neural networks. find
  61. Pattern Recognition, 32(12):1999–2009. https://doi.org/10.1016/S0031 -3203(98)00181-2 Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, volume 25. ↗
  62. Lad, Vedang, Wes Gurnee, and Max Tegmark. 2024. The remarkable robustness of LLMs: Stages of inference? In ICML 2024 Workshop on Mechanistic Interpretability. Lakretz, Yair, Théo Desbordes, Jean-Rémi King, Benoı̂t Crabbé, Maxime Oquab, and Stanislas Dehaene. 2021. Can RNNs learn recursive nested… — [arXiv:2101.02258](https://arxiv.org/abs/2101.02258) find
  63. Lakretz, Yair, German Kruszewski, Theo Desbordes, Dieuwke Hupkes, Stanislas Dehaene, and Marco Baroni. 2019. The emergence of number and syntax units in LSTM language models. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human L…
  64. Lewis, David. 1986. Causation. In Philosophical Papers II. Oxford University Press, pages 159–213. https://doi.org/10.1093/0195036468.003.0006 Lewis, David. 2000. Causation as influence. Journal of Philosophy, 97(4):182–197. https://doi.org/10.2307/2678389 — [doi:10.1093/0195036468.003.0006](https://doi.org/10.1093/0195036468.003.0006) ↗
  65. Counterfactuals. Blackwell. Li, Jiwei, Will Monroe, and Dan Jurafsky. 2017. Understanding neural networks through representation erasure. arXiv:1612.08220. Li, Kenneth, Aspen K Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2023. Emergent world representations: Explo… find
  66. Li, Maximilian, Xander Davies, and Max Nadeau. 2024. Circuit breaking: Removing model behaviors with targeted ablation. arXiv preprint arXiv:2309.05973. — [arXiv:2309.05973](https://arxiv.org/abs/2309.05973) find
  67. Li, Victoria R., Jenny Kaufmann, Martin Wattenberg, David Alvarez-Melis, and Naomi Saphra. 2025. Can interpretation predict behavior on unseen data? arXiv preprint 2507.06445. — [arXiv:2507.06445](https://arxiv.org/abs/2507.06445) find
  68. Lipton, Zachary C. 2018. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue, 16(3):31–57. https://doi.org/10.1145/3236386.3241340 Liu, Nelson F., Matt Gardner, Yonatan Belinkov, Matthew E. Peters, and Noah A. Smith. 2019a… ↗
  69. Lundberg, Scott M. and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, pages 4768–4777.
  70. Lyu, Qing, Marianna Apidianaki, and Chris Callison-Burch. 2024. Towards faithful model explanation in NLP: A survey. Computational Linguistics, 50(2):657–723. https://doi.org/10.1162/coli_a_00511 ↗
  71. Ma, Weicheng, Kai Zhang, Renze Lou, Lili Wang, and Soroush Vosoughi. 2021. Contributions of transformer attention heads in multi- and cross-lingual tasks. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural… find
  72. Marks, Samuel, Can Rager, Eric J. Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. 2025. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models. In The Thirteenth International Conference on Learning Representations. find
  73. Marks, Samuel and Max Tegmark. 2023. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. arXiv preprint arXiv:2310.06824.
  74. McClelland, James L. and David E. Rumelhart. 1985. Distributed memory and the representation of general and specific information. Journal of Experimental Psychology. General, 114(2):159–97. https://doi.org/10.1037/0096 -3445.114.2.159, PubMed: 3159828 McDougall, Callum Stuart, Arthur Conmy, Cody Rus… ↗
  75. Meng, Kevin, David Bau, Alex J. Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in GPT. In Advances in Neural Information Processing Systems. find
  76. Meng, Kevin, Arnab Sen Sharma, Alex J. Andonian, Yonatan Belinkov, and David Bau. 2023. Mass-editing memory in a transformer. In The Eleventh International Conference on Learning Representations. find
  77. Merullo, Jack, Carsten Eickhoff, and Ellie Pavlick. 2024a. Circuit component reuse across tasks in transformer language models. In The Twelfth International Conference on Learning Representations. find
  78. Merullo, Jack, Carsten Eickhoff, and Ellie Pavlick. 2024b. Talking heads: Understanding inter-layer communication in transformer language models. arXiv preprint arXiv:2406.09519. find
  79. Michaud, Eric J., Ziming Liu, Uzay Girit, and Max Tegmark. 2023. The quantization model of neural scaling. In Thirty-seventh Conference on Neural Information Processing Systems.
  80. Mikolov, Tomas, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. Mikolov, Tomas, Martin Karafiát, Lukáš Burget, Jan Cernocký, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model.…
  81. Miller, Joseph, Bilal Chughtai, and William Saunders. 2024. Transformer circuit evaluation metrics are not robust. In First Conference on Language Modeling. find
  82. Mohebbi, Hosein, Jaap Jumelet, Michael Hanna, Afra Alishahi, and Willem Zuidema. 2024. Transformer-specific interpretability. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: Tutorial Abstracts, pages 21–26. https://doi.org/10.18653/v1/2… ↗
  83. Mozer, Michael C. and Paul Smolensky. 1988. Skeletonization: A technique for trimming the fat from a network via relevance assessment. In Advances in Neural Information Processing Systems, volume 1. find
  84. Mueller, Aaron. 2024. Missed causes and ambiguous effects: Counterfactuals pose challenges for interpreting neural networks. In ICML 2024 Workshop on Mechanistic Interpretability. Mueller, Aaron, Atticus Geiger, Sarah Wiegreffe, Dana Arad, Iván Arcuschin, Adam Belfki, Yik Siu Chan, Jaden Fiotto-Kau… find
  85. Mueller, Aaron, Yu Xia, and Tal Linzen. 2022. Causal analysis of syntactic agreement neurons in multilingual language models. In Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL), pages 95–109. https://doi.org/10.18653/v1/2022.conll-1.8 Nanda, Neel and Joseph Blo… ↗
  86. Oikarinen, Tuomas, Subhro Das, Lam M. Nguyen, and Tsui-Wei Weng. 2023. Label-free concept bottleneck models. In The Eleventh International Conference on Learning Representations. find
  87. Olsson, Catherine, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. 2022. In-context learning and induction heads. Transformer Circuits Thread.
  88. Panickssery, Nina, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. 2024. Steering Llama 2 via contrastive activation addition. arXiv preprint arXiv:2312.06681. find
  89. Park, Kiho, Yo Joong Choe, and Victor Veitch. 2023. The linear representation hypothesis and the geometry of large language models. In Causal Representation Learning Workshop at NeurIPS 2023. Paulo, Gonçalo, Alex Mallen, Caden Juang, and Nora Belrose. 2024. Automatically interpreting millions of fe…
  90. Pearl, Judea. 2000. Causality: Models, Reasoning, and Inference. find
  91. Cambridge University Press. Pearl, Judea. 2001. Direct and indirect effects. In Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, pages 411–420. find
  92. Pearl, Judea. 2009. Causality, 2nd edition. find
  93. Cambridge University Press. Pennington, Jeffrey, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. https://doi.org/10.3115/v1/D14 -1162 Prak… ↗
  94. Qiu, Linlu, Liwei Jiang, Ximing Lu, Melanie Sclar, Valentina Pyatkin, Chandra Bhagavatula, Bailin Wang, Yoon Kim, Yejin Choi, Nouha Dziri, and Xiang Ren. 2024. Phenomenal yet puzzling: Testing inductive reasoning capabilities of language models with hypothesis refinement. In The Twelfth Internationa… find
  95. Radford, Alec, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. https://cdn.openai.com/better-language-models/language models are unsupervised multitask learners.pdf Rai, Daking, Yilun Zhou, Shi Feng, Abulhair Saparov, a… ↗
  96. Rajamanoharan, Senthooran, Arthur Conmy, Lewis Smith, Tom Lieberum, Vikrant Varma, János Kramár, Rohin Shah, and Neel Nanda. 2024. Improving dictionary learning with gated sparse autoencoders. arXiv:2404.16014.
  97. Ravfogel, Shauli, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. 2020. Null it out: Guarding protected attributes by iterative nullspace projection. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7237–7256. https://doi.org/10 .18653/v1…
  98. Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. 2016b. “Why should I trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, pages 1135–1144. https://doi.org /10.1145/2939672.2… ↗
  99. Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. 2018. Anchors: High-precision model-agnostic explanations. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on… — [doi:10.1609/aaai](https://doi.org/10.1609/aaai) find
  100. Sajjad, Hassan, Fahim Dalvi, Nadir Durrani, and Preslav Nakov. 2023. On the effect of dropping layers of pre-trained transformer models. Computer Speech & Language, 77:101429. https://doi.org/10.1016/j.csl.2022.101429 Sajjad, Hassan, Nadir Durrani, and Fahim Dalvi. 2022. Neuron-level interpretation… ↗
  101. Sharkey, Lee, Dan Braun, and Beren Millidge. 2023. Taking features out of superposition with sparse autoencoders. Accessed: 2023-05-10. find
  102. Sharma, Arnab Sen, David Atkinson, and David Bau. 2024. Locating and editing factual associations in Mamba. arXiv preprint arXiv:2404.03646.
  103. Shi, Xing, Inkit Padhi, and Kevin Knight. 2016. Does string-based neural MT learn source syntax? In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1526–1534. https://doi.org/10.18653/v1/D16-1159 ↗
  104. Simonyan, Karen, Andrea Vedaldi, and Andrew Zisserman. 2014. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034.
  105. Singh, Chandan, Jeevana Priya Inala, Michel Galley, Rich Caruana, and Jianfeng Gao. 2024a. Rethinking interpretability in the era of large language models. arXiv preprint arXiv:2402.01761.
  106. Singh, Shashwat, Shauli Ravfogel, Jonathan Herzig, Roee Aharoni, Ryan Cotterell, and Ponnurangam Kumaraguru. 2024b. Representation surgery: Theory and practice of affine steering. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. find
  107. Srivastava, Divyansh, Tuomas Oikarinen, and Tsui-Wei Weng. 2023. Corrupting neuron explanations of deep visual features. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1877–1886. https://doi.org/10 .1109/ICCV51070.2023.00180 Strobelt, Hendrik, Sebastian Gehrmann, H… find
  108. Subramani, Nishant, Nivedita Suresh, and Matthew Peters. 2022. Extracting latent steering vectors from pretrained language models. In Findings of the Association for Computational Linguistics: ACL 2022, pages 566–581. https://doi.org/10 .18653/v1/2022.findings-acl.48 Sun, Jiuding, Jing Huang, Sidhar…
  109. Sundararajan, Mukund, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, pages 3319–3328. find
  110. Sutter, Denis, Julian Minder, Thomas Hofmann, and Tiago Pimentel. 2025. The non-linear representation dilemma: Is causal abstraction enough for mechanistic interpretability? arXiv:2507.08802. — [arXiv:2507.08802](https://arxiv.org/abs/2507.08802) find
  111. Syed, Aaquib, Can Rager, and Arthur Conmy. 2023. Attribution patching outperforms automated circuit discovery. In NeurIPS Workshop on Attributing Model Behavior at Scale. https://doi.org/10 .18653/v1/2024.blackboxnlp-1.25 Tamkin, Alex, Mohammad Taufeeque, and Noah D. Goodman. 2023. Codebook features… find
  112. Templeton, Adly, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, et al. 2024. Scaling monosemanticity: Extracting interpretable features from Claude 3 sonnet. find
  113. Transformer Circuits Thread. Theodosis, Emmanouil and Demba Ba. 2023. Learning silhouettes with group sparse autoencoders. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. https://doi.org/10.1109/ICASSP49357 .2023.10095958 Tigges, Cu… ↗
  114. Todd, Eric, Millicent Li, Arnab Sen Sharma, Aaron Mueller, Byron C. Wallace, and David Bau. 2024. Function vectors in large language models. In The Twelfth International Conference on Learning Representations. find
  115. Tucker, Mycal, Peng Qian, and Roger Levy. 2021. What if this modified that? Syntactic interventions with counterfactual embeddings. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 862–875. https://doi.org/10.18653/v1/2021.findings-acl.76 Turner, Alexander Matt,… ↗
  116. Tzeng, F. Y. and K.-L. Ma. 2005. Opening the black box: Data driven visualization of neural networks. In VIS 05. find
  117. IEEE Visualization, 2005, pages 383–390. find
  118. Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30.
  119. Vig, Jesse, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020. Investigating gender bias in language models using causal mediation analysis. In Advances in Neural Information Processing Systems, volume 33, pages 12388–12401. find
  120. Wang, Kevin Ro, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2023. Interpretability in the wild: A circuit for indirect object identification in GPT-2 small. In The Eleventh International Conference on Learning Representations. White, Adam and Artur d’Avila Garcez. 2020.…
  121. Wu, Xing, Shaoqi Peng, Jingwen Li, Jian Zhang, Qun Sun, Weimin Li, Quan Qian, Yue Liu, and Yike Guo. 2024b. Causal inference in the medical domain: A survey. Applied Intelligence, pages 1–24. find
  122. Wu, Zhengxuan, Aryaman Arora, Atticus Geiger, Zheng Wang, Jing Huang, Dan Jurafsky, Christopher D. Manning, and Christopher Potts. 2025. AxBench: Steering LLMs? Even simple baselines outperform sparse autoencoders. arXiv:2501.17148. find
  123. Wu, Zhengxuan, Aryaman Arora, Zheng Wang, Atticus Geiger, Dan Jurafsky, Christopher D. Manning, and Christopher Potts. 2024c. ReFT: Representation finetuning for language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. find
  124. Wu, Zhengxuan, Atticus Geiger, Aryaman Arora, Jing Huang, Zheng Wang, Noah Goodman, Christopher Manning, and Christopher Potts. 2024d. pyvene: A library for understanding and improving PyTorch models via interventions. In Proceedings of the 2024 Conference of the North American Chapter of the Associ… find
  125. Zeiler, Matthew D. and Rob Fergus. 2014. Visualizing and understanding convolutional networks. In Computer Vision – ECCV 2014, pages 818–833. https://doi.org/10.1007/978-3-319 -10590-1 53 Zhong, Zexuan, Zhengxuan Wu, Christopher D. Manning, Christopher Potts, and Danqi Chen. 2023. MQuAKE: Assessing… ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mueller-2026-quest-right-mediator-surveying-mechanistic/