Elamur Research⌕
☰
home / papers / 2026 / Mechanistic Interpretability Should Prioritize Feature Consistency in Sparse Autoencoders

Mechanistic Interpretability Should Prioritize Feature Consistency in Sparse Autoencoders

Paper · 2026 · Xiangchen Song, Aashiq Muhamed, Yujia Zheng et al.

Summary

Sparse autoencoders (SAEs) are a popular way to break LLM activations into sparse, human-readable features. However, two SAEs trained on the same data with identical settings but different random seeds can produce noticeably different feature dictionaries. This position paper argues that seed-to-seed stability should be measured and reported routinely, alongside reconstruction quality and sparsity.

The authors treat two dictionaries as equivalent when they differ only by permutation and rescaling of features. To measure this, they propose PW-MCC, which optimally matches decoder columns across two runs and averages their absolute cosine similarities. They support the proposal in three ways:

  • a theoretical link between idealized TopK SAEs and identifiability results for overcomplete sparse dictionary learning;
  • a synthetic model organism showing that PW-MCC tracks recovery of the true dictionary;
  • experiments on LLM activations comparing several SAE variants.

TopK SAEs turn out to be highly consistent, and better-aligned features also receive more similar automatic explanations. This matters because steering, ablation, unlearning and shared foundation SAEs all assume features are reproducible.

Key points

  • PW-MCC is computed via the Hungarian algorithm. On Gemma-2-2B with d_sae = 65,536, one computation took a few hours offline on a multi-core CPU.
  • In the synthetic setting with matched capacity, TopK SAEs recover the ground-truth dictionary with GT-MCC ≈ 0.97, and PW-MCC reliably tracks this score.
  • On LLM activations, TopK SAEs reach PW-MCC ≈ 0.80. Earlier reports had found cross-seed feature overlap as low as about 30%.
  • Matched feature pairs were binned by cosine similarity and their explanations were scored for similarity by an LLM (GPT Score). Higher similarity strongly predicted more similar explanations, and consistency also varied with feature frequency.
  • Code is available at github.com/xiangchensong/sae-feature-consistency. The authors stress that consistency does not guarantee faithfulness, and that variation due to data, domain or base-model changes was not studied.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Interpretability · 130 works

Cite

Cite as Xiangchen Song, Aashiq Muhamed, Yujia Zheng, Lingjing Kong, Zeyu Tang, Mona T. Diab, Virginia Smith and Kun Zhang. 2026. Mechanistic Interpretability Should Prioritize Feature Consistency in Sparse Autoencoders. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.99

@inproceedings{song2026mechanistic,
  title = {{Mechanistic Interpretability Should Prioritize Feature Consistency in Sparse Autoencoders}},
  author = {Xiangchen Song, Aashiq Muhamed, Yujia Zheng, Lingjing Kong, Zeyu Tang, Mona T. Diab, Virginia Smith and Kun Zhang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.99},
  url = {https://doi.org/10.18653/v1/2026.acl-long.99}
}

References

56 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Samir Abdaljalil, Filippo Pallucchini, Andrea Seveso, Hasan Kurban, Fabio Mercorio, and Erchin Serpedin. 2025. Safe: A sparse autoencoder-based framework for robust query enrichment and hallucination mitigation in llms. arXiv preprint arXiv:2503.03032. find
  2. Emmanuel Ameisen, Jack Lindsey, Adam Pearce, Wes Gurnee, Nicholas L. Turner, Brian Chen, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, and 8 others… find
  3. Sanjeev Arora, Rong Ge, and Ankur Moitra. 2014. New algorithms for learning incoherent and overcomplete dictionaries. In Conference on Learning Theory, pages 779–806. PMLR. find
  4. Yonatan Belinkov. 2022. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):207–219.
  5. Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, and 1 others. 2023. Pythia: A suite for analyzing large language models across training and scaling. In Proceedings of the 40th… find
  6. Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nicholas L Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Alex Tamkin, Karina Nguyen, Brayden McLean, and 5 others. 2023a.… find
  7. Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nicholas L. Turner, Cem Anil, Carson Denison, Amanda Askell, and 1 others. 2023b. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread. find
  8. Bart Bussmann, Patrick Leask, and Neel Nanda. 2024. Batchtopk sparse autoencoders. arXiv preprint arXiv:2412.06410. — [arXiv:2412.06410](https://arxiv.org/abs/2412.06410) find
  9. Bart Bussmann, Noa Nabeshima, Adam Karvonen, and Neel Nanda. 2025. Learning multi-level features with matryoshka sparse autoencoders. arXiv preprint arXiv:2503.17547. — [arXiv:2503.17547](https://arxiv.org/abs/2503.17547) find
  10. Sviatoslav Chalnev, Matthew Siu, and Arthur Conmy. 2024. Improving steering vectors by targeting sparse autoencoder features. arXiv preprint arXiv:2411.02193. — [arXiv:2411.02193](https://arxiv.org/abs/2411.02193) find
  11. David Chanin, James Wilken-Smith, Tomáš Dulka, Hardik Bhatnagar, and Joseph Bloom. 2024. A is for absorption: Studying feature splitting and absorption in sparse autoencoders. arXiv preprint arXiv:2409.14507. — [arXiv:2409.14507](https://arxiv.org/abs/2409.14507) find
  12. Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. 2023. Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600. find
  13. David L Donoho and Michael Elad. 2003. Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization. Proceedings of the National Academy of Sciences, 100(5):2197–2202. find
  14. Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, and 1 others. 2022. Toy models of superposition. arXiv preprint arXiv:2209.10652. find
  15. Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, and 1 others. 2021. A mathematical framework for transformer circuits. Transformer Circuits Thread, 1(1):12. find
  16. Thomas Fel, Ekdeep Singh Lubana, Jacob S Prince, Matthew Kowal, Victor Boutin, Isabel Papadimitriou, Binxu Wang, Martin Wattenberg, Demba Ba, and Talia Konkle. 2025. Archetypal sae: Adaptive and stable dictionary learning for concept extraction in large vision models. arXiv preprint arXiv:2502.12892… — [arXiv:2502.12892](https://arxiv.org/abs/2502.12892) find
  17. Timo Freiesleben, Gunnar König, Christoph Molnar, and Alvaro Tejero-Cantero. 2024. Scientific inference with interpretable machine learning: Analyzing models to learn about real-world phenomena. Minds and Machines, 34(3):32. find
  18. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, and 1 others. 2020. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027. find
  19. Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. 2024. Scaling and evaluating sparse autoencoders. arXiv preprint arXiv:2406.04093. find
  20. Leo Gao, Tom Dupre la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. 2025. Scaling and evaluating sparse autoencoders. In The Thirteenth International Conference on Learning Representations. find
  21. Atticus Geiger, Duligur Ibeling, Amir Zur, Maheep Chaudhary, Sonakshi Chauhan, Jing Huang, Aryaman Arora, Zhengxuan Wu, Noah Goodman, Christopher Potts, and 1 others. 2023. Causal abstraction: A theoretical foundation for mechanistic interpretability. arXiv preprint arXiv:2301.04709. find
  22. Ruben Härle, Felix Friedrich, Manuel Brack, Björn Deiseroth, Patrick Schramowski, and Kristian Kersting. 2024. Scar: Sparse conditioned autoencoders for concept detection and steering in llms. arXiv preprint arXiv:2411.07122. — [arXiv:2411.07122](https://arxiv.org/abs/2411.07122) find
  23. Christopher J Hillar and Friedrich T Sommer. 2015. When can dictionary learning uniquely recover sparse data from subsamples? IEEE Transactions on Information Theory, 61(11):6290–6297. find
  24. Sai Sumedh R Hindupur, Ekdeep Singh Lubana, Thomas Fel, and Demba Ba. 2025. Projecting assumptions: The duality between sparse autoencoders and concept geometry. arXiv preprint arXiv:2503.01822. — [arXiv:2503.01822](https://arxiv.org/abs/2503.01822) find
  25. Zachary C Lipton. 2018. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue, 16(3):31–57. find
  26. Aleksandar Makelov, Georg Lange, and Neel Nanda. 2023. Is this the subspace you are looking for? an interpretability illusion for subspace activation patching. arXiv preprint arXiv:2311.17030. — [arXiv:2311.17030](https://arxiv.org/abs/2311.17030) find
  27. Luke Marks, Alasdair Paren, David Krueger, and Fazl Barez. 2024a. Enhancing neural network interpretability with feature-aligned sparse autoencoders. arXiv preprint arXiv:2411.01220. — [arXiv:2411.01220](https://arxiv.org/abs/2411.01220) find
  28. Aapo Hyvärinen and Erkki Oja. 2000. Independent component analysis: algorithms and applications. Neural networks, 13(4-5):411–430. find
  29. Samuel Marks, Adam Karvonen, and Aaron Mueller. 2024b. Dictionary learning. https://github.com/saprmarks/dictionary_learning. ↗
  30. Alon Jacovi and Yoav Goldberg. 2020. Towards faithfully interpretable nlp systems: How should we define and evaluate faithfulness? In Proceedings of the 58th annual meeting of the association for computational linguistics, pages 4198–4205.
  31. Samuel Marks, Can Rager, Eric J Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. 2024c. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models. arXiv preprint arXiv:2403.19647. find
  32. Shruti Joshi, Andrea Dittadi, Sébastien Lachapelle, and Dhanya Sridhar. 2025. Identifiable steering via sparse autoencoding of multi-concept shifts. arXiv preprint arXiv:2502.12179. — [arXiv:2502.12179](https://arxiv.org/abs/2502.12179) find
  33. Samuel Marks, Can Rager, Eric J Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. 2025. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models. In The Thirteenth International Conference on Learning Representations. find
  34. Adam Karvonen, Can Rager, Jessica Lin, Curt Tigges, Jacob Bloom, Daniel Chanin, Yue-Ting Lau, Euan Farrell, Arthur Conmy, Callum McDougall, Kolawole Ayonrinde, Martin Wearden, Logan Marks, and Neel Nanda. 2024a. SAEBench: A Comprehensive Benchmark for Sparse Autoencoders. https://www.neuronpedia.or… ↗
  35. Adam Karvonen, Benjamin Wright, Can Rager, Rico Angell, Jannik Brinkmann, Logan Smith, Claudio Mayrink Verdun, David Bau, and Samuel Marks. 2024b. Measuring progress in dictionary learning for language model interpretability with board game models. Advances in Neural Information Processing Systems,… find
  36. Harold W Kuhn. 1955. The hungarian method for the assignment problem. Naval research logistics quarterly, 2(1-2):83–97. find
  37. Patrick Leask, Bart Bussmann, Michael Pearce, Joseph Bloom, Curt Tigges, Noura Al Moubayed, Lee Sharkey, and Neel Nanda. 2025. Sparse autoencoders do not find canonical units of analysis. arXiv preprint arXiv:2502.04878. — [arXiv:2502.04878](https://arxiv.org/abs/2502.04878) find
  38. Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, Nicholas L. Turner, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, and 8 others… find
  39. Maxime Méloux, Silviu Maniu, François Portet, and Maxime Peyrard. 2025. Everything, everywhere, all at once: Is mechanistic interpretability identifiable? arXiv preprint arXiv:2502.20914. — [arXiv:2502.20914](https://arxiv.org/abs/2502.20914) find
  40. Aashiq Muhamed, Jacopo Bonato, Mona Diab, and Virginia Smith. 2025. Saes can improve unlearning: Dynamic sparse autoencoder guardrails for precision unlearning in llms. arXiv preprint arXiv:2504.08192. — [arXiv:2504.08192](https://arxiv.org/abs/2504.08192) find
  41. Kyle O’Brien, David Majercak, Xavier Fernandes, Richard Edgar, Jingya Chen, Harsha Nori, Dean Carignan, Eric Horvitz, and Forough Poursabzi-Sangde. 2024. Steering language model refusal with sparse autoencoders. arXiv preprint arXiv:2411.11296. — [arXiv:2411.11296](https://arxiv.org/abs/2411.11296) find
  42. Tuomas Oikarinen, Ge Yan, and Tsui-Wei Weng. 2025. Evaluating neuron explanations: A unified framework with sanity checks. arXiv preprint arXiv:2506.05774. — [arXiv:2506.05774](https://arxiv.org/abs/2506.05774) find
  43. Chris Olah. 2022. Mechanistic interpretability, variables, and the importance of interpretable bases. Transformer Circuits Thread, 2(4). find
  44. Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. 2020. Zoom in: An introduction to circuits. Distill, 5(3):e00024– 001. find
  45. Chris Olah, Arvind Satyanarayan, Ian Johnson, Shan Carter, Ludwig Schubert, Katherine Ye, and Alexander Mordvintsev. 2018. The building blocks of interpretability. Distill, 3(3):e10. find
  46. Charles O’Neill, Alim Gumran, and David Klindt. 2024. Compute optimal inference and provable amortisa-tion gap in sparse autoencoders. arXiv preprint arXiv:2411.13117. — [arXiv:2411.13117](https://arxiv.org/abs/2411.13117) find
  47. Mateusz Pach, Shyamgopal Karthik, Quentin Bouniot, Serge Belongie, and Zeynep Akata. 2025. Sparse autoencoders learn monosemantic features in vision-language models. arXiv preprint arXiv:2504.02821. — [arXiv:2504.02821](https://arxiv.org/abs/2504.02821) find
  48. Gonçalo Paulo and Nora Belrose. 2025. Sparse autoencoders trained on the same data learn different features. arXiv preprint arXiv:2501.16615. — [arXiv:2501.16615](https://arxiv.org/abs/2501.16615) find
  49. Gonçalo Paulo, Alex Mallen, Caden Juang, and Nora Belrose. 2024. Automatically interpreting millions of features in large language models. arXiv preprint arXiv:2410.13928. — [arXiv:2410.13928](https://arxiv.org/abs/2410.13928) find
  50. Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Tom Lieberum, Vikrant Varma, János Kramár, Rohin Shah, and Neel Nanda. 2024a. Improving dictionary learning with gated sparse autoencoders. arXiv preprint arXiv:2404.16014.
  51. Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Nanda. 2024b. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders. arXiv preprint arXiv:2407.14435. — [arXiv:2407.14435](https://arxiv.org/abs/2407.14435) find
  52. Yuchen Sun and Kejun Huang. 2024. Global identifiability of overcomplete dictionary learning via l1 and volume minimization. In The Twelfth International Conference on Learning Representations (ICLR). find
  53. Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, and 1 others. 2024a. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295. find
  54. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, and 1 others. 2024b. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118. find
  55. Kexin Wang and Anna Seigal. 2024. Identifiability of overcomplete independent component analysis. arXiv preprint arXiv:2401.14709. — [arXiv:2401.14709](https://arxiv.org/abs/2401.14709) find
  56. Xudong Zhu, Mohammad Mahdi Khalili, and Zhihui Zhu. 2025. Abstopk: Rethinking sparse autoencoders for bidirectional features. arXiv preprint arXiv:2510.00404. — [arXiv:2510.00404](https://arxiv.org/abs/2510.00404) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/song-2026-mechanistic-interpretability-should-prioritize-feature/