Elamur Research⌕
☰
home / papers / 2024 / Improving Dictionary Learning with Gated Sparse Autoencoders

Improving Dictionary Learning with Gated Sparse Autoencoders

Summary

Sparse autoencoders (SAEs) are used to break language model activations into sparse combinations of interpretable directions. The standard training objective uses an L1 sparsity penalty, which biases the encoder. Its most visible effect is shrinkage: feature activations come out systematically too small. The authors propose the Gated SAE, whose encoder splits two jobs. One path decides which dictionary features are switched on. A second path estimates how strongly each active feature fires. The L1 penalty applies only to the selection path, and the two paths share some weights so parameter count and inference cost stay close to a standard SAE. They trained SAEs on GELU-1L, Pythia-2.8B and Gemma-7B, covering MLP outputs, attention outputs and the residual stream. At equal training compute, Gated SAEs reach a given reconstruction quality with fewer active features, removing shrinkage without losing interpretability. This makes SAE-based tools for steering models and analysing circuits cheaper and more faithful.

Key points

  • Quality is measured with two metrics: L0 (average count of active features) and loss recovered (how much LM cross-entropy is preserved when SAE reconstructions are spliced in).
  • At many sites, Gated SAEs need about half the L0 to match baseline loss recovered. The comparison uses equal compute, with baseline SAEs given 50% more features.
  • Computing the Gated SAE training loss costs roughly 50% more than for a same-width baseline, because of an auxiliary loss term.
  • A small double-blind human study rated Gated SAE features about as interpretable as baseline ones.
  • An ablation study found every key component of the method necessary for strong performance. Gated SAEs also beat other approaches that target shrinkage.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Cite

Cite as Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Tom Lieberum, Vikrant Varma, Janos Kramar, Rohin Shah and Neel Nanda. 2024. Improving Dictionary Learning with Gated Sparse Autoencoders. arXiv:2404.16014.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2404.16014 · arXiv 2404.16014

@misc{rajamanoharan2024improving,
  title = {{Improving Dictionary Learning with Gated Sparse Autoencoders}},
  author = {Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Tom Lieberum, Vikrant Varma, Janos Kramar, Rohin Shah and Neel Nanda},
  year = {2024},
  doi = {10.48550/arxiv.2404.16014},
  eprint = {2404.16014},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2404.16014}
}

References

13 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. M. Aharon, M. Elad, and A. Bruckstein. K-svd: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing, 54(11):4311–4322, 2006. doi: 10.1109/TSP.2006.881199. — [doi:10.1109/tsp.2006.881199](https://doi.org/10.1109/tsp.2006.881199) find
  2. Anthropic AI. Introducing the next generation of Claude. https://www.anthropic.com/index/ introducing-the-next-generation-of-claude , 2024. Accessed: 2024-04-14. ↗
  3. J. Batson, B. Chen, A. Jones, A. Templeton, T. Conerly, J. Marcus, T. Henighan, N. L. Turner, and A. Pearce. Circuits Updates - March 2024. find
  4. Transformer Circuits Thread, 2024. URL https://transformer-circuits.pub/2024/mar-update/index.html . ↗
  5. Y. Bengio. Deep learning of representations: Looking forward, 2013. find
  6. S. Biderman, H. Schoelkopf, Q. G. Anthony, H. Bradley, K. O’Brien, E. Hallahan, M. A. Khan, S. Purohit, U. S. Prashanth, E. Raff, et al. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, pages 2397–2430. PMLR, 2023. find
  7. S. Bills, N. Cammarata, D. Mossing, H. Tillman, L. Gao, G. Goh, I. Sutskever, J. Leike, J. Wu, and W. Saunders. Language models can explain neurons in language models. https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html , 2023. ↗
  8. J. Bloom. Open Source Sparse Autoencoders for all Residual Stream Layers of GPT-2 Small. https://www.alignmentforum.org/posts/f9EgfLSurAiqRJySD/ open-source-sparse-autoencoders-for-all-residual-stream , 2024. ↗
  9. T. Blumensath and M. E. Davies. Gradient pursuits. IEEE Transactions on Signal Processing, 56(6): 2370–2382, 2008. find
  10. T. Bolukbasi, A. Pearce, A. Yuan, A. Coenen, E. Reif, F. Viégas, and M. Wattenberg. An interpretability illusion for bert. arXiv preprint arXiv:2104.07143, 2021. — [arXiv:2104.07143](https://arxiv.org/abs/2104.07143) find
  11. T. Bricken, A. Templeton, J. Batson, B. Chen, A. Jermyn, T. Conerly, N. Turner, C. Anil, C. Denison, A. Askell, R. Lasenby, Y. Wu, S. Kravec, N. Schiefer, T. Maxwell, N. Joseph, Z. Hatfield-Dodds, A. Tamkin, K. Nguyen, B. McLean, J. E. Burke, T. Hume, S. Carter, T. Henighan, and C. Olah. Towards mon… find
  12. R. T. Chen, X. Li, R. B. Grosse, and D. K. Duvenaud. Isolating sources of disentanglement in variational autoencoders. Advances in neural information processing systems, 31, 2018. find
  13. X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. Advances in neural information processing systems, 29, 2016. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rajamanoharan-2024-improving-dictionary-learning-gated-sparse/