Elamur Research⌕
☰
home / papers / 2026 / Safe-SAIL: Towards a Fine-grained Safety Landscape of Large Language Models via Sparse Autoencoder Interpretation Framework

Safe-SAIL: Towards a Fine-grained Safety Landscape of Large Language Models via Sparse Autoencoder Interpretation Framework

Paper · 2026 · Jiaqi Weng, Han Zheng, Hanyu Zhang et al.

Summary

Sparse autoencoders (SAEs) can split a model's tangled activations into interpretable features. However, safety concepts are rare in typical data, and it is unclear which SAE setups produce fine-grained safety features. Explaining every feature is also expensive. Safe-SAIL is a three-stage pipeline (SAE training, automated interpretation, diagnosis toolkit) that addresses both problems.

To choose among SAEs of different sparsity before any explanation is written, the authors build pairs of queries in which a target safety concept is present or removed. They then measure how many latents reliably separate the two versions. A large reasoning model writes the feature explanations. Scoring is done at the segment level: the model predicts whether a feature fires on each chunk of text, rather than predicting a value for every token. This cuts cost roughly in half while staying close to token-level scores.

Applied to Qwen2.5-3B-Instruct, the framework yields an explained feature base for four risk areas. Case studies trace how risky entities and concepts are processed across layers, and suggest why harmful prompts in low-resource languages can evade refusal.

Key points

  • Pre-explanation selection metrics: two metrics, L0,t and I_CDF, are computed on concept-contrastive query pairs. L0,t counts the latents that separate a concept above a threshold t. I_CDF is the expected separation frequency, read as the area under a CDF curve. Features are then filtered with thresholds on precision and recall.
  • Cheaper evaluation: segment-level binary simulation lowers interpretation cost by about 55% and keeps high correlation with token-level simulation.
  • Released resource: TopK SAEs trained on layers 0, 8, 17, 26 and 35 of Qwen2.5-3B-Instruct. The feature base covers 1,758 explained safety features across pornography, politics, violence and terror, including 309 pornography features.
  • Case studies: for a child-exploitation prompt, the activated features move from word detection to scene construction, then to abstract concepts such as exploitation, and finally to a refusal. In the Hindi version of the prompt, these concept features activate weakly or not at all, and the model fails to refuse.
  • Availability and limits: the checkpoints, explanations and toolkit are public on GitHub (Alibaba-AAIG/Safe-SAIL) and ModelScope, but the SAE training data is not released. Causal evidence comes only from preliminary steering experiments.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of CambridgeCambridge · United Kingdom243 works · 121,396 citations🏢Alibaba GroupHangzhou · China168 works · 15,386 citations🎓Renmin University of ChinaBeijing · China114 works · 16,118 citations🎓Zhejiang UniversityHangzhou · China172 works · 12,988 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksBias, fairness & safety · 517 works

Cite

Cite as Jiaqi Weng, Han Zheng, Hanyu Zhang, Ej Zhou, Qinqin He, Jialing Tao, Hui Xue, Zhixuan Chu and Xiting Wang. 2026. Safe-SAIL: Towards a Fine-grained Safety Landscape of Large Language Models via Sparse Autoencoder Interpretation Framework. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.944

@inproceedings{weng2026safe,
  title = {{Safe-SAIL: Towards a Fine-grained Safety Landscape of Large Language Models via Sparse Autoencoder Interpretation Framework}},
  author = {Jiaqi Weng, Han Zheng, Hanyu Zhang, Ej Zhou, Qinqin He, Jialing Tao, Hui Xue, Zhixuan Chu and Xiting Wang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.944},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.944}
}

References

33 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anthropic. 2025. Claude 3.7 sonnet and claude code. find
  2. Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. 2024. Refusal in language models is mediated by a single direction. Advances in Neural Information Processing Systems, 37:136037–136083.
  3. Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Y. Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi. 2025. Monitoring reasoning models for misbehavior and the risks of promoting obfuscation. Preprint, arXiv:2503.11926. find
  4. Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. 2023. Language models can explain neurons in language models. URL https://openaipublic.blob. core. windows. net/neuron-explainer/paper/index. html.(Date accessed:… ↗
  5. Matyas Bohacek, Thomas Fel, Maneesh Agrawala, and Ekdeep Singh Lubana. 2025. Uncovering conceptual blindspots in generative image models using sparse autoencoders. Preprint, arXiv:2506.19708. — [arXiv:2506.19708](https://arxiv.org/abs/2506.19708) find
  6. Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, and 6 others. 2023. To… find
  7. Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. 2023. Sparse autoencoders find highly interpretable features in language models. Preprint, arXiv:2309.08600. find
  8. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. Deepseek-r1: Incentivizing reasoning capability in ll…
  9. Javier Ferrando, Gabriele Sarti, Arianna Bisazza, and Marta R. Costa-jussà. 2024. A primer on the inner workings of transformer-based language models. Preprint, arXiv:2405.00208. find
  10. Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K. Ahmed. 2024. Bias and fairness in large language models: A survey. Preprint, arXiv:2309.00770.
  11. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. 2020. The pile: An 800gb dataset of diverse text for language modeling. Preprint, arXiv:2101.00027. find
  12. Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. 2024. Scaling and evaluating sparse autoencoders. arXiv preprint arXiv:2406.04093. find
  13. Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii, and Dimitris Bertsimas. 2023. Finding neurons in a haystack: Case studies with sparse probing. Preprint, arXiv:2305.01610. find
  14. Bart Bussmann, Patrick Leask, and Neel Nanda. 2024. Batchtopk sparse autoencoders. Preprint, arXiv:2412.06410. — [arXiv:2412.06410](https://arxiv.org/abs/2412.06410) find
  15. Laura Hanu and Unitary team. 2020. Detoxify. Github. https://github.com/unitaryai/detoxify. ↗
  16. Bart Bussmann, Noa Nabeshima, Adam Karvonen, and Neel Nanda. 2025. Learning multi-level features with matryoshka sparse autoencoders. Preprint, arXiv:2503.17547. — [arXiv:2503.17547](https://arxiv.org/abs/2503.17547) find
  17. Zhengfu He, Wentao Shu, Xuyang Ge, Lingjie Chen, Junxuan Wang, Yunhua Zhou, Frances Liu, Qipeng Guo, Xuanjing Huang, Zuxuan Wu, Yu-Gang Jiang, and Xipeng Qiu. 2024. Llama scope: Extracting millions of features from llama-3.1-8b with sparse autoencoders. Preprint, arXiv:2410.20526. find
  18. Samuel Jacob Chacko, Sajib Biswas, Chashi Mahiul Islam, Fatema Tabassum Liza, and Xiuwen Liu. 2024. Adversarial attacks on large language models using regularized relaxation. Preprint, arXiv:2410.19160. — [arXiv:2410.19160](https://arxiv.org/abs/2410.19160) find
  19. Jianhui Chen, Xiaozhi Wang, Zijun Yao, Yushi Bai, Lei Hou, and Juanzi Li. 2024. Finding safety neurons in large language models. arXiv preprint arXiv:2406.14144. — [arXiv:2406.14144](https://arxiv.org/abs/2406.14144) find
  20. Adam Karvonen, Can Rager, Johnny Lin, Curt Tigges, Joseph Bloom, David Chanin, Yeu-Tong Lau, Eoin Farrell, Callum McDougall, Kola Ayonrinde, Demian Till, Matthew Wearden, Arthur Conmy, Samuel Marks, and Neel Nanda. 2025. Saebench: A comprehensive benchmark for sparse autoencoders in language model i… — [arXiv:2503.09532](https://arxiv.org/abs/2503.09532) find
  21. Dami Choi, Vincent Huang, Kevin Meng, Daniel D Johnson, Jacob Steinhardt, and Sarah Schwettmann. 2024. Scaling automatic neuron description. https://transluce.org/neuron-descriptions. ↗
  22. Adam Karvonen, Benjamin Wright, Can Rager, Rico Angell, Jannik Brinkmann, Logan Smith, Claudio Mayrink Verdun, David Bau, and Samuel Marks. 2024. Measuring progress in dictionary learning for language model interpretability with board game models. Preprint, arXiv:2408.00113. — [arXiv:2408.00113](https://arxiv.org/abs/2408.00113) find
  23. Alyssa Lees, Vinh Q. Tran, Yi Tay, Jeffrey Sorensen, Jai Gupta, Donald Metzler, and Lucy Vasserman. 2022. A new generation of perspective api: Efficient multilingual character-level transformers. Preprint, arXiv:2202.11176. find
  24. Haoran Li, Yulin Chen, Jinglong Luo, Jiecong Wang, Hao Peng, Yan Kang, Xiaojin Zhang, Qi Hu, Chunkit Chan, Zenglin Xu, Bryan Hooi, and Yangqiu Song. 2024. Privacy in large language models: Attacks, defenses and future directions. Preprint, arXiv:2310.10383. — [arXiv:2310.10383](https://arxiv.org/abs/2310.10383) find
  25. Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah, and Neel Nanda. 2024. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2. Preprint, arXiv:2408.05147. find
  26. Gonçalo Paulo, Alex Mallen, Caden Juang, and Nora Belrose. 2024. Automatically interpreting millions of features in large language models. Preprint, arXiv:2410.13928. — [arXiv:2410.13928](https://arxiv.org/abs/2410.13928) find
  27. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. find
  28. Qwen Team. 2025. Qwq-32b: Embracing the power of reinforcement learning. find
  29. Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Nanda. 2024. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders. Preprint, arXiv:2407.14435. — [arXiv:2407.14435](https://arxiv.org/abs/2407.14435) find
  30. Leo Schwinn, David Dobre, Stephan Günnemann, and Gauthier Gidel. 2023. Adversarial attacks and defenses in large language models: Old and new threats. Preprint, arXiv:2310.19737. — [arXiv:2310.19737](https://arxiv.org/abs/2310.19737) find
  31. Zhihao Xu, Ruixuan Huang, Changyu Chen, and Xiting Wang. 2024. Uncovering safety risks of large language models through concept activation vector. Advances in Neural Information Processing Systems, 37:116743–116782. find
  32. Zhihao Xu, Ruixuan Huang, Changyu Chen, and Xiting Wang. 2025. Uncovering safety risks of large language models through concept activation vector. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Red Hook, NY, USA. Curran Associates Inc. find
  33. Yiran Zhao, Wenxuan Zhang, Yuxi Xie, Anirudh Goyal, Kenji Kawaguchi, and Michael Shieh. 2025. Understanding and enhancing safety mechanisms of llms via safety-specific neuron. In The Thirteenth International Conference on Learning Representations. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/weng-2026-safe-sail-fine-grained-safety/