Elamur Research⌕
☰
home / papers / 2026 / In-Distribution Steering: Balancing Control and Coherence in Language Model Generation

In-Distribution Steering: Balancing Control and Coherence in Language Model Generation

Summary

Activation steering changes an LLM's behaviour by shifting its hidden states along a concept direction during inference. Most methods use one fixed strength for every prompt and token. As a result, they either fail to change the behaviour or push activations so far that the generated text loses coherence or collapses. The authors propose In-Distribution Steering (IDS), which chooses the steering coefficient separately for each input, layer and token position. A contrastive prompt set, split by whether the model shows the target behaviour, is used to model the activation distribution of the positive class. IDS then finds the largest coefficient that keeps the steered activation within a distance threshold of that distribution. The threshold uses Mahalanobis distance in a PCA-reduced space, and the problem reduces to a quadratic with a closed-form solution. Steering vectors come from diff-mean. Only layers whose F1-score for separating the classes passes a threshold are steered. Evaluated on six LLMs and seven datasets against CAA and MERA, IDS achieves a Pareto-optimal trade-off of steering effectiveness versus how plausible the generated text is. This makes steering more usable in real applications.

Key points

  • The baselines are CAA, which uses a fixed steering factor with diff-mean vectors, and MERA, which uses probe-based vectors with calibrated abstention. IDS is evaluated on both single-token prediction and open-ended generation.
  • Steering performance is reported as SPI, and plausibility as perplexity. Open-ended outputs on refusal, hallucination and myopic reward are graded 0/1 by a GPT judge. The tasks also include SMS spam/ham classification and MMLU.
  • The steering factor comes from the larger root of a quadratic constraint. If no root exists, IDS takes the value that minimises the distance to the target distribution instead.
  • Ablations on Gemma-2-2B and Gemma-2-9B vary four settings: the first steered layer, the PCA explained variance, the F1 threshold and the Mahalanobis threshold. Code is published at github.com/ArthurVogels26/InDistributionSteering.
  • The supplied extract omits the results section, so exact scores cannot be reported here.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/4629fxavjicu

@inproceedings{vogels2026distribution,
  title = {{In-Distribution Steering: Balancing Control and Coherence in Language Model Generation}},
  author = {Vogels, Arthur and Wong, Benjamin and Choho, Yann and Blangero, Annabelle and Bhan, Milan},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/4629fxavjicu},
  url = {https://doi.org/10.63317/4629fxavjicu}
}

References

43 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Guillaume Alain and Yoshua Bengio. 2018. Understanding intermediate layers using linear classifier probes. find
  2. Tiago A Almeida, José María G Hidalgo, and Akebo Yamakami. 2011. Contributions to the study of sms spam filtering: new collection and results. In Proceedings of the 11th ACM symposium on Document engineering, pages 259–262. find
  3. Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. 2024. Refusal in language models is mediated by a single direction. Advances in Neural Information Processing Systems, 37:136037–136083. find our card
  4. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. find
  5. Yoshua Bengio, Sören Mindermann, Daniel Privitera, Tamay Besiroglu, Rishi Bommasani, Stephen Casper, Yejin Choi, Philip Fox, Ben Garfinkel, Danielle Goldfarb, et al. 2025. International ai safety report. arXiv e-prints, pages arXiv–2501. find
  6. Milan Bhan, Yann Choho, Pierre Moreau, Jean-Noel Vittaut, Nicolas Chesneau, and Marie-Jeanne Lesot. 2025a. Towards achieving concept completeness for textual concept bottleneck models. In Findings of the 2025 Conference on Empirical Methods in Natural Language Processing. Association for Computation… find
  7. Milan Bhan, Jean-Noel Vittaut, Nicolas Chesneau, Sarath Chandar, and Marie-Jeanne Lesot. 2025b. Did i faithfully say what i thought? bridging the gap between neural activity and self-explanations in large language models. arXiv preprint arXiv:2506.09277. — [arXiv:2506.09277](https://arxiv.org/abs/2506.09277) find
  8. Milan Bhan, Jean-Noël Vittaut, Nicolas Chesneau, and Marie-Jeanne Lesot. 2024. Self-AMPLIFY: Improving small language models with self post hoc explanations. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 10974–10991, Miami, Florida, USA. Association… find
  9. Paul F Christiano, Jan Leike, Tom B Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. find
  10. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30. find
  11. Yann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy S Liang, and Tatsunori B Hashimoto. 2023. Alpacafarm: A simulation framework for methods that learn from human feedback. Advances in Neural Information Processing Systems, 36:30039–30069. find our card
  12. Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. 2022. Toy models of superposition. Transformer… find
  13. Sicheng Feng, Gongfan Fang, Xinyin Ma, and Xinchao Wang. 2025. Efficient reasoning models: A survey. arXiv preprint arXiv:2504.10903. find
  14. Nicholas Goldowsky-Dill, Bilal Chughtai, Stefan Heimersheim, and Marius Hobbhahn. 2025. Detecting strategic deception with linear probes. In Forty-second International Conference on Machine Learning. find
  15. Anna Hedström, Salim I Amoukou, Tom Bewley, Saumitra Mishra, and Manuela Veloso. 2025. To steer or not to steer? mechanistic error reduction with abstention for language models. In Forty-second International Conference on Machine Learning. find
  16. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. In International Conference on Learning Representations. find our card
  17. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. 2025. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 43(2):… find our card
  18. Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen. 2023. Catastrophic jailbreak of open-source llms via exploiting generation. arXiv preprint arXiv:2310.06987. find our card
  19. Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. 2018. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning, pages 2668–2677. PMLR. find
  20. Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2024. Inference-time intervention: Eliciting truthful answers from a language model. find our card
  21. Samuel Marks and Max Tegmark. 2024. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. find our card
  22. Ggaliwango Marvin, Nakayiza Hellen, Daudi Jjingo, and Joyce Nakatumba-Nabende. 2023. Prompt engineering in large language models. In International conference on data intelligence and cognitive informatics, pages 387–402. Springer. find
  23. Mantas Mazeika, Dan Hendrycks, Huichen Li, Xiaojun Xu, Sidney Hough, Andy Zou, Arezoo Rajabi, Qi Yao, Zihao Wang, Jian Tian, et al. 2023. The trojan detection challenge. In NeurIPS 2022 Competition Track, pages 279–291. PMLR. find
  24. Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, et al. 2024. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. In International Conference on Machine Learning, pages 35181–35224. PML… find our card
  25. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013. Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 746–751, Atlanta, Georgia.… find
  26. Kiho Park, Yo Joong Choe, and Victor Veitch. 2024. The linear representation hypothesis and the geometry of large language models. In Forty-first International Conference on Machine Learning. find our card
  27. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems. find our card
  28. Dehua Peng, Zhipeng Gui, and Huayi Wu. 2023. Interpreting the curse of dimensionality from distance concentration and manifold effect. arXiv preprint arXiv:2401.00422. — [arXiv:2401.00422](https://arxiv.org/abs/2401.00422) find
  29. Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. 2024. Steering llama 2 via contrastive activation addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15504– 15522, Bangkok, Thail… find
  30. Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R Bowman, et al. 2023. Towards understanding sycophancy in language models. arXiv preprint arXiv:2310.13548. find our card
  31. Tianhao Shen, Renren Jin, Yufei Huang, Chuang Liu, Weilong Dong, Zishan Guo, Xinwei Wu, Yan Liu, and Deyi Xiong. 2023. Large language model alignment: A survey. arXiv preprint arXiv:2309.15025. find
  32. Rachel SY Teo, Laziz Abdullaev, and Tan Minh Nguyen. 2025. The blessing and curse of dimensionality in safety alignment. In Second Conference on Language Modeling. find
  33. Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmid. 2024. Steering language models with activation engineering. find our card
  34. Hieu M Vu and Tan Minh Nguyen. 2025. Angular steering: Behavior control via rotation in activation space. In ICML 2025 Workshop on Reliable and Responsible Foundation Models. find
  35. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-Consistency Improves Chain of Thought Reasoning in Language Models. In Proc. of the 11th Int. Conf. on Learning Representations, ICLR23. find our card
  36. Zhichao Wang, Bin Bi, Shiva Kumar Pentyala, Kiran Ramnath, Sougata Chaudhuri, Shubham Mehrotra, Xiang-Bo Mao, Sitaram Asur, et al. 2024. A comprehensive survey of llm alignment techniques: Rlhf, rlaif, ppo, dpo and more. arXiv preprint arXiv:2407.16216. — [arXiv:2407.16216](https://arxiv.org/abs/2407.16216) find
  37. Maurice Weber, Dan Fu, Quentin Anthony, Yonatan Oren, Shane Adams, Anton Alexandrov, Xiaozhong Lyu, Huu Nguyen, Xiaozhe Yao, Virginia Adams, et al. 2024. Redpajama: an open dataset for training large language models. Advances in neural information processing systems, 37:116462–116492. find our card
  38. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2024. Chain-of-thought prompting elicits reasoning in large language models. In Proc. of the 36th Int. Conf. on Neural Information Processing Systems, NIPS ’22, Red Hook, NY, USA. Cu… find our card
  39. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2020. Transformers: State-ofthe-art natural language processing. In Proc. of the Conf. on Empirical Methods in Natural language Processing: system… find
  40. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36. find
  41. Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. 2023a. Representation engineering: A top-down approach to ai transparency. arXiv preprint arXiv:2310.01405. find our card
  42. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023b. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043. 10. Appendix 10.1. find our card
  43. Scientific Libraries We used several open-source libraries in this work: pytorch (Paszke et al., 2019), HuggingFace transformers (Wolf et al., 2020), transformer-lens. 10.2. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/vogels-2026-distribution-steering-balancing-control-coherence/