Elamur Research⌕
☰
home / papers / 2026 / RISER: Orchestrating Latent Reasoning Skills for Adaptive Activation Steering

RISER: Orchestrating Latent Reasoning Skills for Adaptive Activation Steering

Paper · 2026 · Wencheng Ye, Xiaoyang Yuan, Yi Bin et al.

Summary

LLMs reason well in general but are uneven on specialised tasks. Fine-tuning fixes this at high cost and with the risk of forgetting. Existing activation steering usually adds one fixed vector at a hand-tuned strength. RISER, whose name reflects its use of a router that intervenes in the model to steer and improve its reasoning, leaves the model's weights untouched and views reasoning as a collection of combinable "cognitive primitives," a notion loosely drawn from how the prefrontal cortex exerts control.

The authors build the vector library offline: they elicit candidate reasoning vectors, filter them with an LLM judge, and cluster them into a disentangled set. A small Router then reads the last-token hidden state at an intermediate layer and makes two decisions: which vectors to switch on, and how strongly to apply each one. The weighted sum is added to the activation at that layer. The Router is trained with reinforcement learning on task-level rewards.

On seven benchmarks, RISER improves zero-shot accuracy by 3.4–6.5% on average over the base model. It also uses 2–3× fewer tokens than CoT-style reasoning. The Router's choices remain interpretable, which makes this a cheap, transparent route to better reasoning.

Key points

  • Evaluation benchmarks: MATH, GSM8K, GPQA, MMLU-Pro, ARC-C, Ethics and TruthfulQA. Backbones include the Qwen2.5 family (7B/14B/32B) and Llama-3-8B.
  • Comparison with LoRA on Qwen2.5-7B:
    • The LoRA baseline is dense (rank 64, about 160M parameters).
    • RISER matches or slightly beats it, for example 53.3 on MATH and 85.2 on GSM8K.
    • RISER generates far fewer tokens, for example about 1,392 vs 3,853 on MATH.
  • Extensibility to code:
    • A seventh coding primitive was added, and only the Router was updated, using a brief SFT phase on 200 examples.
    • HumanEval pass@1 rose to 59.9%, compared with 56.3% for the base model and 57.2% for static CAA.
    • Results on the original reasoning tasks did not drop noticeably.
  • Cross-model transfer: A trained configuration carries over across Qwen2.5 sizes, but gives no benefit when moved between Qwen and Llama.
  • Overhead: The Router needs roughly 10MB of memory and about 10M FLOPs per call. It handles mixed batches and can run as a vLLM forward hook.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢TencentShenzhen · China134 works · 19,743 citations🎓Tongji UniversityShanghai · China19 works · 2,162 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Wencheng Ye, Xiaoyang Yuan, Yi Bin, Hengyu Jin, Liang Peng, Pengpeng Zeng and Heng Tao Shen. 2026. RISER: Orchestrating Latent Reasoning Skills for Adaptive Activation Steering. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.226

@inproceedings{ye2026riser,
  title = {{RISER: Orchestrating Latent Reasoning Skills for Adaptive Activation Steering}},
  author = {Wencheng Ye, Xiaoyang Yuan, Yi Bin, Hengyu Jin, Liang Peng, Pengpeng Zeng and Heng Tao Shen},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.226},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.226}
}

References

63 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin. 2024. Large language models for mathematical reasoning: Progresses and challenges. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, pages…
  2. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge.
  3. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems.
  4. Guillaume Alain and Yoshua Bengio. 2016. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644. find
  5. Hoagy Cunningham, Aidan Ewart, Logan Riggs Smith, Robert Huben, and Lee Sharkey. 2023. Sparse autoencoders find highly interpretable features in language models. ArXiv, abs/2309.08600. find
  6. Michael L. Anderson. 2010. Neural reuse: A fundamental organizational principle of the brain. Behavioral and Brain Sciences, 33(4):245–66; discussion 266–313. find
  7. Hannah Cyberey and David Evans. 2025. Steering the censorship: Uncovering representation vectors for LLM ”thought” control. In Second Conference on Language Modeling. find
  8. Anthropic. 2024. Model card addendum: Claude 3.5 haiku and upgraded claude 3.5 sonnet. find
  9. Fei Ding and Baiqiao Wang. 2025. Improved supervised fine-tuning for large language models to mitigate catastrophic forgetting. ArXiv, abs/2506.09428. — [arXiv:2506.09428](https://arxiv.org/abs/2506.09428) find
  10. Lukasz Bartoszcze, Sarthak Munshi, Bryan Sukidi, Jennifer Yen, Zejia Yang, David Williams-King, Linh Le, Kosi Asuzu, and Carsten Maple. 2025. Representation engineering for large-language models: Survey and research challenges. find
  11. Reza Bayat, Ali Rahimi-Kalahroudi, Mohammad Pezeshki, Sarath Chandar, and Pascal Vincent. 2025. Steering large language model activations in sparse spaces. Preprint, arXiv:2503.00177. — [arXiv:2503.00177](https://arxiv.org/abs/2503.00177) find
  12. Yonatan Belinkov. 2022. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):207–219.
  13. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  14. Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, and Xing Xie. 2024. A survey on evaluation of large language models. ACM Trans. Intell. Syst. Technol., 15(3).
  15. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 39 others. 202… — Evaluating Large Language Models Trained on Code
  16. Runjin Chen, Andy Arditi, Henry Sleight, Owain Evans, and Jack Lindsey. 2025. Persona vectors: Monitoring and controlling character traits in language models. Preprint, arXiv:2507.21509. — [arXiv:2507.21509](https://arxiv.org/abs/2507.21509) find
  17. Harshwardhan Fartale, Ashish Kattamuri, Rahul Raja, Arpita Vats, Ishita Prasad, and Akshata Kishore Moharir. 2025. Disentangling recall and reasoning in transformer models through layer-wise attention and activation analysis. Preprint, arXiv:2510.03366. — [arXiv:2510.03366](https://arxiv.org/abs/2510.03366) find
  18. William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23(120):1–39. find
  19. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — The Llama 3 Herd of Models
  20. Yixin He and Lumingyuan Tang. 2025. Learning to ponder: Adaptive reasoning in latent space. Preprint, arXiv:2509.24238. — [arXiv:2509.24238](https://arxiv.org/abs/2509.24238) find
  21. Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2021a. Aligning ai with shared human values. Proceedings of the International Conference on Learning Representations (ICLR).
  22. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021b. Measuring massive multitask language understanding. Preprint, arXiv:2009.03300.
  23. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021c. Measuring mathematical problem solving with the math dataset. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, volume 1.
  24. Bertram Hø jer, Oliver Jarvis, and Stefan Heinrich. 2025. Improving reasoning performance in large language models via representation engineering. In International Conference on Representation Learning, volume 2025, pages 44746–44763. find
  25. Maggie Huan, Yuetai Li, Tuney Zheng, Xiaoyu Xu, Seungone Kim, Minxin Du, Radha Poovendran, Graham Neubig, and Xiang Yue. 2025. Does math reasoning improve general llm capabilities? understanding transferability of llm reasoning. Preprint, arXiv:2507.00432. find
  26. Eric Jang, Shixiang Gu, and Ben Poole. 2017. Categorical reparameterization with gumbel-softmax. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
  27. Haoran Jin, Meng Li, Xiting Wang, Zhihao Xu, Minlie Huang, Yantao Jia, and Defu Lian. 2025. Internal value alignment in large language models through controlled value vector activation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)… find
  28. Nancy Kanwisher, Josh Mcdermott, and Marvin M. Chun. 1999. The fusiform face area: A module in human extrastriate cortex specialized for face perception. The Journal of Neuroscience, 17(11). find
  29. Bruce W. Lee, Inkit Padhi, Karthikeyan Natesan Ramamurthy, Erik Miehling, Pierre Dognin, Manish Nagireddy, and Amit Dhurandhar. 2025a. Programming refusal with conditional activation steering. Preprint, arXiv:2409.05907. find
  30. Sunbowen Lee, Qingyu Yin, Chak Tou Leong, Jialiang Zhang, Yicheng Gong, and Xiaoyu Shen. 2025b. Probing the difficulty perception mechanism of large language models. Preprint, arXiv:2510.05969. — [arXiv:2510.05969](https://arxiv.org/abs/2510.05969) find
  31. Hongyu Li, Liang Ding, Meng Fang, and Dacheng Tao. 2024. Revisiting catastrophic forgetting in large language model tuning. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 4297–4308, Miami, Florida, USA. Association for Computational Linguistics. find
  32. Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, Yingying Zhang, Fei Yin, Jiahua Dong, Zhiwei Li, Bao-Long Bi, Ling-Rui Mei, Junfeng Fang, Xiao Liang, Zhijiang Guo, and 2 others. 2025. From system 1 to system 2:…
  33. Mengqi Liao, Xiangyu Xi, Ruinian Chen, Jia Leng, Yangen Hu, Ke Zeng, Shuai Liu, and Huaiyu Wan. 2025. Enhancing efficiency and exploration in reinforcement learning for llms. Preprint, arXiv:2505.18573. find
  34. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. TruthfulQA: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252, Dublin, Ireland. Association for Computational Linguistics…
  35. Samuel Marks and Max Tegmark. 2023. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. ArXiv, abs/2310.06824.
  36. Earl K. Miller and Jonathan D. Cohen. 2001. An integrative theory of prefrontal cortex function. Annual Review of Neuroscience, 24(Volume 24, 2001):167– 202. find
  37. Deepak Babu Piskala, Vijay Raajaa, Sachin Mishra, and Bruno Bozza. 2024. Optiroute dynamic llm routing and selection based on user preferences: Balancing performance, cost, and ethics. International Journal of Computer Applications, 186(51):1–7. find
  38. Joris Postmus and Steven Abreu. 2024. Steering large language models using conceptors: Improving addition-based activation engineering. In MINT: Foundation Model Interventions. find
  39. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. find
  40. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. 2023. Gpqa: A graduate-level google-proof q&a benchmark. ArXiv, abs/2311.12022.
  41. Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. 2024. Steering llama 2 via contrastive activation addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15504–15522, Bangkok, Thaila… find
  42. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  43. Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. 2017. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. find
  44. Idan Shenfeld, Jyothish Pari, and Pulkit Agrawal. 2025. Rl’s razor: Why online reinforcement learning forgets less. arXiv preprint arXiv:2509.04259. — [arXiv:2509.04259](https://arxiv.org/abs/2509.04259) find
  45. Wenhao Shi, Zhiqiang Hu, Yi Bin, Junhua Liu, Yang Yang, See Kiong Ng, Lidong Bing, and Roy Ka-Wei Lee. 2024. Math-llava: Bootstrapping mathematical reasoning for multimodal large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 4663–4680.
  46. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V. Le, Ed H. Chi, Denny Zhou, and Jason Wei. 2022. Challenging big-bench tasks and whether chain-of-thought can solve them. Preprint, arXiv:2210.09261.
  47. Daniel Tan, David Chanin, Aengus Lynch, Brooks Paige, Dimitrios Kanoulas, Adrià Garriga-Alonso, and Robert Kirk. 2024. Analysing the generalisation and reliability of steering vectors. In Advances in Neural Information Processing Systems, volume 37, pages 139179–139212. Curran Associates, Inc. find
  48. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  49. Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte Macdiarmid. 2023. Steering language models with activation engineering.
  50. Marco Valentino, Geonhee Kim, Dhairya Dalal, Zhixue Zhao, and André Freitas. 2025. Mitigating content effects on reasoning in language models through fine-grained activation steering. arXiv preprint arXiv:2505.12189. — [arXiv:2505.12189](https://arxiv.org/abs/2505.12189) find
  51. Constantin Venhoff, Iván Arcuschin, Philip Torr, Arthur Conmy, and Neel Nanda. 2025a. Base models know how to reason, thinking models learn when. arXiv preprint arXiv:2510.07364. — [arXiv:2510.07364](https://arxiv.org/abs/2510.07364) find
  52. Constantin Venhoff, Iván Arcuschin, Philip Torr, Arthur Conmy, and Neel Nanda. 2025b. Understanding reasoning in thinking language models via steering vectors. In Workshop on Reasoning and Planning for Large Language Models. find
  53. Mengru Wang, Ziwen Xu, Shengyu Mao, Shumin Deng, Zhaopeng Tu, Huajun Chen, and Ningyu Zhang. 2025a. Beyond prompt engineering: Robust behavior control in LLMs via steering target atoms. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)…
  54. Ruonan Wang, Runxi Wang, Yunwen Shen, Chengfeng Wu, Qinglin Zhou, and Rohitash Chandra. 2025b. Evaluation of llms for mathematical problem solving. arXiv preprint arXiv:2506.00309. — [arXiv:2506.00309](https://arxiv.org/abs/2506.00309) find
  55. Xin Wang, Haoyang Li, Zeyang Zhang, Haibo Chen, and Wenwu Zhu. 2025c. Modular machine learning: An indispensable path towards new-generation large language models. CoRR, abs/2504.20020. — [arXiv:2504.20020](https://arxiv.org/abs/2504.20020) find
  56. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5,…
  57. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. 2025d. Mmlu-pro: a more robust and challenging multi-task language understanding benchmar…
  58. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, and 1 others. 2022a. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682.
  59. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022b. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pages 24824–24837. Curran Associates, Inc.
  60. Yuyang Wu, Yifei Wang, Ziyu Ye, Tianqi Du, Stefanie Jegelka, and Yisen Wang. 2025. When more is less: Understanding chain-of-thought length in llms. Preprint, arXiv:2502.07266. find
  61. Mohammad Zbeeb, Hasan Abed Al Kader Hammoud, and Bernard Ghanem. 2025. Reasoning vectors: Transferring chain-of-thought capabilities via task arithmetic. Preprint, arXiv:2509.01363. — [arXiv:2509.01363](https://arxiv.org/abs/2509.01363) find
  62. Hanyu Zhang, Xiting Wang, Chengao Li, Xiang Ao, and Qing He. 2025a. Controlling large language models through concept activation vectors. Proceedings of the AAAI Conference on Artificial Intelligence, 39(24):25851–25859. find
  63. Yi-Kai Zhang, De-Chuan Zhan, and Han-Jia Ye. 2025b. Capability instruction tuning. Proceedings of the AAAI Conference on Artificial Intelligence, 39:25958–25966. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ye-2026-riser-orchestrating-latent-reasoning-skills/