Elamur Research⌕
☰
home / papers / 2026 / ReasoningGuard: Safeguarding Large Reasoning Models with Inference-time Safety Aha Moments

ReasoningGuard: Safeguarding Large Reasoning Models with Inference-time Safety Aha Moments

Paper · 2026 · Yuquan Wang, Mi Zhang, Yining Wang et al.

Summary

Large Reasoning Models (LRMs) such as DeepSeek-R1 often recognise a malicious request early on, yet still produce policy-violating content in the middle and later parts of their reasoning chains. The authors call this superficial safety alignment. Existing defences either require expensive fine-tuning and expert models, or are cheap but weak and prone to over-refusal.

ReasoningGuard is a training-free defence that operates during decoding, in two stages:

  • Stage 1: it watches last-layer attention through a sliding window to find attention-sink tokens. These mark a transition in the reasoning and serve as intervention points, where a crafted safety "aha" phrase is inserted to prompt self-reflection on safety.
  • Stage 2: a scaling sampling procedure ranks candidate continuations with an attention-based score (IAS) and keeps the safest competent path.

The method was tested on several open LRMs against four kinds of jailbreak attacks, including ones aimed at the reasoning process. It beats nine existing safeguards, avoids exaggerated safety, and preserves utility at low cost. This matters because LRMs that expose their reasoning chains to users can be protected without retraining.

Key points

  • Models evaluated: DeepSeek-R1 distilled models (R1-Qwen-7B, R1-Llama-8B, up to 32B), plus Phi-4-reasoning, QwQ-32B and Qwen3-4B-Thinking-2507 for transferability.
  • Baselines: compared with nine defences, both training-based and training-free (e.g., Paraphrase, Self-Reminder, SmoothLLM, ThinkingI, SAFEPATH-ZS). It is the strongest of the training-free group.
  • Utility: accuracy on MMLU, GPQA Diamond and MATH500 drops only minimally, whereas Paraphrase and SmoothLLM noticeably hurt performance.
  • Efficiency: generation-time overhead (ATGR on AdvBench) is about 5% on R1-Qwen-7B and 9% on R1-Llama-8B.
  • Transferability and ablations: on AdvBench, QwQ-32B's reasoning harmfulness score falls from 2.28 to 1.50. Ablations show the search-space size k matters little once k ≥ 5, and a smaller window W_max both improves defence and lowers compute.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Fudan UniversityShanghai · China167 works · 20,987 citations🎓East China University of Science and TechnologyShanghai · China10 works · 87 citations🎓Shanghai University of Electric PowerYangpu · China3 works · 13 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksBias, fairness & safety · 517 works

Cite

Cite as Yuquan Wang, Mi Zhang, Yining Wang, Geng Hong, Mi Wen, Xiaoyu You and Min Yang. 2026. ReasoningGuard: Safeguarding Large Reasoning Models with Inference-time Safety Aha Moments. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1453

@inproceedings{wang2026reasoningguard,
  title = {{ReasoningGuard: Safeguarding Large Reasoning Models with Inference-time Safety Aha Moments}},
  author = {Yuquan Wang, Mi Zhang, Yining Wang, Geng Hong, Mi Wen, Xiaoyu You and Min Yang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1453},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1453}
}

References

61 references, 29 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marah Abdin, Sahaj Agarwal, Ahmed Awadallah, Vidhisha Balachandran, Harkirat Behl, Lingjiao Chen, Gustavo de Rosa, Suriya Gunasekar, Mojan Javaheripi, Neel Joshi, Piero Kauffmann, Yash Lara, Caio César Teodoro Mendes, Arindam Mitra, Besmira Nushi, Dimitris Papailiopoulos, Olli Saarikivi, Shital Shah… find
  2. Gabriel Alon and Michael Kamfonas. 2023. Detecting language model attacks with perplexity. Preprint, arXiv:2308.14132.
  3. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. 2024. Jailbreaking black box large language models in twenty queries. Preprint, arXiv:2310.08419.
  4. Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. 2025. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models. Preprint, arXiv:2503.09567.
  5. Google DeepMind. 2025a. Gemini 2.5: Our most intelligent ai model. find
  6. Google DeepMind. 2025b. Gemini flash thinking. Accessed: 2025-03-15. find
  7. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  8. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021a. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR).
  9. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021b. Measuring mathematical problem solving with the MATH dataset. In NeurIPS.
  10. Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. 2025. Vision-r1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749. find
  11. Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. 2023. Baseline defenses for adversarial attacks against aligned language models. Preprint, arXiv:2309.00614.
  12. Wonje Jeung, Sangyeon Yoon, Minsuk Kahng, and Albert No. 2025. Safepath: Preventing harmful reasoning in chain-of-thought via early alignment. arXiv preprint arXiv:2505.14667. — [arXiv:2505.14667](https://arxiv.org/abs/2505.14667) find
  13. Fengqing Jiang, Zhangchen Xu, Yuetai Li, Luyao Niu, Zhen Xiang, Bo Li, Bill Yuchen Lin, and Radha Poovendran. 2025. Safechain: Safety of language models with long chain-of-thought reasoning capabilities. Preprint, arXiv:2502.12025. find
  14. Team Kimi, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, and 1 others. 2025. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599. — Kimi k1.5: Scaling Reinforcement Learning with LLMs
  15. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199– 22213.
  16. Martin Kuo, Jianyi Zhang, Aolin Ding, Qinsi Wang, Louis DiValentin, Yujia Bao, Wei Wei, Hai Li, and Yiran Chen. 2025. H-cot: Hijacking the chain-of-thought safety reasoning mechanism to jailbreak large reasoning models, including openai o1/o3, deepseek-r1, and gemini 2.0 flash thinking. Preprint, ar… — [arXiv:2502.12893](https://arxiv.org/abs/2502.12893) find
  17. Zenon Lamprou, Frank Polick, and Yashar Moshfeghi. 2025. Aligning brain activity with advanced transformer models: Exploring the role of punctuation in semantic processing. Preprint, arXiv:2501.06278. — [arXiv:2501.06278](https://arxiv.org/abs/2501.06278) find
  18. Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, and Bo Han. 2024. Deepinception: Hypnotize large language model to be jailbreaker. Preprint, arXiv:2311.03191.
  19. Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, Yingying Zhang, Fei Yin, Jiahua Dong, Zhiwei Li, Bao-Long Bi, Ling-Rui Mei, Junfeng Fang, Zhijiang Guo, Le Song, and Cheng-Lin Liu. 2025. From system 1 to system 2…
  20. Jiacheng Liang, Tanqiu Jiang, Yuhui Wang, Rongyi Zhu, Fenglong Ma, and Ting Wang. 2025. Autoran: Automated hijacking of safety reasoning in large reasoning models. Preprint, arXiv:2505.10846. find
  21. Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, Kailong Wang, and Yang Liu. 2024. Jailbreaking chatgpt via prompt engineering: An empirical study. Preprint, arXiv:2305.13860.
  22. Yue Liu, Hongcheng Gao, Shengfang Zhai, Xia Jun, Tianyi Wu, Zhiwei Xue, Yulin Chen, Kenji Kawaguchi, Jiaheng Zhang, and Bryan Hooi. 2025. Guardreasoner: Towards reasoning-based llm safeguards. arXiv preprint arXiv:2501.18492. — [arXiv:2501.18492](https://arxiv.org/abs/2501.18492) find
  23. AI @ Meta Llama Team. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.
  24. Sara Vera Marjanović, Arkil Patel, Vaibhav Adlakha, Milad Aghajohari, Parishad BehnamGhader, Mehar Bhatia, Aditi Khandelwal, Austin Kraft, Benno Krojer, Xing Han Lù, Nicholas Meade, Dongchan Shin, Amirhossein Kazemnejad, Gaurav Kamath, Marius Mosbach, Karolina Stańczak, and Siva Reddy. 2025. Deeps… find
  25. William Marslen-Wilson, Lorraine Tyler, and Mark Seidenberg. 1978. Sentence processing and the clause boundary, pages 219–246. find
  26. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393.
  27. Jianmo Ni, Gustavo Hernández Ábrego, Noah Constant, Ji Ma, Keith B. Hall, Daniel Cer, and Yinfei Yang. 2021. Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models. Preprint, arXiv:2108.08877.
  28. OpenAI. 2024a. Hello GPT-4o. find
  29. OpenAI. 2024b. Learning to reason with llms. https://openai.com/index/ learning-to-reason-with-llms. ↗
  30. Cheng Wang, Yue Liu, Baolong Li, Duzhen Zhang, Zhongzhi Li, and Junfeng Fang. 2025a. Safety in large reasoning models: A survey. Preprint, arXiv:2504.17704. find
  31. Yining Wang, Mi Zhang, Junjie Sun, Chenyue Wang, Min Yang, Hui Xue, Jialing Tao, Ranjie Duan, and Jiexi Liu. 2025b. Mirage in the eyes: Hallucination attack on multi-modal large language models with only attention sink. Preprint, arXiv:2501.15269. — [arXiv:2501.15269](https://arxiv.org/abs/2501.15269) find
  32. OpenAI. 2025. Introducing openai o3 and o4-mini. https://openai.com/index/ introducing-o3-and-o4-mini/. ↗
  33. Zijun Wang, Haoqin Tu, Jieru Mei, Bingchen Zhao, Yisen Wang, and Cihang Xie. 2024. Attngcg: Enhancing jailbreaking attacks on llms with attention manipulation. Preprint, arXiv:2410.09040. — [arXiv:2410.09040](https://arxiv.org/abs/2410.09040) find
  34. Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2024. Fine-tuning aligned language models compromises safety, even when users do not intend to! In The Twelfth International Conference on Learning Representations.
  35. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-thought prompting elicits reasoning in large language models. Preprint, arXiv:2201.11903.
  36. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. 2023. Gpqa: A graduate-level google-proof q&a benchmark. Preprint, arXiv:2311.12022.
  37. Alexander Robey, Eric Wong, Hamed Hassani, and George J Pappas. 2023. Smoothllm: Defending large language models against jailbreaking attacks. arXiv preprint arXiv:2310.03684.
  38. Paul Röttger, Hannah Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. 2024. XSTest: A test suite for identifying exaggerated safety behaviours in large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational L…
  39. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. Preprint, arXiv:2402.03300.
  40. Charles Spearman. 1961. The proof and measurement of association between two things. find
  41. Mingjie Sun, Xinlei Chen, J Zico Kolter, and Zhuang Liu. 2024. Massive activations in large language models. In First Conference on Language Modeling. find
  42. Qwen Team. 2024. Qwen2.5: A party of foundation models. find
  43. Qwen Team. 2025a. Qwen3 technical report. Preprint, arXiv:2505.09388.
  44. Qwen Team. 2025b. Qwq-32b: Embracing the power of reinforcement learning. find
  45. Wikipedia contributors. 2025. Spearman’s rank correlation coefficient. Wikipedia. https://en.wikipedia.org/wiki/Spearman% 27s_rank_correlation_coefficient. ↗
  46. Tong Wu, Chong Xiang, Jiachen T. Wang, and Prateek Mittal. 2025. Effectively controlling reasoning models through thinking intervention. Preprint, arXiv:2503.24370. — [arXiv:2503.24370](https://arxiv.org/abs/2503.24370) find
  47. Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2024a. Efficient streaming language models with attention sinks. Preprint, arXiv:2309.17453. find
  48. Zeguan Xiao, Yan Yang, Guanhua Chen, and Yun Chen. 2024b. Distract large language models for automatic jailbreak attack. Preprint, arXiv:2403.08424. find
  49. Tinghao Xie, Xiangyu Qi, Yi Zeng, Yangsibo Huang, Udari Madhushani Sehwag, Kaixuan Huang, Luxi He, Boyi Wei, Dacheng Li, Ying Sheng, Ruoxi Jia, Bo Li, Kai Li, Danqi Chen, Peter Henderson, and Prateek Mittal. 2025. SORRY-bench: Systematically evaluating large language model safety refusal. In The Thi… find
  50. Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu. 2023. Defending chatgpt against jailbreak attack via self-reminders. Nature Machine Intelligence, 5(12):1486–1496. find
  51. Zhangchen Xu, Fengqing Jiang, Luyao Niu, Jinyuan Jia, Bill Yuchen Lin, and Radha Poovendran. 2024. Safedecoding: Defending against jailbreak attacks via safety-aware decoding. arXiv preprint arXiv:2402.08983.
  52. An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. 2024. Qwen2.5-math technical report: Toward mathematical expert model via self-improvement. a…
  53. Yang Yao, Xuan Tong, Ruofan Wang, Yixu Wang, Lu-jundong Li, Liang Liu, Yan Teng, and Yingchun Wang. 2025. A mousetrap: Fooling large reasoning models for jailbreak with chain of iterative chaos. arXiv preprint arXiv:2502.15806. find
  54. Zhongzhi Yu, Zheng Wang, Yonggan Fu, Huihong Shi, Khalid Shaikh, and Yingyan Celine Lin. 2024. Unveiling and harnessing hidden attention sinks: Enhancing large language models without training through attention calibration. Preprint, arXiv:2406.15765. — [arXiv:2406.15765](https://arxiv.org/abs/2406.15765) find
  55. Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. 2024. How Johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge AI safety by humanizing LLMs. ArXiv preprint, abs/2401.06373.
  56. Yichi Zhang, Zihao Zeng, Dongbai Li, Yao Huang, Zhijie Deng, and Yinpeng Dong. 2025. Realsafe-r1: Safety-aligned deepseek-r1 without compromising reasoning capability. arXiv preprint arXiv:2504.10081. — [arXiv:2504.10081](https://arxiv.org/abs/2504.10081) find
  57. Baihui Zheng, Boren Zheng, Kerui Cao, Yingshui Tan, Zhendong Liu, Weixun Wang, Jiaheng Liu, Jian Yang, Wenbo Su, Xiaoyong Zhu, Bo Zheng, and Kaifu Zhang. 2025. Beyond safe answers: A benchmark for evaluating true risk awareness in large reasoning models. Preprint, arXiv:2505.19690. — [arXiv:2505.19690](https://arxiv.org/abs/2505.19690) find
  58. Kaiwen Zhou, Chengzhi Liu, Xuandong Zhao, Shreed-har Jangam, Jayanth Srinivasa, Gaowen Liu, Dawn Song, and Xin Eric Wang. 2025a. The hidden risks of large reasoning models: A safety assessment of r1. Preprint, arXiv:2502.12659.
  59. Kaiwen Zhou, Xuandong Zhao, Jayanth Srinivasa, Gaowen Liu, Aosong Feng, Dawn Song, and Xin Eric Wang. 2025b. Safekey: Amplifying aha-moment insights for safety reasoning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 25407–25423.
  60. Junda Zhu, Lingyong Yan, Shuaiqiang Wang, Dawei Yin, and Lei Sha. 2025. Reasoning-to-defend: Safety-aware reasoning can defend large language models from jailbreaking. arXiv preprint arXiv:2502.12970.
  61. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. Preprint, arXiv:2307.15043.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-reasoningguard-safeguarding-large-reasoning-models/