Elamur Research⌕
☰
home / papers / 2026 / Red Teaming Large Reasoning Models

Red Teaming Large Reasoning Models

Paper · 2026 · Jiawei Chen, Yang Yang, Chao Yu et al.

Summary

Large Reasoning Models (LRMs) produce explicit chains of thought. This visible reasoning creates attack surfaces that standard LLM evaluations miss, and earlier benchmarks examined single threats in isolation without comparing LRMs to their base LLMs. The authors introduce Rt-LRM, a unified red-teaming benchmark that measures LRM trustworthiness along three axes: truthfulness, safety and efficiency. It covers two threat families. CoT-hijacking manipulates the reasoning trace directly. Prompt-induced impacts work indirectly, through jailbreak prompts or triggers that cause overthinking. The suite has 30 tasks built from newly created, refined and augmented datasets, along with a modular toolbox for reproducible scoring using accuracy, attack success rate and overthinking rate (OR). The authors tested 26 models and paired LRMs with their base LLMs. Most LRMs show weak trustworthiness and are often more fragile than LLMs when attacks target reasoning. Overthinking is widespread, and longer, harder reasoning tends to lower reliability. The work suggests that stronger reasoning does not automatically yield safer models and calls for defenses tailored to reasoning.

Key points

  • The 30 tasks split into 9 for truthfulness, 10 for safety and 11 for efficiency. Data comes from 6 datasets built from scratch, 4 refined existing sets and 9 sets augmented with extra prompts.
  • The 26 models include 22 open-source and 4 proprietary ones, grouped by training regime (SFT+RL, SFT-only, RL-only).
  • GLM-Z1-32B had an OR above 70% on every efficiency task. Claude-Sonnet-4 was among the most efficient models overall but hit 94% OR on task E.8.
  • Safety violations are high for many models; for example, DeepMath reached 94.29% on copyright-violation prompts. Claude-4 had the lowest violation rates.
  • SFT+RL models tended to be more truthful and safer, while RL-only models overthought less. The authors present this as correlational, not causal.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China333 works · 87,019 citations🎓Shanghai Jiao Tong UniversityShanghai · China149 works · 16,161 citations🎓Beihang UniversityBeijing · China65 works · 8,452 citations🎓East China Normal UniversityShanghai · China50 works · 2,174 citations🎓Tsinghua Shenzhen International Graduate SchoolShenzhen · China28 works · 3,035 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksBias, fairness & safety · 517 works

Cite

Cite as Jiawei Chen, Yang Yang, Chao Yu, Yu Tian, Zhi Cao, Xue Yang, Linghao Li, Hang Su and Zhaoxia Yin. 2026. Red Teaming Large Reasoning Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1034

@inproceedings{chen2026red,
  title = {{Red Teaming Large Reasoning Models}},
  author = {Jiawei Chen, Yang Yang, Chao Yu, Yu Tian, Zhi Cao, Xue Yang, Linghao Li, Hang Su and Zhaoxia Yin},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1034},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1034}
}

References

63 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and 1 others. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732.
  2. Jiawei Chen, Xiao Yang, Zhengwei Fang, Yu Tian, Yinpeng Dong, Zhaoxia Yin, and Hang Su. 2024a. Auto-breach: Universal and adaptive jailbreaking with efficient wordplay-guided optimization. arXiv preprint arXiv:2405.19668. — [arXiv:2405.19668](https://arxiv.org/abs/2405.19668) find
  3. Jiawei Chen, Xiao Yang, Heng Yin, Mingzhi Ma, Bihui Chen, Jianteng Peng, Yandong Guo, Zhaoxia Yin, and Hang Su. 2023. Advfas: A robust face anti-spoofing framework against adversarial examples. Computer Vision and Image Understanding, 235:103779. find
  4. Yulong Chen, Yang Liu, Jianhao Yan, Xuefeng Bai, Ming Zhong, Yinghao Yang, Ziyi Yang, Chenguang Zhu, and Yue Zhang. 2024b. See what llms cannot answer: A self-challenge framework for uncovering llm weaknesses. arXiv preprint arXiv:2408.08978. — [arXiv:2408.08978](https://arxiv.org/abs/2408.08978) find
  5. Yu Cui, Bryan Hooi, Yujun Cai, and Yiwei Wang. 2025. Process or result? manipulated ending tokens can mislead reasoning llms to ignore the correct reasoning steps. arXiv preprint arXiv:2503.19326. — [arXiv:2503.19326](https://arxiv.org/abs/2503.19326) find
  6. Karin de Langis, Jong Inn Park, Bin Hu, Khanh Chi Le, Andreas Schramm, Michael C Mensink, Andrew Elfenbein, and Dongyeop Kang. 2025. A framework for robust cognitive evaluation of llms. arXiv preprint arXiv:2504.02789. — [arXiv:2504.02789](https://arxiv.org/abs/2504.02789) find
  7. DeepSeek-AI. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. Preprint, arXiv:2501.12948.
  8. Zhichen Dong, Zhanhui Zhou, Chao Yang, Jing Shao, and Yu Qiao. 2024. Attacks, defenses and evaluations for llm conversation safety: A survey. arXiv preprint arXiv:2402.09283. find
  9. Junfeng Fang, Yukai Wang, Ruipeng Wang, Zijun Yao, Kun Wang, An Zhang, Xiang Wang, and Tat-Seng Chua. 2025. Safemlrm: Demystifying safety in multi-modal large reasoning models. arXiv preprint arXiv:2504.08813. — [arXiv:2504.08813](https://arxiv.org/abs/2504.08813) find
  10. Aaron Fanous, Jacob Goldberg, Ank A Agarwal, Joanna Lin, Anson Zhou, Roxana Daneshjou, and Sanmi Koyejo. 2025. Syceval: Evaluating llm sycophancy. arXiv preprint arXiv:2502.08177. — [arXiv:2502.08177](https://arxiv.org/abs/2502.08177) find
  11. Melody Y Guan, Manas Joglekar, Eric Wallace, Saachi Jain, Boaz Barak, Alec Helyar, Rachel Dias, Andrea Vallone, Hongyu Ren, Jason Wei, and 1 others. 2024. Deliberative alignment: Reasoning enables safer language models. arXiv preprint arXiv:2412.16339.
  12. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  13. Lukas Helff, Felix Friedrich, Manuel Brack, Patrick Schramowski, and Kristian Kersting. 2024. Llava-guard: Vlm-based safeguard for vision dataset curation and safety assessment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8322–8326. find
  14. Xiangdong Hu, Yangyang Jiang, Qin Hu, and Xiaojun Jia. 2026. Gambit: A gamified jailbreak framework for multimodal large language models. arXiv preprint arXiv:2601.03416.
  15. Peichun Hua, Hao Li, Shanghao Shi, Zhiyuan Yu, and Ning Zhang. 2025. Rethinking jailbreak detection of large vision language models with representational contrastive scoring. arXiv preprint arXiv:2512.12069.
  16. Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, Zachary Yahn, Yichang Xu, and Ling Liu. 2025. Safety tax: Safety alignment makes your large reasoning models less reasonable. arXiv preprint arXiv:2503.00555. — [arXiv:2503.00555](https://arxiv.org/abs/2503.00555) find
  17. Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, and 1 others. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186. — Qwen2.5-Coder Technical Report
  18. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720.
  19. Hongru Ji, Yuyin Fan, Meng Zhao, Xianghua Li, Lianwei Wu, and Chao Gao. 2026. Stride-ed: A strategy-grounded stepwise reasoning framework for empathetic dialogue systems. Preprint, arXiv:2604.07100.
  20. Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. 2023a. Beavertails: Towards improved safety alignment of llm via a human-preference dataset. Advances in Neural Information Processing Systems, 36:24678–24704.
  21. Martin Kuo, Jianyi Zhang, Aolin Ding, Qinsi Wang, Louis DiValentin, Yujia Bao, Wei Wei, Hai Li, and Yiran Chen. 2025. H-cot: Hijacking the chain-of-thought safety reasoning mechanism to jailbreak large reasoning models, including openai o1/o3, deepseek-r1, and gemini 2.0 flash thinking. arXiv prepri… — [arXiv:2502.12893](https://arxiv.org/abs/2502.12893) find
  22. Shalom Lappin. 2024. Assessing the strengths and weaknesses of large language models. Journal of Logic, Language and Information, 33(1):9–20. find
  23. Xinyu Li, Tianjin Huang, Ronghui Mu, Xiaowei Huang, and Gaojie Jin. 2025. Pot: Inducing overthinking in llms via black-box iterative optimization. arXiv preprint arXiv:2508.19277. — [arXiv:2508.19277](https://arxiv.org/abs/2508.19277) find
  24. Yu Li, Tian Lan, and Zhengling Qi. 2026a. When right meets wrong: Bilateral context conditioning with reward-confidence correction for grpo. arXiv preprint arXiv:2603.13134. — [arXiv:2603.13134](https://arxiv.org/abs/2603.13134) find
  25. Yu Li, Rui Miao, Zhengling Qi, and Tian Lan. 2026b. Arise: Agent reasoning with intrinsic skill evolution in hierarchical reinforcement learning. arXiv preprint arXiv:2603.16060. — [arXiv:2603.16060](https://arxiv.org/abs/2603.16060) find
  26. Jiacheng Liang, Tanqiu Jiang, Yuhui Wang, Rongyi Zhu, Fenglong Ma, and Ting Wang. 2025. Autoran: Weak-to-strong jailbreaking of large reasoning models. arXiv preprint arXiv:2505.10846. — [arXiv:2505.10846](https://arxiv.org/abs/2505.10846) find
  27. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, and 1 others. 2022. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110. find
  28. Leon Lin, Hannah Brown, Kenji Kawaguchi, and Michael Shieh. 2025. Single character perturbations break llm alignment. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 27473–27481. find
  29. Lin Ling, Fazle Rabbi, Song Wang, and Jinqiu Yang. 2025. Bias unveiled: Investigating social bias in llm-generated code. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 27491–27499. find
  30. Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung. 2023b. Towards mitigating llm hallucination via self reflection. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1827–1843.
  31. Shuaitong Liu, Renjue Li, Lijia Yu, Lijun Zhang, Zhiming Liu, and Gaojie Jin. 2025. Badthink: Triggered overthinking attacks on chain-of-thought reasoning in large language models. arXiv preprint arXiv:2511.10714. — [arXiv:2511.10714](https://arxiv.org/abs/2511.10714) find
  32. Fengqing Jiang, Zhangchen Xu, Yuetai Li, Luyao Niu, Zhen Xiang, Bo Li, Bill Yuchen Lin, and Radha Poovendran. 2025. Safechain: Safety of language models with long chain-of-thought reasoning capabilities. arXiv preprint arXiv:2502.12025. find
  33. Shuyuan Liu, Jiawei Chen, Shouwei Ruan, Hang Su, and Zhaoxia Yin. 2024. Exploring the robustness of decision-level through adversarial attacks on llm-based embodied models. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 8120–8128. find
  34. Aisha Khatun and Daniel G Brown. 2024. Trutheval: A dataset to evaluate llm truthfulness and reliability. arXiv preprint arXiv:2406.01855. — [arXiv:2406.01855](https://arxiv.org/abs/2406.01855) find
  35. Yang Liu, Yuanshun Yao, Jean-Francois Ton, Xiaoying Zhang, Ruocheng Guo, Hao Cheng, Yegor Klochkov, Muhammad Faaiz Taufiq, and Hang Li. 2023. Trustworthy llms: a survey and guideline for evaluating large language models’ alignment. arXiv preprint arXiv:2308.05374.
  36. Yu Tian, Xiao Yang, Jingyuan Zhang, Yinpeng Dong, and Hang Su. 2023. Evil geniuses: Delving into the safety of llm-based agents. arXiv preprint arXiv:2311.11855. — [arXiv:2311.11855](https://arxiv.org/abs/2311.11855) find
  37. Yiding Liu, Weixue Lu, Suqi Cheng, Daiting Shi, Shuaiqiang Wang, Zhicong Cheng, and Dawei Yin. 2021. Pre-trained language model for web-scale retrieval in baidu search. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 3365–3375. find
  38. Cheng Wang, Yue Liu, Baolong Bi, Duzhen Zhang, Zhong-Zhi Li, Yingwei Ma, Yufei He, Shengju Yu, Xinfeng Li, Junfeng Fang, and 1 others. 2025a. Safety in large reasoning models: A survey. arXiv preprint arXiv:2504.17704. find
  39. Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, and 1 others. 2025. Deepscaler: Surpassing o1-preview with a 1.5 b model by scaling rl. Notion Blog. find
  40. Maximilian Mozes, Xuanli He, Bennett Kleinberg, and Lewis D Griffin. 2023. Use of llms for illicit purposes: Threats, prevention measures, and vulnerabilities. arXiv preprint arXiv:2308.12833. — [arXiv:2308.12833](https://arxiv.org/abs/2308.12833) find
  41. Meghana Rajeev, Rajkumar Ramamurthy, Prapti Trivedi, Vikas Yadav, Oluwanifemi Bamgbose, Sathwik Tejaswi Madhusudan, James Zou, and Nazneen Rajani. 2025. Cats confuse reasoning llm: Query agnostic adversarial triggers for reasoning models. arXiv preprint arXiv:2503.01781. — [arXiv:2503.01781](https://arxiv.org/abs/2503.01781) find
  42. Qibing Ren, Hao Li, Dongrui Liu, Zhanxu Xie, Xiaoya Lu, Yu Qiao, Lei Sha, Junchi Yan, Lizhuang Ma, and Jing Shao. 2024. Derail yourself: Multi-turn llm jailbreak attack through self-discovered clues. CoRR. find
  43. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  44. Yichen Shi, Yuhao Gao, Yingxin Lai, Hongyang Wang, Jun Feng, Lei He, Jun Wan, Changsheng Chen, Zitong Yu, and Xiaochun Cao. 2025. Shield: An evaluation benchmark for face spoofing and forgery detection with multimodal large language models. Visual Intelligence, 3(1):9. find
  45. Wai Man Si, Mingjie Li, Michael Backes, and Yang Zhang. 2025. Excessive reasoning attack on reasoning llms. arXiv preprint arXiv:2506.14374. — [arXiv:2506.14374](https://arxiv.org/abs/2506.14374) find
  46. Chengbing Wang, Yang Zhang, Wenjie Wang, Xiaoyan Zhao, Fuli Feng, Xiangnan He, and Tat-Seng Chua. 2025b. Think-while-generating: On-the-fly reasoning for personalized long-form generation. arXiv preprint arXiv:2512.06690. — [arXiv:2512.06690](https://arxiv.org/abs/2512.06690) find
  47. Hanchen Wang, Tianfan Fu, Yuanqi Du, Wenhao Gao, Kexin Huang, Ziming Liu, Payal Chandak, Shengchao Liu, Peter Van Katwyk, Andreea Deac, and 1 others. 2023. Scientific discovery in the age of artificial intelligence. Nature, 620(7972):47–60. find
  48. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36:80079–80110.
  49. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  50. Xiao Yang, Jiawei Chen, Jun Luo, Zhengwei Fang, Yinpeng Dong, Hang Su, and Jun Zhu. 2025. Mla-trust: Benchmarking trustworthiness of multimodal llm agents in gui environments. arXiv preprint arXiv:2506.01616. — [arXiv:2506.01616](https://arxiv.org/abs/2506.01616) find
  51. Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. 2025. Limo: Less is more for reasoning. arXiv preprint arXiv:2502.03387.
  52. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, and 16 others. 2025. Dapo: An open-source llm reinforcement learning sys…
  53. Zhe Su, Xuhui Zhou, Sanketh Rangreji, Anubha Kabra, Julia Mendelsohn, Faeze Brahman, and Maarten Sap. 2024. Ai-liedar: Examine the trade-off between utility and truthfulness in llm agents. arXiv preprint arXiv:2409.09013. find
  54. Wojciech Zaremba, Evgenia Nitishinskaya, Boaz Barak, Stephanie Lin, Sam Toyer, Yaodong Yu, Rachel Dias, Eric Wallace, Kai Xiao, Johannes Heidecke, and 1 others. 2025. Trading inference-time compute for adversarial robustness. arXiv preprint arXiv:2501.18841. find
  55. Wrick Talukdar and Anjanava Biswas. 2024. Improving large language model (llm) fidelity through context-aware grounding: A systematic approach to reliability and veracity. arXiv preprint arXiv:2408.04023. — [arXiv:2408.04023](https://arxiv.org/abs/2408.04023) find
  56. Xinyi Zeng, Yuying Shang, Jiawei Chen, Jingyuan Zhang, and Yu Tian. 2024. Root defence strategies: Ensuring safety of llm at the decoding level. arXiv preprint arXiv:2410.06809. — [arXiv:2410.06809](https://arxiv.org/abs/2410.06809) find
  57. Jianyi Zhang, Hao Frank Yang, Ang Li, Xin Guo, Pu Wang, Haiming Wang, Yiran Chen, and Hai Li. 2024. Mllm-fl: Multimodal large language model assisted federated learning on heterogeneous and long-tailed data. arXiv e-prints, pages arXiv–2409. find
  58. Mohan Zhang, Yihua Zhang, Jinghan Jia, Zhangyang Wang, Sijia Liu, and Tianlong Chen. 2025a. One token embedding is enough to deadlock your large reasoning model. arXiv preprint arXiv:2510.15965. — [arXiv:2510.15965](https://arxiv.org/abs/2510.15965) find
  59. Nan Zhang, Yusen Zhang, Prasenjit Mitra, and Rui Zhang. 2025b. When reasoning meets compression: Benchmarking compressed large reasoning models on complex reasoning tasks. arXiv preprint arXiv:2504.02010. — [arXiv:2504.02010](https://arxiv.org/abs/2504.02010) find
  60. Zhexin Zhang, Xian Qi Loye, Victor Shea-Jay Huang, Junxiao Yang, Qi Zhu, Shiyao Cui, Fei Mi, Lifeng Shang, Yingkang Wang, Hongning Wang, and 1 others. 2025c. How should we enhance the safety of large reasoning models: An empirical study. arXiv preprint arXiv:2505.15404.
  61. Baihui Zheng, Boren Zheng, Kerui Cao, Yingshui Tan, Zhendong Liu, Weixun Wang, Jiaheng Liu, Jian Yang, Wenbo Su, Xiaoyong Zhu, and 1 others. 2025. Beyond safe answers: A benchmark for evaluating true risk awareness in large reasoning models. arXiv preprint arXiv:2505.19690. — [arXiv:2505.19690](https://arxiv.org/abs/2505.19690) find
  62. Lifan Zheng, Jiawei Chen, Qinghong Yin, Jingyuan Zhang, Xinyi Zeng, and Yu Tian. 2026. Rethinking the reliability of multi-agent system: A perspective from byzantine fault tolerance. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 35012–35020. find
  63. Kaiwen Zhou, Xuandong Zhao, Gaowen Liu, Jayanth Srinivasa, Aosong Feng, Dawn Song, and Xin Eric Wang. 2025. Safekey: Amplifying aha-moment insights for safety reasoning. arXiv preprint arXiv:2505.16186.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/chen-2026-red-teaming-large-reasoning-models/