Beyond Surface-Level Detection: Towards Cognitive-Driven Defense Against Jailbreak Attacks via Meta-Operations Reasoning
Paper · 2026 · Rui Pu, Chaozhuo Li, Rui Ha et al.
Summary
Jailbreak defenses for LLMs usually match known harmful patterns or apply fixed rules, so they often fail against new attack styles. The authors' own measurements show attack success rising for jailbreaks published in 2025. The paper proposes Cognitive-Driven Defense (CDD), which treats a jailbreak prompt as a combination of a small set of "meta-operations." These are elementary disguising moves such as replacement, translation, or reversal. The meta-operations were derived from thirteen jailbreak techniques using Interpersonal Deception Theory. A dataset links each prompt to its meta-operations, a reasoning chain and a safe response. The reasoning chain first reads the prompt as a whole, then inspects suspicious parts locally. SFT on this data teaches the model to recognise known manipulations. A modified RL algorithm, EG-GRPO, then adds an entropy-based reward that favours finding new meta-operations. CDD reaches state-of-the-art defense results and generalises to unseen attacks at modest inference cost. This matters because defenses no longer have to be patched for every new attack.
Key points
- The taxonomy has 21 meta-operations (fundamental and advanced) derived from 13 jailbreak techniques. Each training sample is a triplet of prompt, reasoning chain and safe answer.
- EG-GRPO combines four rewards: format, accuracy, entropy-based exploration and logic consistency. The exploration reward gives 0.5 for a newly generated meta-operation and −0.2 for an already known one.
- Ablation on Qwen2.5-7B: undefended ASR on ReNeLLM is 93.92% and on CodeAttack 78.66%. SFT alone lowers these to 3.21% and 4.56%. Full CDD lowers ASR to 0.00% on the seen attacks (PAIR, ReNeLLM, CodeAttack). On unseen attacks such as Flip and SeqAR, full CDD reaches 2.10–4.05%.
- Usability on Llama3.1-8B / Qwen2.5-7B: AlpacaEval WinRate is 28.96% / 23.78%, against 30.06% / 24.58% with no defense. Or-Bench refusal is 6.89% / 8.15%. STAIR scores best on usability.
- Overhead measured by ATGR is small: 1.121× on Llama3.1-8B-Instruct, 1.135× on Qwen2.5-7B-Instruct and 1.176× on DeepSeek-R1-Distill-Qwen-7B. Stated limitations are partly manual annotation and no evaluation of multi-turn or multimodal jailbreaks.
Institutions
Fields, topics and concepts
📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works
🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksBias, fairness & safety · 517 works
Cite
Cite as Rui Pu, Chaozhuo Li, Rui Ha, Litian Zhang, Lirong Qiu and Xi Zhang. 2026. Beyond Surface-Level Detection: Towards Cognitive-Driven Defense Against Jailbreak Attacks via Meta-Operations Reasoning. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).
@inproceedings{pu2026beyond,
title = {{Beyond Surface-Level Detection: Towards Cognitive-Driven Defense Against Jailbreak Attacks via Meta-Operations Reasoning}},
author = {Rui Pu, Chaozhuo Li, Rui Ha, Litian Zhang, Lirong Qiu and Xi Zhang},
booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
year = {2026},
doi = {10.18653/v1/2026.acl-long.125},
url = {https://doi.org/10.18653/v1/2026.acl-long.125}
}References
- Gabriel Alon and Michael Kamfonas. 2023. Detecting language model attacks with perplexity. arXiv preprint arXiv:2308.14132.
- John Robert Anderson and Jane Crawford. 1995. Cognitive psychology and its implications. find
- Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. 2025. Jailbreaking leading safety-aligned llms with simple adaptive attacks. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net. find
- David B Buller and Judee K Burgoon. 1996. Interpersonal deception theory. Communication theory, 6(3):203–242. find
- Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. 2023. Jailbreaking black box large language models in twenty queries. CoRR, abs/2310.08419.
- Zhaorun Chen, Zhuokai Zhao, Wenjie Qu, Zichen Wen, Zhiguang Han, Zhihong Zhu, Jiaheng Zhang, and Huaxiu Yao. 2024. PANDORA: Detailed LLM jailbreaking via collaborated phishing agents with decomposed reasoning. In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models. find
- Daixuan Cheng, Shaohan Huang, Xuekai Zhu, Bo Dai, Wayne Xin Zhao, Zhenliang Zhang, and Furu Wei. 2025. Reasoning with exploration: An entropy perspective. arXiv preprint arXiv:2506.14758. find
- Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. 2025. Sft memorizes, rl generalizes: A comparative study of foundation model post-training. arXiv preprint arXiv:2501.17161.
- Justin Cui, Wei-Lin Chiang, Ion Stoica, and Cho-Jui Hsieh. 2024. Or-bench: An over-refusal benchmark for large language models. arXiv preprint arXiv:2405.20947.
- Peng Ding, Jun Kuang, Dan Ma, Xuezhi Cao, Yunsen Xian, Jiajun Chen, and Shujian Huang. 2023. A wolf in sheep’s clothing: Generalized nested jailbreak prompts can fool large language models easily. CoRR, abs/2311.08268. find
- Zhichen Dong, Zhanhui Zhou, Chao Yang, Jing Shao, and Yu Qiao. 2024. Attacks, defenses and evaluations for LLM conversation safety: A survey. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: L… find
- Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
- Yann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Alpacafarm: A simulation framework for methods that learn from human feedback. In Advances in Neural Information Processing Systems 36: Annual Conferen…
- Jonathan St BT Evans. 2003. In two minds: dual-process accounts of reasoning. Trends in cognitive sciences, 7(10):454–459. find
- Melody Y Guan, Manas Joglekar, Eric Wallace, Saachi Jain, Boaz Barak, Alec Heylar, Rachel Dias, Andrea Vallone, Hongyu Ren, Jason Wei, et al. 2024. Deliberative alignment: Reasoning enables safer language models. arXiv preprint arXiv:2412.16339.
- Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025a. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
- Songyang Liu, Chaozhuo Li, Jiameng Qiu, Xi Zhang, Feiran Huang, Litian Zhang, Yiming Hei, and Philip S Yu. 2025a. The scales of justitia: A comprehensive survey on safety evaluation of llms. arXiv preprint arXiv:2506.11094. — [arXiv:2506.11094](https://arxiv.org/abs/2506.11094) find
- Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025b. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
- Songyang Liu, Chaozhuo Li, Chenxu Wang, Jinyu Hou, Zejian Chen, Litian Zhang, Zheng Liu, Qiwei Ye, Yiming Hei, Xi Zhang, et al. 2026. Clawkeeper: Comprehensive safety protection for openclaw agents through skills, plugins, and watchers. arXiv preprint arXiv:2603.24414. — [arXiv:2603.24414](https://arxiv.org/abs/2603.24414) find
- Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720.
- Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. 2023. Baseline defenses for adversarial attacks against aligned language models. ArXiv preprint, abs/2309.00614.
- Fengqing Jiang, Zhangchen Xu, Yuetai Li, Luyao Niu, Zhen Xiang, Bo Li, Bill Yuchen Lin, and Radha Poovendran. 2025. Safechain: Safety of language models with long chain-of-thought reasoning capabilities. In ICLR 2025 Workshop on Bidirectional Human-AI Alignment. find
- Fengqing Jiang, Zhangchen Xu, Luyao Niu, Zhen Xiang, Bhaskar Ramasubramanian, Bo Li, and Radha Poovendran. 2024. Artprompt: ASCII art-based jailbreak attacks against aligned llms. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL…
- Mintong Kang and Bo Li. 2025. R 2 -guard: Robust reasoning enabled LLM guardrail via knowledge-enhanced logical reasoning. In The Thirteenth International Conference on Learning Representations. find
- Chaozhuo Li, Pengbo Wang, Chenxu Wang, Litian Zhang, Zheng Liu, Qiwei Ye, Yuanbo Xu, Feiran Huang, Xi Zhang, and Philip S Yu. 2025. Loki’s dance of illusions: A comprehensive survey of hallucination in large language models. arXiv preprint arXiv:2507.02870. — [arXiv:2507.02870](https://arxiv.org/abs/2507.02870) find
- Tong Liu, Yingjie Zhang, Zhe Zhao, Yinpeng Dong, Guozhu Meng, and Kai Chen. 2024a. Making them ask and answer: Jailbreaking large language models in few queries via disguise and reconstruction. In 33rd USENIX Security Symposium, USENIX Security 2024, Philadelphia, PA, USA, August 14-16, 2024. USENIX… find
- Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2024b. AutoDAN: Generating stealthy jailbreak prompts on aligned large language models. In The Twelfth International Conference on Learning Representations.
- Yue Liu, Hongcheng Gao, Shengfang Zhai, Jun Xia, Tianyi Wu, Zhiwei Xue, Yulin Chen, Kenji Kawaguchi, Jiaheng Zhang, and Bryan Hooi. 2025b. Guardreasoner: Towards reasoning-based llm safeguards. arXiv preprint arXiv:2501.18492. — [arXiv:2501.18492](https://arxiv.org/abs/2501.18492) find
- Yue Liu, Xiaoxin He, Miao Xiong, Jinlan Fu, Shumin Deng, and Bryan Hooi. 2024c. Flipattack: Jailbreak llms via flipping. arXiv preprint arXiv:2410.02832. — [arXiv:2410.02832](https://arxiv.org/abs/2410.02832) find
- Huijie Lv, Xiao Wang, Yuansen Zhang, Caishuang Huang, Shihan Dou, Junjie Ye, Tao Gui, Qi Zhang, and Xuanjing Huang. 2024. Codechameleon: Personalized encryption framework for jailbreaking large language models. arXiv preprint arXiv:2402.16717. — [arXiv:2402.16717](https://arxiv.org/abs/2402.16717) find
- Skywork o1 Team. 2024. Skywork-o1 open series. https://huggingface.co/Skywork. ↗
- Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, and Peter Henderson. 2025. Safety alignment should be made more than just a few tokens deep. In The Thirteenth International Conference on Learning Representations. find
- Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. 2024. Drattack: Prompt decomposition and reconstruction makes powerful LLM jailbreakers. CoRR, abs/2402.16914. — [arXiv:2402.16914](https://arxiv.org/abs/2402.16914) find
- Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2023. Fine-tuning aligned language models compromises safety, even when users do not intend to! CoRR, abs/2310.03693.
- Shi Lin, Hongming Yang, Rongchang Li, Xun Wang, Changting Lin, Wenpeng Xing, and Meng Han. 2024. Llms can be dangerous reasoners: Analyzing-based jailbreak attack on large language models. arXiv preprint arXiv:2407.16205. — [arXiv:2407.16205](https://arxiv.org/abs/2407.16205) find
- Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, et al. 2025. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. find
- Venkat K. Raman. 1997. Atomic and ion collisions in solids and at surfaces. theory, simulation, and applications. roger smith, mario jakas, dave ashworth, bob oven, mark bowyer, ivan chakarov, and roger webb. cambridge university press, 309 pp, 1997, ISBN 0-521-44022. J. Chem. Inf. Comput. Sci., 37(… find
- Qibing Ren, Chang Gao, Jing Shao, Junchi Yan, Xin Tan, Wai Lam, and Lizhuang Ma. 2024. Codeattack: Revealing safety generalization challenges of large language models via code completion. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, A… find
- Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2024. "do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, CCS 2024, Salt Lake City, UT…
- Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023. Stanford alpaca: an instruction-following llama model (2023). find
- Qwen Team. 2024. Qwq: Reflect deeply on the boundaries of the unknown. find
- Chenxu Wang, Chaozhuo Li, Songyang Liu, Zejian Chen, Jinyu Hou, Ji Qi, Rui Li, Litian Zhang, Qiwei Ye, Zheng Liu, et al. 2026. The devil behind moltbook: Anthropic safety is always vanishing in self-evolving ai societies. arXiv preprint arXiv:2602.09877. — [arXiv:2602.09877](https://arxiv.org/abs/2602.09877) find
- Xunguang Wang, Wenxuan Wang, Zhenlan Ji, Zongjie Li, Pingchuan Ma, Daoyuan Wu, and Shuai Wang. 2025. Stshield: Single-token sentinel for real-time jailbreak detection in large language models. CoRR, abs/2503.17932. — [arXiv:2503.17932](https://arxiv.org/abs/2503.17932) find
- Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems.
- Tianyu Wu, Lingrui Mei, Ruibin Yuan, Lujun Li, Wei Xue, and Yike Guo. 2024. You know what i’m saying: Jailbreak attack via implicit reference. arXiv preprint arXiv:2410.03857. — [arXiv:2410.03857](https://arxiv.org/abs/2410.03857) find
- Zihui Wu, Haichang Gao, Jianping He, and Ping Wang. 2025. The dark side of function calling: Pathways to jailbreaking large language models. In Proceedings of the 31st International Conference on Computational Linguistics, COLING 2025, Abu Dhabi, UAE, January 19-24, 2025, pages 584–592. Association…
- Shijie Xia, Yiwei Qin, Xuefeng Li, Yan Ma, Run-Ze Fan, Steffi Chern, Haoyang Zou, Fan Zhou, Xiangkun Hu, Jiahe Jin, et al. 2025. Generative ai act ii: Test time scaling drives cognition engineering. arXiv preprint arXiv:2504.13828. — [arXiv:2504.13828](https://arxiv.org/abs/2504.13828) find
- Zeguan Xiao, Yan Yang, Guanhua Chen, and Yun Chen. 2024. Distract large language models for automatic jailbreak attack. arXiv preprint arXiv:2403.08424. find
- Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu. 2023a. Defending chatgpt against jailbreak attack via self-reminders. Nat. Mac. Intell., 5(12):1486–1496. find
- Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu. 2023b. Defending chatgpt against jailbreak attack via self-reminders. Nature Machine Intelligence, 5(12):1486–1496. find
- Zhangchen Xu, Fengqing Jiang, Luyao Niu, Jinyuan Jia, Bill Yuchen Lin, and Radha Poovendran. 2024. Safedecoding: Defending against jailbreak attacks via safety-aware decoding. arXiv preprint arXiv:2402.08983.
- Yan Yang, Zeguan Xiao, Xin Lu, Hongru Wang, Xuetao Wei, Hailiang Huang, Guanhua Chen, and Yun Chen. 2025. Seqar: Jailbreak llms with sequential auto-generated characters. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Hu… find
- Jiahao Yu, Xingwei Lin, Zheng Yu, and Xinyu Xing. 2023. GPTFUZZER: red teaming large language models with auto-generated jailbreak prompts. CoRR, abs/2309.10253.
- Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu. 2024. GPT-4 is too smart to be safe: Stealthy chat with llms via cipher. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net…
- Di Zhang, Xiaoshui Huang, Dongzhan Zhou, Yuqiang Li, and Wanli Ouyang. 2024a. Accessing gpt-4 level mathematical olympiad solutions via monte carlo tree self-refine with llama-3 8b. arXiv preprint arXiv:2406.07394. find
- Yichi Zhang, Siyuan Zhang, Yao Huang, Zeyu Xia, Zhengwei Fang, Xiao Yang, Ranjie Duan, Dong Yan, Yinpeng Dong, and Jun Zhu. 2025a. Stair: Improving safety alignment with introspective reasoning. arXiv preprint arXiv:2502.02384. — [arXiv:2502.02384](https://arxiv.org/abs/2502.02384) find
- Yingjie Zhang, Tong Liu, Zhe Zhao, Guozhu Meng, and Kai Chen. 2025b. Align in depth: Defending jailbreak attacks via progressive answer detoxification. arXiv preprint arXiv:2503.11185. — [arXiv:2503.11185](https://arxiv.org/abs/2503.11185) find
- Yuqi Zhang, Liang Ding, Lefei Zhang, and Dacheng Tao. 2024b. Intention analysis prompting makes large language models a good jailbreak defender. arXiv preprint arXiv:2401.06561. — [arXiv:2401.06561](https://arxiv.org/abs/2401.06561) find
- Yuqi Zhang, Liang Ding, Lefei Zhang, and Dacheng Tao. 2025c. Intention analysis makes LLMs a good jailbreak defender. In Proceedings of the 31st International Conference on Computational Linguistics, pages 2947–2968, Abu Dhabi, UAE. Association for Computational Linguistics.
- Jianan Zhao, Meng Qu, Chaozhuo Li, Hao Yan, Qian Liu, Rui Li, Xing Xie, and Jian Tang. 2022. Learning on large-scale text-attributed graphs via variational inference. arXiv preprint arXiv:2210.14709. — [arXiv:2210.14709](https://arxiv.org/abs/2210.14709) find
- Xuandong Zhao, Xianjun Yang, Tianyu Pang, Chao Du, Lei Li, Yu-Xiang Wang, and William Yang Wang. 2024. Weak-to-strong jailbreaking on large language models. arXiv preprint arXiv:2401.17256. — [arXiv:2401.17256](https://arxiv.org/abs/2401.17256) find
- Chujie Zheng, Fan Yin, Hao Zhou, Fandong Meng, Jie Zhou, Kai-Wei Chang, Minlie Huang, and Nanyun Peng. 2024. Prompt-driven llm safeguarding via directed representation optimization. arXiv preprint arXiv:2401.18018. find
- Jingnan Zheng, Xiangtian Ji, Yijun Lu, Chenhang Cui, Weixiang Zhao, Gelei Deng, Zhenkai Liang, An Zhang, and Tat-Seng Chua. 2025. Rsafe: Incentivizing proactive reasoning to build robust and adaptive llm safeguards. arXiv preprint arXiv:2506.07736. find
- Andy Zhou, Bo Li, and Haohan Wang. 2024. Robust prompt optimization for defending language models against jailbreaking attacks. arXiv preprint arXiv:2401.17263. — [arXiv:2401.17263](https://arxiv.org/abs/2401.17263) find
- Junda Zhu, Lingyong Yan, Shuaiqiang Wang, Dawei Yin, and Lei Sha. 2025. Reasoning-to-defend: Safety-aware reasoning can defend large language models from jailbreaking. arXiv preprint arXiv:2502.12970.
- Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. 2024. Multilingual machine translation with large language models: Empirical results and analysis. In Findings of the Association for Computational Linguistics: NAACL 2024, Mexico City, Mexico,…
- Qingsong Zou, Jingyu Xiao, Qing Li, Zhi Yan, Yuhang Wang, Li Xu, Wenxuan Wang, Kuofeng Gao, Ruoyu Li, and Yong Jiang. 2025. Queryattack: Jailbreaking aligned large language models using structured non-natural query language. arXiv preprint arXiv:2502.09723. find
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/papers/pu-2026-beyond-surface-level-detection-cognitive/