Elamur Research⌕
☰
home / papers / 2025 / from Benign import Toxic: Jailbreaking the Language Model via Adversarial Metaphors

from Benign import Toxic: Jailbreaking the Language Model via Adversarial Metaphors

Summary

The paper argues that black-box jailbreaks which try to extract harmful content from an LLM in a single step are becoming easier to detect, because the malicious intent stays visible in the prompt. The authors suggest a different route: get the model to explain a harmless analogue of the forbidden task, then ask it to translate that explanation back into the real domain. Their framework, AVATAR, has two parts. Adversarial Entity Mapping (AEM) breaks a harmful query into key entities and asks a pool of crowdsourced LLMs, sampled at high temperature, to propose benign metaphorical counterparts. Candidates are filtered with Internal Consistency Similarity (ICS), computed over BGE-M3 embeddings, and Conceptual Disparity (CD), which together balance usefulness against how well the intent is hidden. Metaphor-Induced Reasoning (MIR) then embeds the chosen metaphor in a multi-turn dialogue and has the target model calibrate the gap between the metaphorical and the actual answer. AVATAR needs no training and reports state-of-the-art attack success rates that transfer across several LLMs. The work exposes a reasoning-based weakness in current safety alignment.

Key points

  • AVATAR exceeds 92% ASR on GPT-4o within 3 retries and also elicits harmful output from reasoning models such as ChatGPT-o1. It was evaluated on AdvBench and Harmbench.
  • The attack is formalized as building a directed semantic graph, where the goal is to recover the relations among harmful entities by way of benign entities and the mappings between the two sets.
  • The defenses tested, adaptive system prompts and target-side summarization, cut ASR by about 16–33 points on AdvBench but do not eliminate the attack.
  • Weaker crowdsourced models such as Interlm2.5-7B supplied more effective metaphors than GPT-4o-mini or Llama-3.1-70B, likely because they have looser safety alignment. Cognitive, Logical and Concrete analogies were the dominant successful types.
  • The authors disclosed the findings to the maintainers of the GLM, Qwen, Llama and GPT models and restricted the code release.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Language models & pretraining · Bias, fairness & safety · Idioms & multiword expressions

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.238

@inproceedings{yan2025benign,
  title = {{from Benign import Toxic: Jailbreaking the Language Model via Adversarial Metaphors}},
  author = {Yan, Yu and Sun, Sheng and Duan, Zenghao and Liu, Teli and Liu, Min and Yin, Zhiyi and Lei, Jingyu and Li, Qi},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.238},
  url = {https://doi.org/10.18653/v1/2025.acl-long.238}
}

References

43 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. find
  2. Nicholas Carlini, Milad Nasr, Christopher A. Choquette-Choo, Matthew Jagielski, Irena Gao, Anas Awadalla, Pang Wei Koh, Daphne Ippolito, Katherine Lee, Florian Tramèr, and Ludwig Schmidt. 2023. Are aligned neural networks adversarially aligned? ArXiv, abs/2306.15447. find
  3. Zhiyuan Chang, Mingyang Li, Yi Liu, Junjie Wang, Qing Wang, and Yang Liu. 2024. Play guessing game with llm: Indirect jailbreak attack with implicit clues. arXiv preprint arXiv:2402.09091. find
  4. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. 2023. Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419. find our card
  5. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. Preprint, arXiv:2402.03216. find our card
  6. Yixin Cheng, Markos Georgopoulos, Volkan Cevher, and Grigorios G Chrysos. 2024. Leveraging the context through multi-round interactions for jailbreaking attacks. arXiv preprint arXiv:2402.09177. — [arXiv:2402.09177](https://arxiv.org/abs/2402.09177) find
  7. Peng Ding, Jun Kuang, Dan Ma, Xuezhi Cao, Yunsen Xian, Jiajun Chen, and Shujian Huang. 2024. A wolf in sheep’s clothing: Generalized nested jailbreak prompts can fool large language models easily. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computationa… find
  8. Zenghao Duan, Wenbin Duan, Zhiyi Yin, Yinghan Shen, Shaoling Jing, Jie Zhang, Huawei Shen, and Xueqi Cheng. 2025. Related knowledge perturbation matters: Rethinking multiple pieces of knowledge editing in same-subject. arXiv preprint arXiv:2502.06868. find
  9. Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. 2024. Bias and fairness in large language models: A survey. Computational Linguistics, pages 1–79. find our card
  10. Wenhan Xiong Grattafiori, Alexandre Défossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier, Thomas Scialom, and Gabriel Synnaeve. 2023. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950. find our card
  11. Divij Handa, Advait Chirmule, Bimal Gajera, and Chitta Baral. 2024. Jailbreaking proprietary large language models using word substitution cipher. arXiv preprint arXiv:2402.10601. find
  12. Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. 2022. Toxigen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1:… find our card
  13. Jiaming Ji, Bo Chen, Haoyu Lou, Difan Hong, Bowen Zhang, Xing Pan, and Diyi Yang. 2024. Aligner: Efficient alignment by learning to correct. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, pages 90853–90890. find
  14. Fengqing Jiang, Zhangchen Xu, Luyao Niu, Zhen Xiang, Bhaskar Ramasubramanian, Bo Li, and Radha Poovendran. 2024. Artprompt: Ascii art-based jailbreak attacks against aligned llms. arXiv preprint arXiv:2402.11753. find
  15. Aounon Kumar, Chirag Agarwal, Suraj Srinivas, AJ Li, S Feizi, and H Lakkaraju. 2024. Certifying llm safety against adversarial prompting. arxiv 2024. arXiv preprint arXiv:2309.02705. find our card
  16. Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, Jie Huang, and Yangqiu Song. 2023a. Multi-step jailbreaking privacy attacks on chatgpt. ArXiv, abs/2304.05197. find our card
  17. Tianlong Li, Zhenghua Wang, Wenhao Liu, Muling Wu, Shihan Dou, Changze Lv, Xiaohua Wang, Xiaoqing Zheng, and Xuan-Jing Huang. 2025. Revisiting jailbreaking for large language models: A representation engineering perspective. In Proceedings of the 31st International Conference on Computational Lingui… find
  18. Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. 2024. Drattack: Prompt decomposition and reconstruction makes powerful llm jailbreakers. Preprint, arXiv:2402.16914. — [arXiv:2402.16914](https://arxiv.org/abs/2402.16914) find
  19. Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, and Bo Han. 2023b. Deepinception: Hypnotize large language model to be jailbreaker. ArXiv, abs/2311.03191. find our card
  20. Zichao Lin, Shuyan Guan, Wending Zhang, Huiyan Zhang, Yugang Li, and Huaping Zhang. 2024. Towards trustworthy llms: a review on debiasing and dehallucinating in large language models. Artificial Intelligence Review, 57(9):1–50. find
  21. Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Yang Liu. 2023. Jailbreaking chatgpt via prompt engineering: An empirical study. ArXiv, abs/2305.13860. find our card
  22. Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, et al. 2024. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. In Forty-first International Conference on Machine Learning. find our card
  23. Bailin Wang, Zi Wang, Xuezhi Wang, Yuan Cao, Rif A Saurous, and Yoon Kim. 2024a. Grammar prompting for domain-specific language generation with large language models. Advances in Neural Information Processing Systems, 36. find
  24. Zhenhua Wang, Wei Xie, Baosheng Wang, Enze Wang, Zhiwen Gui, Shuoyoucheng Ma, and Kai Chen. 2024b. Foot in the door: Understanding large language model jailbreaking via cognitive psychology. arXiv preprint arXiv:2402.15690. — [arXiv:2402.15690](https://arxiv.org/abs/2402.15690) find
  25. Zeming Wei, Yifei Wang, and Yisen Wang. 2023. Jailbreak and guard aligned language models with only few in-context demonstrations. arXiv preprint arXiv:2310.06387. find our card
  26. Xikang Yang, Xuehai Tang, Songlin Hu, and Jizhong Han. 2024. Chain of attack: a semantic-driven contextual multi-turn attacker for llm. arXiv preprint arXiv:2405.05610. — [arXiv:2405.05610](https://arxiv.org/abs/2405.05610) find
  27. Sibo Yi, Yule Liu, Zhen Sun, Tianshuo Cong, Xinlei He, Jiaxing Song, Ke Xu, and Qi Li. 2024. Jailbreak attacks and defenses against large language models: A survey. arXiv preprint arXiv:2407.04295. find
  28. Zheng-Xin Yong, Cristina Menghini, and Stephen H Bach. 2023. Low-resource languages jailbreak gpt-4. arXiv preprint arXiv:2310.02446. find our card
  29. Jiahao Yu, Xingwei Lin, and Xinyu Xing. 2023. Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts. arXiv preprint arXiv:2309.10253. find our card
  30. Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. 2023. Tree of attacks: Jailbreaking black-box llms automatically. arXiv preprint arXiv:2312.02119. find our card
  31. Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu. 2024. GPT-4 is too smart to be safe: Stealthy chat with LLMs via cipher. In The Twelfth International Conference on Learning Representations. find our card
  32. Roberto Navigli, Simone Conia, and Björn Ross. 2023. Biases in large language models: origins, inventory, and discussion. ACM Journal of Data and Information Quality, 15(2):1–21. find
  33. Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. 2024. How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms. arXiv preprint arXiv:2401.06373. find
  34. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744. find our card
  35. Mark Russinovich, Ahmed Salem, and Ronen Eldan. 2024. Great, now write an article about that: The crescendo multi-turn llm jailbreak attack. ArXiv, abs/2404.01833. find
  36. Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. 2023. Large language models in medicine. Nature medicine, 29(8):1930– 1940. find
  37. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. 2023a. Siren’s song in the ai ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.01219. find our card
  38. Zhexin Zhang, Junxiao Yang, Pei Ke, and Minlie Huang. 2023b. Defending large language models against jailbreaking attacks through goal prioritization. arXiv preprint arXiv:2311.09096. find
  39. Yue Zhou, Henry Peng Zou, Barbara Di Eugenio, and Yang Zhang. 2024a. Large language models are involuntary truth-tellers: Exploiting fallacy failure for jailbreak attacks. In EMNLP, pages 13293–13304. find
  40. Zhenhong Zhou, Jiuyang Xiang, Haopeng Chen, Quan Liu, Zherui Li, and Sen Su. 2024b. Speak out of turn: Safety vulnerability of large language models in multi-turn dialogue. ArXiv, abs/2402.17262. — [arXiv:2402.17262](https://arxiv.org/abs/2402.17262) find
  41. Zhenhong Zhou, Haiyang Yu, Xinghua Zhang, Rongwu Xu, Fei Huang, and Yongbin Li. 2024c. How alignment and jailbreak work: Explain LLM safety through intermediate hidden states. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 2461–2488, Miami, Florida, USA. Association… find
  42. Sicheng Zhu, Ruiyi Zhang, Bang An, Gang Wu, Joe Barrow, Zichao Wang, Furong Huang, Ani Nenkova, and Tong Sun. 2023. Autodan: Interpretable gradient-based adversarial attacks on large language models. In First Conference on Language Modeling. find
  43. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yan-2025-benign-import-toxic-jailbreaking-language/