Elamur Research⌕
☰
home / papers / 2024 / Jailbreak Attacks and Defenses Against Large Language Models: A Survey

Jailbreak Attacks and Defenses Against Large Language Models: A Survey

Paper · 2024 · Sibo Yi, Yule Liu, Zhen Sun et al.

Summary

Large language models such as ChatGPT and Gemini undergo safety alignment so they refuse harmful requests. Carefully engineered adversarial prompts, known as jailbreaks, can still push them into producing content that violates usage policies. Earlier surveys mostly catalogued attacks and paid little attention to the matching countermeasures. This survey addresses that gap with a single taxonomy that covers both sides.

Attacks are grouped by how much the attacker can see of the target model:

  • White-box methods exploit gradients, output logits, or fine-tuning access.
  • Black-box methods rely on templates, rewritten prompts, or a helper LLM that writes attack prompts.

Defenses are split by whether they leave the protected model unchanged (prompt-level) or modify or extend it (model-level). The authors chart which defenses also work against attack families they were not built for. They also review evaluation metrics, datasets and frameworks.

Their central observation is that attacks keep getting stronger while needing less knowledge of the target, which makes them more practical. This raises the urgency of robust defenses. The paper is a useful map for anyone securing deployed LLMs.

Key points

  • White-box attacks fall into three families: gradient-based, logits-based and fine-tuning-based.
    • The gradient-based family includes GCG (Zou et al.), whose optimized adversarial suffix transferred to ChatGPT, Bard and Claude.
    • Follow-ups such as ARCA, AutoDAN and ASETF aim for more readable suffixes.
  • Black-box attacks fall into three families:
    • template completion (scenario nesting, context-based, code injection);
    • prompt rewriting (cipher, low-resource languages, genetic-algorithm-based);
    • LLM-based generation.
  • Prompt-level defenses include perplexity-based prompt detection, prompt perturbation and system prompt safeguards.
  • Model-level defenses include SFT-based and RLHF-based training, gradient and logit analysis, refinement, and proxy defense (a separate secure LLM that filters outputs).
  • Evaluation uses attack success rate (ASR) and Perplexity as metrics.
    • Surveyed datasets include AdvBench, HarmBench, JailbreakBench, StrongREJECT, Do-not-Answer, XSTEST, SafetyBench, plus the Chinese-focused SC-Safety and Safety-Prompts.
    • Surveyed frameworks include EasyJailbreak, JailbreakBench and JailbreakEval.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China415 works · 94,028 citations🎓The Hong Kong University of Science and Technology (Guangzhou)Guangzhou · China67 works · 1,765 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksBias, fairness & safety · 587 works

Cite

Cite as Sibo Yi, Yule Liu, Zhen Sun, Tianshuo Cong, Xinlei He, Jiaxing Song, Ke Xu and Qi Li. 2024. Jailbreak Attacks and Defenses Against Large Language Models: A Survey. arXiv:2407.04295.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2407.04295 · arXiv 2407.04295

@misc{yi2024jailbreak,
  title = {{Jailbreak Attacks and Defenses Against Large Language Models: A Survey}},
  author = {Sibo Yi, Yule Liu, Zhen Sun, Tianshuo Cong, Xinlei He, Jiaxing Song, Ke Xu and Qi Li},
  year = {2024},
  doi = {10.48550/arxiv.2407.04295},
  eprint = {2407.04295},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2407.04295}
}

References

2 references, 2 of them with a card of ours. ↗ leads to the work.

  1. [1] Gabriel Alon and Michael Kamfonas. Detecting Language Model Attacks with Perplexity. CoRR abs/2308.14132, 2023. 12, 16
  2. [2] Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks. CoRR abs/2404.02151, 2024. 4

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yi-2024-jailbreak-attacks-defenses-against-large/