Elamur Research⌕
☰
home / papers / 2024 / How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs

How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs

Paper · 2024 · Yi Zeng, Hongpeng Lin, Jingwen Zhang et al.

Summary

Most jailbreak research treats LLMs as algorithmic systems attacked by experts with optimized, often unreadable prompts. It largely ignores ordinary users who simply talk to models the way they would talk to people. This paper reframes LLMs as human-like communicators and asks whether persuasion alone can push them into unsafe behaviour. The authors draw on decades of social science work to build a taxonomy of persuasion techniques. They use it to create a Persuasive Paraphraser that automatically rewrites plain harmful requests into readable persuasive adversarial prompts (PAP). A broad scan first covers 14 policy-based risk categories. The paraphraser is then fine-tuned on successful PAP and used to probe target models iteratively with different techniques. Without specialized optimization, PAP exceeds 92% attack success on Llama 2-7b Chat, GPT-3.5 and GPT-4 within 10 trials, beating algorithm-focused baselines. Existing post-hoc defenses cope poorly with PAP. The proposed adaptive defenses also stop other attack types, though they reduce utility somewhat. The study shows that everyday-style persuasion is a serious and underexplored safety risk.

Key points

  • Attack success rate above 92% (10 trials) on Llama 2-7b Chat, GPT-3.5 and GPT-4, outperforming baselines such as GCG, PAIR, GPTFuzzer and low-resource-language attacks.
  • The taxonomy is the only openly released artifact (GitHub: CHATS-lab/persuasive_jailbreaker). The jailbreak data are available to vetted researchers on request, and the fine-tuned paraphraser is withheld.
  • Claude resisted PAP. The authors propose adaptive safety system prompts and a summarization-based defense, which also counter non-persuasive jailbreaks, suggesting a common mechanism.
  • The authors shared their findings with Meta and OpenAI ahead of publishing, which means the PAP examples in the paper may have stopped working.
  • Limitations: only single-turn persuasion was tested, techniques were sampled equally rather than at realistic frequencies, and the real-world harmfulness of jailbroken outputs remains uncertain.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Stanford UniversityStanford · United States506 works · 330,053 citations🎓Renmin University of ChinaBeijing · China139 works · 17,752 citations🎓Northeastern UniversityBoston · United States68 works · 7,316 citations🎓Virginia TechBlacksburg · United States39 works · 43,418 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksSocial Sciences · 1,954 worksSafety Research · 626 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksBias, fairness & safety · 587 works

Cite

Cite as Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia and Weiyan Shi. 2024. How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs. arXiv:2401.06373.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2401.06373 · arXiv 2401.06373

@misc{zeng2024how,
  title = {{How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs}},
  author = {Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia and Weiyan Shi},
  year = {2024},
  doi = {10.48550/arxiv.2401.06373},
  eprint = {2401.06373},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2401.06373}
}

References

1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Discouragement (a) Persuasion techniques mappings to social science literature. Table 5: The detailed mapping of the persuasion taxonomy in this paper to decades of research in psychology, communication, sociology, and marketing. find

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zeng-2024-johnny-persuade-llms-jailbreak-them/