Elamur Research
☰
home / papers / 2023 / Multilingual Jailbreak Challenges in Large Language Models

Multilingual Jailbreak Challenges in Large Language Models

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/2310.06474

Summary

Safety alignment of LLMs has been built and tested mostly in English, so it is unclear whether these protections hold when people write in other languages. The authors first machine-translated 15 harmful prompts from the GPT-4 report into 30 languages. They grouped the languages into high-, medium- and low-resource tiers by their share of CommonCrawl. ChatGPT's average unsafe rate rose from about 11% to 55% as resource level fell.

They then created MultiJail, a jailbreak benchmark in which native speakers translated English harmful queries into 9 languages. Using it, they tested ChatGPT and GPT-4 in two settings:

  • Unintentional: a non-English user accidentally slips past the safeguards.
  • Intentional: an attacker pairs a multilingual query with an English jailbreak instruction.

Low-resource languages produced roughly three times more harmful output than high-resource ones. As a fix, the authors propose Self-Defence, which follows the Self-Instruct approach. The model writes its own multilingual safety data for fine-tuning, so no human annotation is needed. The work matters for speakers of less-resourced languages, including Ukrainian, which the study places in the medium-resource tier.

Key points

  • Preliminary study: 15 prompts × 30 languages gave 450 examples, translated with Google Translate. Resource tiers were defined by CommonCrawl share: above 1%, 0.1–1%, and below 0.1%. Outputs were back-translated to English and labelled safe, unsafe or invalid by humans and by GPT-4 acting as a judge.
  • MultiJail languages: en, plus zh, it and vi (high-resource), ar, ko and th (medium), and bn, sw and jv (low). The data is released at github.com/DAMO-NLP-SG/multilingual-safety-for-LLMs.
  • Intentional scenario: unsafe rates reached 80.92% for ChatGPT (GPT-3.5-turbo-0613) and 40.71% for GPT-4. With adaptive multilingual attacks, they rose to nearly 100% and 79.05% respectively.
  • Self-Defence results: fine-tuning ChatGPT lowered unsafe output by 6.24% in the unintentional scenario and by 20.92% in the intentional one.
  • Safety–usefulness trade-off: with 100% safety training data, average safety rose from 59.67 to 90.67. Over the same change, usefulness (mean accuracy on XNLI and X-CSQA) fell from 46.67 to 35.28.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

167 papers of our library cite it (of the 140673 whose bibliographies we read); 10 of them have a page here, 157 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2310.06474 · arXiv 2310.06474

@misc{deng2023multilingual,
  title = {{Multilingual Jailbreak Challenges in Large Language Models}},
  author = {Deng, Yue and Zhang, Wenxuan and Pan, Sinno Jialin and Bing, Lidong},
  year = {2023},
  doi = {10.48550/arxiv.2310.06474},
  eprint = {2310.06474},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2310.06474}
}

References

40 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anthropic. Model card and evaluations for claude models. 2023. URL https://www-files. anthropic.com/production/images/Model-Card-Claude-2.pdf. find
  2. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Sh… find
  3. Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, and Pascale Fung. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. CoRR, abs/2302.04023, 20… find our card
  4. Rishabh Bhardwaj and Soujanya Poria. Red-teaming large language models using chain of utterances for safety-alignment. ArXiv, abs/2308.09662, 2023. URL https://doi.org/10.48550/arXiv.2308.09662. ↗
  5. Anirban Chakraborty, Manaar Alam, Vishal Dey, Anupam Chattopadhyay, and Debdeep Mukhopadhyay. A survey on adversarial attacks and defences. CAAI Trans. Intell. Technol., 6(1):25–45, 2021. URL https://doi.org/10.1049/cit2.12028. — [doi:10.1049/cit2.12028](https://doi.org/10.1049/cit2.12028) ↗
  6. Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Kaijie Zhu, Hao Chen, Linyi Yang, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, and Xing Xie. A survey on evaluation of large language models. CoRR, abs/2307.03109, 2023. URL https://doi.org/10.48550/arXi… ↗ our card
  7. Lingjiao Chen, Matei Zaharia, and James Zou. How is chatgpt’s behavior changing over time? CoRR, abs/2307.09009, 2023. URL https://doi.org/10.48550/arXiv.2307.09009. — [doi:10.48550/arxiv.2307.09009](https://doi.org/10.48550/arxiv.2307.09009) ↗
  8. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL https: //lmsys.org/blog/2023-03-30-vicuna… find
  9. Jonathan H. Choi, Kristin E. Hickman, Amy Monahan, and Daniel Schwarcz. ChatGPT Goes to Law School, 2023. URL https://papers.ssrn.com/abstract=4335905. ↗
  10. Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA,… find
  11. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. XNLI: evaluating cross-lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - N… find
  12. Gelei Deng, Yi Liu, Yuekang Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, and Yang Liu. Jailbreaker: Automated jailbreak across multiple large language model chatbots. CoRR, abs/2307.08715, 2023. URL https://doi.org/10.48550/arXiv.2307.08715. — [doi:10.48550/arxiv.2307.08715](https://doi.org/10.48550/arxiv.2307.08715) ↗
  13. Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, Andy Jones, Sam Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El Showk, Stanislav Fort, Zac Hatfield-Dodds, Tom Henighan… find our card
  14. Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. Toxigen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long… find our card
  15. Julian Hazell. Large language models can be used to effectively scale spear phishing campaigns. CoRR, abs/2305.06972, 2023. URL https://doi.org/10.48550/arXiv.2305.06972. — [doi:10.48550/arxiv.2305.06972](https://doi.org/10.48550/arxiv.2305.06972) ↗
  16. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, 2020. URL https://openreview.net/forum?id=rygGQyrFvH. ↗
  17. Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Xing Wang, and Zhaopeng Tu. Is chatgpt A good translator? A preliminary study. CoRR, abs/2301.08745, 2023. URL https://doi.org/10.48550/arXiv.2301.08745. ↗ our card
  18. Viet Dac Lai, Nghia Trung Ngo, Amir Pouran Ben Veyseh, Hieu Man, Franck Dernoncourt, Trung Bui, and Thien Huu Nguyen. Chatgpt beyond english: Towards a comprehensive evaluation of large language models in multilingual learning. CoRR, abs/2304.05613, 2023. URL https: //doi.org/10.48550/arXiv.2304.056… find our card
  19. Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, and Yangqiu Song. Multi-step jailbreaking privacy attacks on chatgpt. CoRR, abs/2304.05197, 2023. URL https://doi.org/10.48550/arXiv.2304.05197. ↗ our card
  20. Bill Yuchen Lin, Seyeon Lee, Xiaoyang Qiao, and Xiang Ren. Common sense beyond english: Evaluating and improving multilingual language models for commonsense reasoning. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confere… find
  21. Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Yang Liu. Jailbreaking chatgpt via prompt engineering: An empirical study. CoRR, abs/2305.13860, 2023. URL https://doi.org/10.48550/arXiv.2305.13860. ↗ our card
  22. Todor Markov, Chong Zhang, Sandhini Agarwal, Florentine Eloundou Nekoul, Theodore Lee, Steven Adler, Angela Jiang, and Lilian Weng. A holistic approach to undesired content detection in the real world. In Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference o… find our card
  23. AAAI Press, 2023. URL https: //doi.org/10.1609/aaai.v37i12.26752. — [doi:10.1609/aaai.v37i12.26752](https://doi.org/10.1609/aaai.v37i12.26752) find
  24. Xuan-Phi Nguyen, Wenxuan Zhang, Xin Li, Mahani Mahani Aljunied, Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Sen Yang, Chaoqun Liu, Hang Zhang, and Lidong Bing. Seallms - large language models for southeast asia. CoRR, abs/2312.00738, 2023. URL https://doi. org/10.48550/arXiv.2312.00738. find
  25. OpenAI. Chatgpt. 2023a. URL https://openai.com/chatgpt. ↗
  26. OpenAI. GPT-4 technical report. CoRR, abs/2303.08774, 2023b. URL https://doi.org/10.48550/arXiv.2303.08774. ↗ our card
  27. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. Train… find
  28. Ethan Perez, Saffron Huang, H. Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. Red teaming language models with language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, U… find our card
  29. Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang. Is chatgpt a general-purpose natural language processing task solver? CoRR, abs/2302.06476, 2023. URL https://doi.org/10.48550/arXiv.2302.06476. ↗ our card
  30. Saed Rezayi, Zhengliang Liu, Zihao Wu, Chandra Dhakal, Bao Ge, Haixing Dai, Gengchen Mai, Ninghao Liu, Chen Zhen, Tianming Liu, and Sheng Li. Exploring new frontiers in agricultural NLP: investigating the potential of large language models for food applications. CoRR, abs/2306.11892, 2023. URL https… — [doi:10.48550/arxiv.2306.11892](https://doi.org/10.48550/arxiv.2306.11892) find
  31. Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. ”do anything now”: Characterizing and evaluating in-the-wild jailbreak prompts on large language models. CoRR, abs/2308.03825, 2023. URL https://doi.org/10.48550/arXiv.2308.03825. ↗ our card
  32. Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Kumar Tanwani, Heather Cole-Lewis, Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul Gamble, Chris Kelly, Nathaneal Schärli, Aakanksha Chowdhery, Philip Andrew Mansfield, Blaise Agüera y Arc… find our card
  33. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… find
  34. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers… find
  35. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does LLM safety training fail? CoRR, abs/2307.02483, 2023. URL https://doi.org/10.48550/arXiv.2307.02483. ↗ our card
  36. Johannes Welbl, Amelia Glaese, Jonathan Uesato, Sumanth Dathathri, John Mellor, Lisa Anne Hendricks, Kirsty Anderson, Pushmeet Kohli, Ben Coppin, and Po-Sen Huang. Challenges in detoxifying language models. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Findings… find
  37. Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu. GPT-4 is too smart to be safe: Stealthy chat with llms via cipher. CoRR, abs/2308.06463, 2023. URL https://doi.org/10.48550/arXiv.2308.06463. ↗ our card
  38. Wenxuan Zhang, Sharifah Mahani Aljunied, Chang Gao, Yew Ken Chia, and Lidong Bing. M3exam: A multilingual, multimodal, multilevel benchmark for examining large language models. CoRR, abs/2306.05179, 2023a. URL https://doi.org/10.48550/arXiv.2306.05179. ↗
  39. Wenxuan Zhang, Yue Deng, Bing Liu, Sinno Jialin Pan, and Lidong Bing. Sentiment analysis in the era of large language models: A reality check. CoRR, abs/2305.15005, 2023b. URL https://doi.org/10.48550/arXiv.2305.15005. ↗ our card
  40. Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. CoRR, abs/2307.15043, 2023. URL https://doi.org/10.48550/arXiv.2307.15043. ↗ our card