Elamur Research⌕
☰
home / papers / 2024 / Jailbreaking LLMs with Arabic Transliteration and Arabizi

Jailbreaking LLMs with Arabic Transliteration and Arabizi

Paper · 2024 · Mansour Al Ghanim, Saleh Almohaimeed, Mengxin Zheng et al.

Summary

The paper asks whether safety alignment in commercial LLMs holds up when harmful requests are written in Arabic, and especially in Arabic written with Latin characters. The authors took the 520 harmful behaviours of AdvBench, machine-translated them into standardized Arabic, and proofread the output. They then converted the Arabic into transliteration and chatspeak (Arabizi) using their own letter-by-letter mapping tables. They sent all versions to GPT-4 and Claude 3 Sonnet and labelled every response by hand.

In standardized Arabic, both models reliably refused, even when prefix injection was added. The Latin-script forms, however, often slipped past the safeguards. They also exposed odd word-triggered behaviours that the Arabic-script form kept hidden.

As a defence, the authors propose a system prompt with three steps:

  • convert the input back to its original script,
  • translate it to English using phonemic and collocation knowledge,
  • then answer.

On GPT-4 transliteration, this prompt cut unsafe outputs from about 12% to under 1%. The work shows that safety training must cover non-canonical writing forms, not just canonical scripts.

Key points

  • Data and models: AdvBench (520 harmful prompts) was translated with GPT-3.5-turbo-0125, proofread, and tested on GPT-4 and Claude 3 Sonnet. All outputs were judged manually.

  • Four input forms:

    • standardized Arabic,
    • transliteration (own mapping, checked against Lexilogos, Greek letters avoided),
    • chatspeak with digits,
    • chatspeak-no-numbers, which both models understood better.

  • Standardized Arabic is robust: prompts were refused even with prefix injection. Transliteration and chatspeak produced unsafe content.

  • Hidden behaviours: some word combinations triggered unexpected outputs, namely a copyright-refusal statement in Claude-3 and a "Google AI assistant" persona in GPT-4. Small sentence-level and word-level edits could turn earlier refusals into unsafe answers.

  • Mitigation results (GPT-4, transliteration, Table 3):

    CategoryNo defenceFull system prompt
    Unsafe12.12%0.77%
    Misunderstanding28.65%15.58%
    Direct refusal13.27%49.80%

    Dropping the phoneme/collocation sentence raised unsafe outputs to 1.15%.

Interesting ideas

  • I1. Mapping a non-standard writing form back to its canonical form before interpreting it removes a whole class of surface-variation failures, provided the mapping is consistent.
  • I2. Naive letter-by-letter conversion produced wrong words, and the authors found that checking against sound patterns and typical word pairings was needed to restore correct vocabulary.
  • I3. Instead of a binary safe/unsafe score, outputs were sorted into finer categories: direct refusal, advice, translation, misunderstanding and unsafe. This separates "did not understand" from "understood and complied".
  • I4. Ablating a single instruction sentence (w-3) showed which part of the prompt carried the safety gain, a cheap way to attribute effects inside a prompt.
  • I5. Changing only the surface form of an input exposed hidden behaviours tied to specific words, which works as a probe for lexical shortcuts.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): A normalization pass for leetspeak, chat spellings or romanized loanwords before tagging would let the tagger and lemmatizer see canonical words.
  • I2 (tokens): When recovering lemmas from noisy spellings, candidates could be ranked by collocation plausibility with neighbouring words, not just by edit distance.
  • I3 (learn): Evaluation reports could split errors into "failed to parse/understand" versus "understood but answered wrongly", so that fixes target the right stage.
  • I4 (learn): LLM teacher-tier prompts could be ablated sentence by sentence against a fixed silver set, to keep only the instructions that measurably improve label quality.
  • I5 (learn): Respelled or lightly perturbed copies of test sentences could serve as negative controls, checking that parser and frame decisions do not hinge on memorized surface tokens.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Central FloridaOrlando · United States13 works · 39,673 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 3Research subjects and languages the work is about.Language models & pretraining · 3,320 worksSemitic languages · 2,908 worksBias, fairness & safety · 517 works

Cite

Cite as Mansour Al Ghanim, Saleh Almohaimeed, Mengxin Zheng, Yan Solihin and Qian Lou. 2024. Jailbreaking LLMs with Arabic Transliteration and Arabizi. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.emnlp-main.1034

@inproceedings{ghanim2024jailbreaking,
  title = {{Jailbreaking LLMs with Arabic Transliteration and Arabizi}},
  author = {Mansour {Al Ghanim}, Saleh Almohaimeed, Mengxin Zheng, Yan Solihin and Qian Lou},
  booktitle = {Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.emnlp-main.1034},
  url = {https://doi.org/10.18653/v1/2024.emnlp-main.1034}
}

References

35 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. ↗
  2. Mohamed Al-Badrashiny, Ramy Eskander, Nizar Habash, and Owen Rambow. 2014. Automatic transliteration of romanized dialectal arabic. In Proceedings of the eighteenth conference on computational natural language learning, pages 30–38.
  3. Mansour Al Ghanim, Muhammad Santriaji, Qian Lou, and Yan Solihin. 2023. Trojbits: A hardware aware inference-time attack on transformer-based language models. In ECAI 2023, pages 60–68. IOS Press. ↗
  4. Emad A Alghamdi, Reem I Masoud, Deema Alnuhait, Afnan Y Alomairi, Ahmed Ashraf, and Mohamed Zaytoon. 2024. Aratrust: An evaluation of trustworthiness for llms in arabic. arXiv preprint arXiv:2403.09017. ↗
  5. Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258. ↗
  6. Nicholas Carlini, Milad Nasr, Christopher A Choquette-Choo, Matthew Jagielski, Irena Gao, Pang Wei W Koh, Daphne Ippolito, Florian Tramer, and Ludwig Schmidt. 2024. Are aligned neural networks adversarially aligned? Advances in Neural Information Processing Systems, 36. find
  7. Achraf Chalabi and Hany Gerges. 2012. Romanized arabic transliteration. In Proceedings of the Second Workshop on Advances in Text Input Methods, pages 89–96.
  8. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30. ↗
  9. Junjie Chu, Yugeng Liu, Ziqing Yang, Xinyue Shen, Michael Backes, and Yang Zhang. 2024. Comprehensive assessment of jailbreak attacks against llms. arXiv preprint arXiv:2402.05668. ↗
  10. Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. 2023. Multilingual jailbreak challenges in large language models. arXiv preprint arXiv:2310.06474. ↗
  11. Mengnan Du, Fengxiang He, Na Zou, Dacheng Tao, and Xia Hu. 2023. Shortcut learning of large language models in natural language understanding. Communications of the ACM, 67(1):110–120. ↗
  12. Josh A Goldstein, Girish Sastry, Micah Musser, Renee DiResta, Matthew Gentzel, and Katerina Sedova. 2023. Generative language models and automated influence operations: Emerging threats and potential mitigations. arXiv preprint arXiv:2301.04246. ↗
  13. Nizar Habash, Abdelhadi Soudi, and Timothy Buckwalter. 2007. On arabic transliteration. Arabic computational morphology: Knowledge-based and empirical methods, pages 15–22. ↗
  14. Mingyu Jin, Suiyuan Zhu, Beichen Wang, Zihao Zhou, Chong Zhang, Yongfeng Zhang, et al. 2024. Attackeval: How to evaluate the effectiveness of jailbreak attacking on large language models. arXiv preprint arXiv:2401.09002. ↗
  15. Lexilogos. 2002. Transliteration conversion. https://www.lexilogos.com/keyboard/ arabic_conversion.htm. ↗
  16. Jie Li, Yi Liu, Chongyang Liu, Ling Shi, Xiaoning Ren, Yaowen Zheng, Yang Liu, and Yinxing Xue. 2024. A cross-language investigation into jailbreak attacks in large language models. arXiv preprint arXiv:2401.16765. ↗
  17. Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Yang Liu. 2023. Jailbreaking chatgpt via prompt engineering: An empirical study. arXiv preprint arXiv:2305.13860. ↗
  18. Qian Lou, Xin Liang, Jiaqi Xue, Yancheng Zhang, Rui Xie, and Mengxin Zheng. 2024. Cr-utp: Certified robustness against universal text perturbations on large language models. In Findings of the Association for Computational Linguistics ACL 2024, pages 9863– 9875. find
  19. Qian Lou, Yepeng Liu, and Bo Feng. 2022. Trojtext: Test-time invisible textual trojan insertion. In The Eleventh International Conference on Learning Representations. ↗
  20. Dong Lu, Tianyu Pang, Chao Du, Qian Liu, Xianjun Yang, and Min Lin. 2024. Test-time backdoor attacks on multimodal large language models. arXiv preprint arXiv:2402.08577. ↗
  21. Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018. Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. arXiv preprint arXiv:1808.08745. ↗
  22. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744. ↗
  23. Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2023. Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693. ↗
  24. Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2023. " do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. arXiv preprint arXiv:2308.03825. ↗
  25. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. ↗
  26. Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. Universal adversarial triggers for attacking and analyzing nlp. arXiv preprint arXiv:1908.07125. ↗
  27. Yao Wan, Zhou Zhao, Min Yang, Guandong Xu, Haochao Ying, Jian Wu, and Philip S Yu. 2018. Improving automatic source code summarization via deep reinforcement learning. In Proceedings of the 33rd ACM/IEEE international conference on automated software engineering, pages 397–407. ↗
  28. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461. ↗
  29. Boxin Wang, Wei Ping, Chaowei Xiao, Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Bo Li, Anima Anandkumar, and Bryan Catanzaro. 2022. Exploring the limits of domain-adaptive training for detoxifying large-scale language models. Advances in Neural Information Processing Systems, 35:35811– 35824. find
  30. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2024. Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36.
  31. Zhuohan Xie, Trevor Cohn, and Jey Han Lau. 2023. The next chapter: A study of large language models in storytelling. arXiv preprint arXiv:2301.09790. ↗
  32. Jiaqi Xue, Mengxin Zheng, Yebowen Hu, Fei Liu, Xun Chen, and Qian Lou. 2024a. Badrag: Identifying vulnerabilities in retrieval augmented generation of large language models. arXiv preprint arXiv:2406.00083. ↗
  33. Jiaqi Xue, Mengxin Zheng, Ting Hua, Yilin Shen, Yepeng Liu, Ladislau Bölöni, and Qian Lou. 2024b. Trojllm: A black-box trojan prompt attack on large language models. Advances in Neural Information Processing Systems, 36. find
  34. Xianjun Yang, Xiao Wang, Qi Zhang, Linda Petzold, William Yang Wang, Xun Zhao, and Dahua Lin. 2023. Shadow alignment: The ease of subverting safely-aligned language models. arXiv preprint arXiv:2310.02949. ↗
  35. Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. 2024. A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing, page 100211. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ghanim-2024-jailbreaking-llms-arabic-transliteration-arabizi/