Elamur Research⌕
☰
home / papers / 2026 / When Safety Alignment Fails to Generalize: Probing with Language Game Jailbreaks

When Safety Alignment Fails to Generalize: Probing with Language Game Jailbreaks

Paper · 2026 · Zewen Long, Yu Peng, Fangming Dong et al.

Summary

Safety-aligned LLMs are supposed to refuse harmful requests. This paper asks whether those refusals depend on what a prompt means or only on how it looks. Long et al. probe this with "language games": rule-based rewritings of text, such as inserting a syllable before each vowel, that change a prompt's appearance while the model can still decode it. They use four traditional games and also design eight custom games whose rules can be parameterized and minimally tweaked. They also present AutoLanJail, an automated pipeline that generates, tests and revises such transformations. The authors frame the jailbreaks as a diagnostic tool for measuring how far alignment generalizes, not as a goal in themselves. They run experiments on open- and closed-source models and add LoRA safety fine-tuning of Llama-3.1-70B. A model hardened against one game stays vulnerable to related games, and even to the same game with a different inserted string. The results suggest that fine-tuning-based alignment learns surface patterns rather than abstract safety principles. The authors therefore argue that safety evaluation should cover systematic linguistic variation.

Key points

  • The four natural games examined are Alfa Balfa, Aigy Paigy, Leetspeak and Ubbi Dubbi. Custom games Cus1–Cus8 include letter insertion, letter shuffling or rotation, vowel substitution and glyph-like symbol replacement.
  • After LoRA fine-tuning of Llama-3.1-70B on a single custom format, attack success rate (ASR) on that format falls to 1–8%. On unseen custom formats, ASR reaches as high as 75%.
  • Fine-tuning on seven formats (Cus2–Cus8) leaves the held-out Cus1 highly exploitable, while the trained formats stay suppressed.
  • Changing only Cus1's inserted string bypasses the defense: ASR is 2% with the trained "-a-" but 98% with "@p@", 94% with "*z*", 90% with "&k&" and 75% with "ˆmˆ".
  • Stated limitations include coverage of only language-game-style shifts and two safety benchmarks. Harmfulness was judged by human annotators, and fine-tuning was done with LoRA on one backbone model.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏛️Chinese Academy of SciencesBeijing · China246 works · 24,952 citations🔬Institute of AutomationBeijing · China87 works · 14,883 citations🎓University of Chinese Academy of SciencesBeijing · China188 works · 8,744 citations🔬Institute of Information EngineeringBeijing · China41 works · 1,485 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSocial Sciences · 1,954 worksSafety Research · 626 works

🧭 Topics 2Research subjects and languages the work is about.Bias, fairness & safety · 587 worksInterpretability · 149 works

Cite

Cite as Zewen Long, Yu Peng, Fangming Dong, Congyi Li, Xingmao Guan, Shu Wu and Kai Chen. 2026. When Safety Alignment Fails to Generalize: Probing with Language Game Jailbreaks. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.739

@inproceedings{long2026when,
  title = {{When Safety Alignment Fails to Generalize: Probing with Language Game Jailbreaks}},
  author = {Zewen Long, Yu Peng, Fangming Dong, Congyi Li, Xingmao Guan, Shu Wu and Kai Chen},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.739},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.739}
}

References

30 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. AI@Meta. 2024. Llama 3.1 model card. https://github.com/meta-llama/ llama-models/blob/main/models/ llama3_1/MODEL_CARD.md. Accessed: 2024-09-01. ↗
  2. Maksym Andriushchenko and Nicolas Flammarion. 2024. Does refusal training in llms generalize to the past tense? In Neurips Safe Generative AI Workshop 2024. find
  3. Boaz Barak. 2023. Another jailbreak for gpt4: Talk to it in morse code. https://twitter.com/boazbaraktcs/status/ 1637657623100096513. ↗
  4. Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Röttger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. 2023. Safety-tuned llamas: Lessons from improving the safety of large language models that follow instructions. arXiv preprint arXiv:2309.07875.
  5. Zhiyuan Chang, Mingyang Li, Yi Liu, Junjie Wang, Qing Wang, and Yang Liu. 2024. Play guessing game with llm: Indirect jailbreak attack with implicit clues. arXiv preprint arXiv:2402.09091. find
  6. Lichang Chen, Khalid Saifullah, Ming Li, Tianyi Zhou, and Heng Huang. 2023. Claude2-alpaca: Instruction tuning datasets distilled from claude. https://github.com/Lichang-Chen/ claude2-alpaca. ↗
  7. Liuji Chen, Hao Gao, Jinghao Zhang, Qiang Liu, Shu Wu, and Liang Wang. 2025. Select me! when you need a tool: A black-box text attack on tool selection. find
  8. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30. find
  9. Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. 2023. Safe rlhf: Safe reinforcement learning from human feedback. arXiv preprint arXiv:2310.12773.
  10. Michael Han Daniel Han and Unsloth team. 2023. Unsloth. find
  11. Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. 2023. Multilingual jailbreak challenges in large language models. arXiv preprint arXiv:2310.06474.
  12. Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al. 2022. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. arXiv preprint arXiv:2209.07858.
  13. Xingang Guo, Fangxu Yu, Huan Zhang, Lianhui Qin, and Bin Hu. 2024. Cold-attack: Jailbreaking llms with stealthiness and controllability. arXiv preprint arXiv:2402.08679. — [arXiv:2402.08679](https://arxiv.org/abs/2402.08679) find
  14. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685.
  15. Jiaming Ji, Donghai Hong, Borong Zhang, Boyuan Chen, Josef Dai, Boren Zheng, Tianyi Qiu, Boxun Li, and Yaodong Yang. 2024. Pku-saferlhf: A safety alignment preference dataset for llama family models. arXiv preprint arXiv:2406.15513. — [arXiv:2406.15513](https://arxiv.org/abs/2406.15513) find
  16. Fengqing Jiang, Zhangchen Xu, Luyao Niu, Zhen Xiang, Bhaskar Ramasubramanian, Bo Li, and Radha Poovendran. 2024. Artprompt: Ascii art-based jailbreak attacks against aligned llms. arXiv preprint arXiv:2402.11753.
  17. Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, Jie Huang, Fanpu Meng, and Yangqiu Song. 2023. Multi-step jailbreaking privacy attacks on chatgpt. arXiv preprint arXiv:2304.05197.
  18. Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wangmeng Zuo, Dahua Lin, Yu Qiao, and Jing Shao. 2024. Salad-bench: A hierarchical and comprehensive safety benchmark for large language models. arXiv preprint arXiv:2402.05044.
  19. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2023. Autodan: Generating stealthy jailbreak prompts on aligned large language models. arXiv preprint arXiv:2310.04451.
  20. Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. 2024. Tree of attacks: Jailbreaking black-box llms automatically. Advances in Neural Information Processing Systems, 37:61065–61105.
  21. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744.
  22. Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. 2022. Red teaming language models with language models, february 2022a. URL https://arxiv.org/abs/2202.03286 v1. ↗
  23. Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, and Peter Henderson. 2025. Safety alignment should be made more than just a few tokens deep. In The Thirteenth International Conference on Learning Representations.
  24. Bhaktipriya Radharapu, Kevin Robinson, Lora Aroyo, and Preethi Lahoti. 2023. Aart: Ai-assisted red-teaming with diverse data generation for new llm-powered applications. arXiv preprint arXiv:2311.08592. find
  25. Alexander Robey, Eric Wong, Hamed Hassani, and George J. Pappas. 2024. Smoothllm: Defending large language models against jailbreaking attacks.
  26. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2024. Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36.
  27. Wikipedia contributors. 2024a. Language game. [Online; accessed 12-September-2024]. find
  28. Wikipedia contributors. 2024b. Leet. [Online; accessed 12-September-2024]. find
  29. Junfei Wu, Qiang Liu, Ding Wang, Jinghao Zhang, Shu Wu, Liang Wang, and Tieniu Tan. 2024. Logical closed loop: Uncovering object hallucinations in large vision-language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 6944–6962. Association for Computational Ling… find
  30. Shu Wu, Zekun Li, Yunyue Su, Zeyu Cui, Xiaoyu Zhang, and Liang Wang. 2025. Graphfm: Graph factorization machines for feature interaction modelling. Machine Intelligence Research, 22(2):239–253. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/long-2026-when-safety-alignment-fails-generalize/