Elamur Research⌕
☰
home / papers / 2026 / Enhancing the Transferability of Jailbreak Attacks on Large Language Models via Exploiting Reparameterization Invariance

Enhancing the Transferability of Jailbreak Attacks on Large Language Models via Exploiting Reparameterization Invariance

Paper · 2026 · Ao Wang, Xinghao Yang, Yongshun Gong et al.

Summary

Token-level jailbreak attacks tune adversarial prefixes or suffixes using gradients from an open-source proxy model. They succeed on that proxy but transfer poorly to other models, including proprietary ones. The authors argue the cause is geometric. Ordinary Euclidean gradients depend on the proxy's specific parameterization, so the optimized perturbations overfit to that model. They propose RIGJ (Reparameterization Invariance Gradient-based Jailbreak), which replaces Euclidean steps with natural-gradient steps. Step size is measured by KL divergence between output distributions, and the Fisher information matrix serves as the metric. Because different LLMs model similar language distributions, this output-space geometry is largely shared, so the update directions should be less tied to any one model. To make this practical, the method combines:

  • a diagonal Fisher approximation,
  • Gumbel–Softmax relaxation with temperature annealing,
  • an objective anchored on harmful–benign sample pairs,
  • joint optimization over many samples.

RIGJ improves cross-model attack success and harmfulness scores at lower compute cost. The work is framed as a tool for stress-testing safety alignment.

Key points

  • Compared with prior attacks, RIGJ raises cross-model Attack Success Rate by 14.9% and Average Harmfulness Score by 1.23.
  • Natural gradients are shown to be reparameterization-invariant. Euclidean steps would transfer consistently only if the mapping between models were an isometry, which real architectures violate.
  • Evaluation uses Advbench and Harmbench. A Euclidean ablation produces less diverse prefixes, with zlib complexity of about 0.74–0.76 versus about 0.87 for RIGJ, though RIGJ prefixes have higher PPL.
  • RIGJ produces a universal prefix and requires a white-box surrogate. Its stated limitations are that it assumes surrogate–target distributional similarity and estimates the Fisher matrix from finite samples.
  • Code is released on GitHub (nohuma/AISafety_transfer_jailbreak_RIGJ_2026), but the authors do not release ready-made jailbreak templates.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Technology SydneySydney · Australia18 works · 12,833 citations🎓Shandong UniversityJinan · China17 works · 789 citations🎓China University of Petroleum, East ChinaQingdao · China2 works · 3 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksBias, fairness & safety · 517 works

Cite

Cite as Ao Wang, Xinghao Yang, Yongshun Gong, Wei Liu, Bao-di Liu and Weifeng Liu. 2026. Enhancing the Transferability of Jailbreak Attacks on Large Language Models via Exploiting Reparameterization Invariance. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.357

@inproceedings{wang2026enhancing,
  title = {{Enhancing the Transferability of Jailbreak Attacks on Large Language Models via Exploiting Reparameterization Invariance}},
  author = {Ao Wang, Xinghao Yang, Yongshun Gong, Wei Liu, Bao-di Liu and Weifeng Liu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.357},
  url = {https://doi.org/10.18653/v1/2026.acl-long.357}
}

References

39 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shun-ichi Amari. 1998. Natural gradient works efficiently in learning. Neural Computation, 10(2):251– 276. find
  2. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. 2023. Jailbreaking black box large language models in twenty queries. Preprint, arXiv:2310.08419.
  3. Chiang et al. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90% chatgpt quality. https://lmsys.org/blog/2023-03-30-vicuna/. Accessed: 2023-5-30. ↗
  4. Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. 2017. Sharp minima can generalize for deep nets. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, page 1019–1028. JMLR.org. find
  5. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, and et al. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.
  6. Xingang Guo, Fangxu Yu, Huan Zhang, Lianhui Qin, and Bin Hu. 2024. Cold-attack: jailbreaking llms with stealthiness and controllability. In Proceedings of the 41st International Conference on Machine Learning, ICML’24, pages 16974 – 17002. find
  7. Kai Hu, Weichen Yu, Yining Li, Tianjun Yao, Xiang Li, Wenhe Liu, Lijun Yu, Zhiqiang Shen, Kai Chen, and Matt Fredrikson. 2024. Efficient llm jailbreak via adaptive dense-to-sparse constrained optimization. In Proceedings of the 38th International Conference on Neural Information Processing Systems,… find
  8. Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen. 2024. Catastrophic jailbreak of open-source llms via exploiting generation. In International Conference on Learning Representations.
  9. Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa. 2023. Llama guard: Llm-based input-output safeguard for human-ai conversations. Preprint, arXiv:2312.06674.
  10. Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. 2023. Baseline defenses for adversarial attacks against aligned language models. Preprint, arXiv:2309.00614.
  11. Xiaojun Jia, Tianyu Pang, Chao Du, Yihao Huang, Jindong Gu, Yang Liu, Xiaochun Cao, and Min Lin. 2025. Improved techniques for optimization-based jailbreaking on large language models. In International Conference on Learning Representations. find
  12. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, and et al. 2023. Mistral 7b. Preprint, arXiv:2310.06825. find
  13. Fengqing Jiang, Zhangchen Xu, Luyao Niu, Zhen Xiang, Bhaskar Ramasubramanian, Bo Li, and Radha Poovendran. 2024. ArtPrompt: ASCII art-based jailbreak attacks against aligned LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page…
  14. D. P. Kingma and J. Ba. 2015. Adam: A method for stochastic optimization. International Conference on Learning Representations.
  15. Simon Lermen, Charlie Rogers-Smith, and Jeffrey Ladish. 2023. Lora fine-tuning efficiently undoes safety training in llama 2-chat 70b. Preprint, arXiv:2310.20624. find
  16. Zeyi Liao and Huan Sun. 2024. Amplegcg: Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed llms. In Conference on Language Modeling, COLM’24. find
  17. Runqi Lin, Bo Han, Fengwang Li, and Tongling Liu. 2025. Understanding and enhancing the transferability of jailbreaking attacks. In International Conference on Machine Learning. find
  18. Hongfu Liu, Yuxi Xie, Ye Wang, and Michael Shieh. 2024a. Advancing adversarial suffix transfer learning on aligned large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 7213–7224, Miami, Florida, USA. Association for Computational Lin… find
  19. Xiaogeng Liu, Peiran Li, G. Edward Suh, Yevgeniy Vorobeychik, Zhuoqing Mao, Somesh Jha, Patrick McDaniel, Huan Sun, Bo Li, and Chaowei. 2025a. Autodan-turbo: A lifelong agent for strategy self-exploration to jailbreak llms. In International Conference on Learning Representations. find
  20. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2024b. Autodan: Generating stealthy jailbreak prompts on aligned large language models. In International Conference on Learning Representations. optimization. In Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Pro…
  21. Yue Liu, Xiaoxin He, Miao Xiong, Jinlan Fu, Shumin Deng, YINGWEI MA, Jiaheng Zhang, and Bryan Hooi. 2025b. Flipattack: Jailbreak llms via flipping. In International Conference on Machine Learning. find
  22. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, and et al. 2023. Llama 2: Open foundation and fine-tuned chat models. Preprint, arXiv:2307.09288.
  23. James Martens. 2020. New insights and perspectives on the natural gradient method. Journal of Machine Learning Research, 21(146):1–76. find
  24. Haochun Wang, Chi Liu, Nuwa Xi, Zewen Qiang, Sendong Zhao, Bing Qin, and Ting Liu. 2023. Huatuo: Tuning llama model with chinese medical knowledge. Preprint, arXiv:2304.06975.
  25. Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. 2024. HarmBench: A standardized evaluation framework for automated red teaming and robust refusal. In Proceedings of the 41st International Con…
  26. Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. 2024. Tree of attacks: Jailbreaking black-box llms automatically. In Advances in Neural Information Processing Systems, volume 37, pages 61065–61105. Curran Associates, Inc.
  27. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, and et al. 2024. Gpt-4 technical report. Preprint, arXiv:2303.08774.
  28. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Train…
  29. Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2024. Fine-tuning aligned language models compromises safety, even when users do not intend to! In International Conference on Learning Representations.
  30. Mubashar Raza, Zarmina Jahangir, Muhammad Bilal Riaz, Muhammad Jasim Saeed, and Muhammad Awais Sattar. 2025. Industrial applications of large language models. Scientific Reports, 15(1):13755. find
  31. Alexander Robey, Eric Wong, Hamed Hassani, and George J. Pappas. 2024. Smoothllm: Defending large language models against jailbreaking attacks. Preprint, arXiv:2310.03684.
  32. John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. 2015. Trust region policy Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu. 2023. Defending chatgpt against jailbreak attack via self-reminders. Nature Machine Intell… find
  33. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, and 43 others. 2024. Qwen2 technical report. Preprint, arXiv:240… find
  34. Junxiao Yang, Zhexin Zhang, Shiyao Cui, Hongning Wang, and Minlie Huang. 2025. Guiding not forcing: Enhancing the transferability of jailbreaking attacks on LLMs via removing superfluous constraints. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1… find
  35. Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. 2024. How johnny can persuade LLMs to jailbreak them: Rethinking persuasion to challenge AI safety by humanizing LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Lon…
  36. Huaqin Zhao, Zhengliang Liu, Zihao Wu, Yiwei Li, Tianze Yang, Peng Shu, Shaochen Xu, Haixing Dai, Lin Zhao, Hanqi Jiang, Yi Pan, Junhao Chen, Yifan Zhou, Gengchen Mai, Ninghao Liu, and Tianming Liu. 2024. Revolutionizing finance with llms: An overview of applications and insights. Preprint, arXiv:24…
  37. Xuandong Zhao, Xianjun Yang, Tianyu Pang, Chao Du, Lei Li, Yu-Xiang Wang, and William Yang Wang. 2025. Weak-to-strong jailbreaking on large language models. In Proceedings of the 42rd International Conference on Machine Learning. find
  38. Chujie Zheng, Fan Yin, Hao Zhou, Fandong Meng, Jie Zhou, Kai-Wei Chang, Minlie Huang, and Nanyun Peng. 2024. On prompt-driven safeguarding for large language models. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages… find
  39. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. Preprint, arXiv:2307.15043.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-enhancing-transferability-jailbreak-attacks-large/