Elamur Research⌕
☰
home / papers / 2026 / GAMBIT: A Gamified Jailbreak Framework for Multimodal Large Language Models

GAMBIT: A Gamified Jailbreak Framework for Multimodal Large Language Models

Paper · 2026 · Xiangdong Hu, Yangyang Jiang, Qin Hu et al.

Summary

Multimodal large language models can still be pushed into producing harmful output, but earlier visual jailbreaks mostly make the modified image task harder and treat the model as a passive solver. This works less well against reasoning models with chain-of-thought, which often spot malicious intent during deliberation. GAMBIT changes the strategy: it makes the model an active player in a game. The framework has three parts. First, a harmful image is split into a grid and shuffled with a key-based permutation, and the offending keyword in the text is masked. Second, the query is wrapped in a competitive "intelligence competition" with a rival and scoring pressure, drawing on flow theory and a prompt-level pseudo-reinforcement signal. Third, an adaptive search, adapted from CL-GSO, tunes the prompt components. Tested on HADES-style categories across reasoning and non-reasoning MLLMs, GAMBIT outperforms the VisCRA, HADES and SI-ATTACK baselines. The work shows that a model's own goal pursuit can weaken its safety attention. It argues for defenses that operate inside the reasoning chain rather than at the input.

Key points

  • Reported ASR: 92.13% on Gemini 2.5 Flash, 91.20% on QvQ-MAX and 85.87% on GPT-4o. Responses were judged by Llama-Guard-3 with a double-verification protocol.
  • In the grid-size analysis, a 4×4 split gave the best trade-off. Even 2×2 raised GPT-4o Self-Harm ASR by 13.34 points, while 8×8 overloaded the models.
  • Without adaptive search, the gamified prompt reached 69.33% on Self-Harm. Classic text jailbreaks scored far lower (DAN 8.00%, AIM 10.67%).
  • The input-level defense AdaShield-S cut GPT-4o ASR by under 10 points in every tested category (e.g., Privacy fell from 95.33% to 92.00%).
  • The authors propose two mitigations: a safety-aware chain-of-thought step and system prompts that rank safety above game or role-play rules. Stated limitations include higher token cost and reliance on a single automatic judge.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 4Research subjects and languages the work is about.Language models & pretraining · 3,320 worksReasoning · 1,426 worksVision & language · 657 worksBias, fairness & safety · 517 works

Cite

Cite as Xiangdong Hu, Yangyang Jiang, Qin Hu and Xiaojun Jia. 2026. GAMBIT: A Gamified Jailbreak Framework for Multimodal Large Language Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.367

@inproceedings{hu2026gambit,
  title = {{GAMBIT: A Gamified Jailbreak Framework for Multimodal Large Language Models}},
  author = {Xiangdong Hu, Yangyang Jiang, Qin Hu and Xiaojun Jia},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.367},
  url = {https://doi.org/10.18653/v1/2026.acl-long.367}
}

References

48 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Cem Anil, Esin Durmus, Nina Panickssery, Mrinank Sharma, Joe Benton, Sandipan Kundu, Joshua Batson, Meg Tong, Jesse Mu, Daniel Ford, and 1 others. 2024. Many-shot jailbreaking. Advances in Neural Information Processing Systems, 37. find
  2. Sebastian Deterding, Dan Dixon, Rilla Khaled, and Lennart Nacke. 2011. From game design elements to gamefulness: Defining “gamification”. In Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments (MindTrek ’11), pages 9–15. find
  3. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Hunen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, and 1 others. 2025. Qwen2.5-vl technical report. arXiv preprint arXiv:25…
  4. Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, and 1 others. 2022. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. arXiv preprint arXiv:2209.07858.
  5. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, and 1 others. 2022. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073.
  6. Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jérémy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, and 1 others. 2023. Open problems and fundamental limitations of reinforcement learning from human feedback. arXiv preprint arXiv:2307.15217. find
  7. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. 2023. Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419.
  8. Jiawei Chen, Yang Yang, Chao Yu, Yu Tian, Zhi Cao, Xue Yang, Linghao Li, Hang Su, and Zhaoxia Yin. 2025. Red teaming large reasoning models. arXiv preprint arXiv:2512.00412.
  9. Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuehui Wang, Qingyun Li, Yiming Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang, Tan Jiang, Bo Wang, and 1 others. 2024. Expanding performance boundaries of open-source multi…
  10. Jianfeng Chi, Ujjwal Karn, Hongyuan Zhan, Eric Smith, Javier Rando, Yiming Zhang, Kate Plawiak, Zacharie Delpierre Coudert, Kartikeya Upasani, and Mahesh Pasupuleti. 2024. Llama guard 3 vision: Safeguarding human-ai image understanding conversations. arXiv preprint arXiv:2411.10414. — [arXiv:2411.10414](https://arxiv.org/abs/2411.10414) find
  11. Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. 2023. Figstep: Jailbreaking large vision-language models via typographic visual prompts. arXiv preprint arXiv:2311.05608. find
  12. Weiyang Guo, Zesheng Shi, Zeen Zhu, Yuan Zhou, Min Zhang, and Jing Li. 2026. Backdoors in rlvr: Jailbreak backdoors in llms from verifiable reward. Preprint, arXiv:2604.09748.
  13. Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, Lihang Pan, Shuaiqi Duan, Weihan Wang, Yan Wang, Yean Cheng, Zehai He, and 1 others. 2025. GLM-4.5V and GLM-4.1V-Thinking: Towards versatile multimodal reasoning with scalable reinforcement learni…
  14. Peichun Hua, Hao Li, Shanghao Shi, Zhiyuan Yu, and Ning Zhang. 2025. Rethinking jailbreak detection of large vision language models with representational contrastive scoring. Preprint, arXiv:2512.12069.
  15. Yao Huang, Yitong Sun, Shouwei Ruan, Yichi Zhang, Yinpeng Dong, and Xingxing Wei. 2025. Breaking the ceiling: Exploring the potential of jailbreak attacks through expanding strategy space. arXiv preprint arXiv:2505.21277. find
  16. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. GPT-4o system card. arXiv preprint arXiv:2410.21276.
  17. Xiaojun Jia, Jie Liao, qi Guo, Teng Ma, Simeng Qin, Ranjie Duan, Tianlin Li, Yihao Huang, Zhitao Zeng, Dongxian Wu, Yiming Li, wenqi Ren, Xiaochun Cao, and Yang Liu. 2025. Omnisafebench-mm: A unified benchmark and toolbox for multimodal jailbreak attack-defense evaluatio. arXiv preprint arXiv:2512.0… — [arXiv:2512.06589](https://arxiv.org/abs/2512.06589) find
  18. Xiaojun Jia, Tianyu Pang, Chao Du, Yihao Huang, Jindong Gu, Yang Liu, Xiaochun Cao, and Min Lin. 2024. Improved techniques for optimization-based jailbreaking on large language models. arXiv preprint arXiv:2405.21018. — [arXiv:2405.21018](https://arxiv.org/abs/2405.21018) find
  19. Hengle Jiang and Ke Tang. 2026. Why agents compromise safety under pressure. Preprint, arXiv:2603.14975. find
  20. Yifan Li, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, and Ji-Rong Wen. 2024. Images are achilles’ heel of alignment: Exploiting visual vulnerabilities for jailbreaking multimodal large language models. arXiv preprint arXiv:2403.09792. — [arXiv:2403.09792](https://arxiv.org/abs/2403.09792) find
  21. Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, and 1 others. 2025. From system 1 to system 2: A survey of reasoning large language models. arXiv preprint arXiv:2502.17419.
  22. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023a. Visual instruction tuning. Advances in neural information processing systems, 36.
  23. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2024. AutoDAN: Generating stealthy jailbreak prompts on aligned large language models. In The Twelfth International Conference on Learning Representations.
  24. Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, Kailong Wang, and Yang Liu. 2023b. Jailbreaking chatgpt via prompt engineering: An empirical study. arXiv preprint arXiv:2305.13860.
  25. Weidi Luo, Tianyu Lu, Qiming Zhang, Xiaogeng Liu, Bin Hu, Yue Zhao, Jieyu Zhao, Song Gao, Patrick McDaniel, Zhen Xiang, and Chaowei Xiao. 2026. Doxing via the lens: Revealing location-related privacy leakage on multi-modal large reasoning models. In The Fourteenth International Conference on Learnin… find
  26. Zhenxing Niu, Haodong Ren, Xinbo Gao, Gang Hua, and Rong Jin. 2024. Jailbreaking attack against multimodal large language model. arXiv preprint arXiv:2402.02309. — [arXiv:2402.02309](https://arxiv.org/abs/2402.02309) find
  27. Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, and 1 others. 2021. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114.
  28. OpenAI Team. 2025a. Gpt-5.1: A smarter, more conversational chatgpt. https://openai.com/index/ gpt-5-1/. ↗
  29. OpenAI Team. 2025b. Introducing openai o3 and o4-mini. https://openai.com/index/ introducing-o3-and-o4-mini/. ↗
  30. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744.
  31. Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. 2023. Visual adversarial examples jailbreak aligned large language models. arXiv preprint arXiv:2306.13213. find
  32. Qwen Team. 2025. Qvq-max: A vision-language model with advanced visual reasoning capabilities. Technical preview, Alibaba Group. Available at: https://qwenlm.github.io/blog/qvq-max-preview/. ↗
  33. Tarun Raheja, Nilay Pochhi, and F.D.C.M. Curie. 2024. Recent advancements in llm red-teaming: Techniques, defenses, and ethical considerations. arXiv preprint arXiv:2410.09097. — [arXiv:2410.09097](https://arxiv.org/abs/2410.09097) find
  34. Rusheb Shah, Quentin Feuillade-Montixi, Soroush Pour, Arush Tagade, Stephen Casper, and Javier Rando. 2023. Scalable and transferable black-box jailbreaks for language models via persona modulation. arXiv preprint arXiv:2311.03348. — [arXiv:2311.03348](https://arxiv.org/abs/2311.03348) find
  35. Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo, and Chaowei Xiao. 2024. Jailbreakv: A benchmark for assessing the robustness of multimodal large language models against jailbreak attacks. In First Conference on Language Modeling. find
  36. Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2023. "do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. arXiv preprint arXiv:2308.03825.
  37. Siyuan Ma, Weidi Luo, Yu Wang, and Xiaogeng Liu. 2024. Visual-roleplay: Universal jailbreak attack on multimodal large language models via role-playing image character. Preprint, arXiv:2405.20773. — [arXiv:2405.20773](https://arxiv.org/abs/2405.20773) find
  38. B. Sima, L. Cong, W. Wang, and K. He. 2025. Viscra: A visual chain reasoning attack for jailbreaking multimodal large language models. arXiv preprint arXiv:2505.19684.
  39. Ziqi Miao, Yi Ding, Lijun Li, and Jing Shao. 2025. Visual contextual attack: Jailbreaking mllms with image-driven context injection. arXiv preprint arXiv:2507.02844.
  40. Cheng Wang, Yue Liu, Baolong Li, Duzhen Zhang, Zhongzhi Li, and Junfeng Fang. 2025a. Safety in large reasoning models: A survey. arXiv preprint arXiv:2504.17704. find
  41. Cheng Wang, Zeming Wei, Qin Liu, and Muhao Chen. 2025b. False sense of security: Why probing-based malicious input detection fails to generalize. arXiv preprint arXiv:2509.03888.
  42. Siyuan Wang, Zhuohan Long, Zhihao Fan, and Zhongyu Wei. 2024a. From llms to mllms: Exploring the landscape of multimodal jailbreaking. arXiv preprint arXiv:2406.14859. find
  43. Yu Wang, Xiaogeng Liu, Yu Li, Muhao Chen, and Chaowei Xiao. 2024b. AdaShield: Safeguarding multimodal large language models from structure-based attack via adaptive shield prompting. In European Conference on Computer Vision (ECCV). find
  44. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: How does llm safety training fail? In Advances in Neural Information Processing Systems.
  45. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824– 24837. xAI Team. 2024. Grok-2 beta release. http…
  46. Shiji Zhao, Ranjie Duan, Fengxiang Wang, Chi Chen, Caixin Kang, Shouwei Ruan, Jialing Tao, YueFeng Chen, Hui Xue, and Xingxing Wei. 2025. Jailbreaking multimodal large language models via shuffle inconsistency. arXiv preprint arXiv:2501.04931. — [arXiv:2501.04931](https://arxiv.org/abs/2501.04931) find
  47. Hao Zheng, Zirui Pang, Ling Li, and Zhijie Deng. 2025. Offside: Benchmarking unlearning misinformation in multimodal large language models. Preprint, arXiv:2510.22535.
  48. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hu-2026-gambit-gamified-jailbreak-framework-multimodal/