Elamur Research⌕
☰
home / papers / 2026 / Enhancing the Safety of Medical Vision-Language Models by Synthetic Demonstrations

Enhancing the Safety of Medical Vision-Language Models by Synthetic Demonstrations

Paper · 2026 · Zhiyu Xue, Reza Abbasi-Asl and Ramtin Pedarsani

Summary

Generative medical vision-language models (Med-VLMs) can be pushed into answering malicious requests, such as using a scan to commit fraud. Jailbreak attacks on the image or the prompt can bypass whatever refusal behaviour these models have. Fixing this through fine-tuning is costly because medical safety data requires expert annotation. Overly cautious defences also create a second problem: legitimate clinical questions get refused.

The authors propose a defence applied only at inference time. It relies on in-context learning with LLM-generated demonstrations, so no clinicians are needed. They build two kinds of examples covering many imaging modalities and body regions:

  • Harmful-Refusal (H-R): harmful questions paired with refusals.
  • Benign-Affirmative (B-A): ordinary clinical questions paired with helpful answers.

The demonstrations are placed in the model's context before the user query. Using H-R demonstrations alone makes models safer. With only a few demonstrations, however, the models start rejecting benign queries. A larger demonstration budget reduces this over-defense. A mixed B-A/H-R set with a tunable ratio offers a workable safety–utility balance when the budget is small. The approach matters because it is cheap, needs no retraining, and fits deployed clinical assistants.

Key points

  • The evaluation draws on medical imaging datasets spanning nine modalities, including CT, MRI, X-ray, dermoscopy, mammography and funduscopy.
  • Robustness is tested against four jailbreak attacks:
    • PGD, which adds noise to the image;
    • GCG, which appends an adversarial suffix to the prompt;
    • the template-based attacks AIM and Refusal Suppression (RS).
  • Results are reported as attack success rate (ASR) and refusal rate (RR). Mixing methods and mixing ratios are compared as trade-off curves.
  • H-R-only demonstrations cause over-defense under few-shot budgets. Mixed demonstrations restore answers to benign queries while still refusing harmful ones.
  • The code is released at github.com/chrisyxue/Med_Demon. The stated limitations are:
    • synthetic examples may miss nuanced clinical reasoning;
    • test queries are short and unambiguous;
    • biases in the LLMs that generate the demonstrations may carry over.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of California, Santa BarbaraSanta Barbara · United States67 works · 10,550 citations🎓University of California, San FranciscoSan Francisco · United States4 works · 9,908 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 2Research subjects and languages the work is about.Vision & language · 811 worksBias, fairness & safety · 587 works

Cite

Cite as Zhiyu Xue, Reza Abbasi-Asl and Ramtin Pedarsani. 2026. Enhancing the Safety of Medical Vision-Language Models by Synthetic Demonstrations. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-long.197

@inproceedings{xue2026enhancing,
  title = {{Enhancing the Safety of Medical Vision-Language Models by Synthetic Demonstrations}},
  author = {Zhiyu Xue, Reza Abbasi-Asl and Ramtin Pedarsani},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.197},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.197}
}

References

41 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Cem Anil, Esin Durmus, Nina Panickssery, Mrinank Sharma, Joe Benton, Sandipan Kundu, Joshua Batson, Meg Tong, Jesse Mu, Daniel Ford, et al. 2025. Many-shot jailbreaking. Advances in Neural Information Processing Systems, 37:129696–129742. find
  3. Simran Arora, Avanika Narayan, Mayee F Chen, Laurel Orr, Neel Guha, Kush Bhatia, Ines Chami, Frederic Sala, and Christopher Ré. 2022. Ask me anything: A simple strategy for prompting language models. arXiv preprint arXiv:2210.02441.
  4. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. 2023. Jailbreaking black box large language models in twenty queries. In R0-FoMo: Robustness of Few-shot and Zero-shot Learning in Large Foundation Models.
  5. Sizhe Chen, Julien Piet, Chawin Sitawarin, and David Wagner. 2024a. Struq: Defending against prompt injection with structured queries. arXiv preprint arXiv:2402.06363. — [arXiv:2402.06363](https://arxiv.org/abs/2402.06363) find
  6. Zhihong Chen, Maya Varma, Jean-Benoit Delbrouck, Magdalini Paschali, Louis Blankemeier, Dave Van Veen, Jeya Maria Jose Valanarasu, Alaa Youssef, Joseph Paul Cohen, Eduardo Pontes Reis, et al. 2024b. Chexagent: Towards a foundation model for chest x-ray interpretation. arXiv preprint arXiv:2401.12208… — [arXiv:2401.12208](https://arxiv.org/abs/2401.12208) find
  7. Simon Geisler, Tom Wollschläger, MHI Abdalla, Johannes Gasteiger, and Stephan Günnemann. 2024. Attacking large language models with projected gradient descent. arXiv preprint arXiv:2402.09154. — [arXiv:2402.09154](https://arxiv.org/abs/2402.09154) find
  8. Shantanu Ghosh, Clare B Poynton, Shyam Visweswaran, and Kayhan Batmanghelich. 2024. Mammo-clip: A vision language foundation model to enhance data efficiency and robustness in mammography. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 632–642. Sprin… find
  9. Iryna Hartsock and Ghulam Rasool. 2024. Vision-language models for medical report generation and visual question answering: A review. Frontiers in Artificial Intelligence, 7:1430984. find
  10. Xijie Huang, Xinyuan Wang, Hantao Zhang, Jiawen Xi, Jingkun An, Hao Wang, and Chengwei Pan. 2024. Cross-modality jailbreak and mismatched attacks on medical multimodal large language models. arXiv preprint arXiv:2405.20775. — [arXiv:2405.20775](https://arxiv.org/abs/2405.20775) find
  11. Noor Hussein, Fahad Shamshad, Muzammal Naseer, and Karthik Nandakumar. 2024. Promptsmooth: Certifying robustness of medical vision-language models via prompt learning. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 698–708. Springer. find
  12. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825. find
  13. Heegyu Kim, Sehyun Yuk, and Hyunsouk Cho. 2024. Break the breakout: Reinventing lm defense against jailbreak attacks with self-refinement. arXiv preprint arXiv:2402.15180. — [arXiv:2402.15180](https://arxiv.org/abs/2402.15180) find
  14. Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. 2023. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4015–4026.
  15. Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. 2024a. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems, 36.
  16. Hao Li, Han Liu, Dewei Hu, Jiacheng Wang, and Ipek Oguz. 2024b. Prism: A promptable and robust interactive segmentation model with visual prompts. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 389–399. Springer. find
  17. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2024a. Visual instruction tuning. Advances in neural information processing systems, 36.
  18. Jiachang Liu, Dinghan Shen, Yizhe Zhang, William B Dolan, Lawrence Carin, and Weizhu Chen. 2022. What makes good in-context examples for gpt-3? In Proceedings of Deep Learning Inside Out (Dee-LIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pages 1…
  19. Zhendong Liu, Yuanbi Nie, Yingshui Tan, Xiangyu Yue, Qiushi Cui, Chongjun Wang, Xiaoyong Zhu, and Bo Zheng. 2024b. Safety alignment for vision language models. arXiv preprint arXiv:2405.13581. — [arXiv:2405.13581](https://arxiv.org/abs/2405.13581) find
  20. Ming Y Lu, Bowen Chen, Drew FK Williamson, Richard J Chen, Melissa Zhao, Aaron K Chow, Kenji Ikemura, Ahrong Kim, Dimitra Pouli, Ankush Patel, et al. 2024. A multimodal generative ai copilot for human pathology. Nature, 634(8033):466– 473. find
  21. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Rethinking the role of demonstrations: What makes in-context learning work? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11048– 11064.
  22. Obioma Pelka, Sven Koitka, Johannes Rückert, Felix Nensa, and Christoph M Friedrich. 2018. Radiology objects in context (roco): a multimodal image dataset. In Intravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis: 7th Joint Inte… find
  23. Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. 2024. Visual adversarial examples jailbreak aligned large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 21527–21536.
  24. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748– 8763. P…
  25. Laria Reynolds and Kyle McDonell. 2021. Prompt programming for large language models: Beyond the few-shot paradigm. In Extended abstracts of the 2021 CHI conference on human factors in computing systems, pages 1–7.
  26. Sanjay Subramanian, Lucy Lu Wang, Ben Bogin, Sachin Mehta, Madeleine van Zuylen, Sravanthi Parasa, Sameer Singh, Matt Gardner, and Hannaneh Hajishirzi. 2020. Medicat: A dataset of medical images, captions, and textual references. Findings of the Association for Computational Linguistics: EMNLP. find
  27. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  28. Neeraj Varshney, Pavel Dolin, Agastya Seth, and Chitta Baral. 2024. The art of defending: A systematic evaluation and analysis of llm defense strategies on safety and over-defensiveness. In Findings of the Association for Computational Linguistics ACL 2024, pages 13111–13128. find
  29. Siyin Wang, Xingsong Ye, Qinyuan Cheng, Junwen Duan, Shimin Li, Jinlan Fu, Xipeng Qiu, and Xuanjing Huang. 2024. Cross-modality safety alignment. arXiv preprint arXiv:2406.15279. — [arXiv:2406.15279](https://arxiv.org/abs/2406.15279) find
  30. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2024. Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36.
  31. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022. Emergent abilities of large language models. Transactions on Machine Learning Research.
  32. Zeming Wei, Yifei Wang, Ang Li, Yichuan Mo, and Yisen Wang. 2023. Jailbreak and guard aligned language models with only few in-context demonstrations. arXiv preprint arXiv:2310.06387.
  33. Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye, and Lingpeng Kong. 2023. Self-adaptive in-context learning: An information compression perspective for in-context example selection and ordering. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)…
  34. Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu. 2023. Defending chatgpt against jailbreak attack via self-reminders. Nature Machine Intelligence, 5(12):1486–1496. find
  35. Jiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Lingpeng Kong. 2023. Compositional exemplars for in-context learning. In International Conference on Machine Learning, pages 39818–39833. PMLR.
  36. Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530.
  37. Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu, Jaspreet Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naveen Valluri, et al. 2023. Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs. arXiv preprint arXiv:2303.00915. find
  38. Omkar Thawkar, Abdelrahman Shaker, Sahal Shaji Mullappilly, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, Jorma Laaksonen, and Fahad Shahbaz Khan. 2023. Xraygpt: Chest radiographs summarization using medical vision-language models. arXiv preprint arXiv:2306.07971. — [arXiv:2306.07971](https://arxiv.org/abs/2306.07971) find
  39. Yujun Zhou, Yufei Han, Haomin Zhuang, Kehan Guo, Zhenwen Liang, Hongyan Bao, and Xiangliang Zhang. 2024. Defending jailbreak prompts via in-context adversarial game. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 20084–20105. find
  40. Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592.
  41. Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xue-2026-enhancing-safety-medical-vision-language/