Elamur Research⌕
☰
home / papers / 2024 / Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models

Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models

Paper · 2024 · Yongshuo Zong, Ondrej Bohdal, Tingyang Yu et al.

Summary

Vision large language models (VLLMs) are usually built by fine-tuning an already safety-aligned LLM on vision-language instruction data. The authors show that this step weakens the original alignment, so the resulting models follow harmful requests and fall to simple jailbreaks more often than their base LLMs. Analysing LLaVA-v1.5 and MiniGPT-v2, they trace part of the problem to harmful samples in LLM-generated instruction sets. Removing those samples restores safety only partially. As a remedy, they build VLGuard, which they present as the first safety instruction-following dataset for VLLMs. It pairs images with safe and unsafe instructions across several harm categories and includes an evaluation suite. VLGuard can be used for post-hoc fine-tuning of an existing model or mixed into standard instruction tuning. Both strategies cut attack success rates sharply, often to near zero for several black-box attacks, while helpfulness stays the same or improves. Because the dataset is small, the safety gain costs little compute, which makes it a practical baseline for safeguarding open VLLMs.

Key points

  • Setup: The analysed models are LLaVA-v1.5 (7B and 13B, full fine-tuning and LoRA) and MiniGPT-v2.
    • Safety was tested on AdvBench (500 harmful instructions, vanilla and with suffix injection).
    • Exaggerated safety was tested on XSTest (200 unsafe and 250 safe prompts).
    • Helpfulness was tested on MMLU and AlpacaEval 2.0.
    • ASR was measured by string matching, with extra checks via Llama-Guard and GPT-4V.
  • Diagnostic findings:
    • Harmful content appears in fine-tuning sources such as ShareGPT and Unnatural Instruction.
    • LoRA variants turned out less safe than fully fine-tuned ones trained on identical data.
    • Retraining on a "clean" dataset lowers ASR, but it stays above the base LLMs.
  • Resource: The VLGuard training split contains 2,000 images. Its curation relied on GPT-4V, whose cost limited the scale. Code is available at github.com/ys-zong/VLGuard.
  • Human evaluation: VLGuard-tuned models beat the originals on unsafe inputs in 86.67–100% of cases. On safe inputs the win rate was 42–55%, meaning helpfulness was roughly preserved.
  • Limitations: The authors acknowledge that sophisticated attacks may still jailbreak the models. The work was published at ICML 2024.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksBias, fairness & safety · 587 works

Cite

Cite as Yongshuo Zong, Ondrej Bohdal, Tingyang Yu, Yongxin Yang and Timothy Hospedales. 2024. Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models. arXiv:2402.02207.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2402.02207 · arXiv 2402.02207

@misc{zong2024safety,
  title = {{Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models}},
  author = {Yongshuo Zong, Ondrej Bohdal, Tingyang Yu, Yongxin Yang and Timothy Hospedales},
  year = {2024},
  doi = {10.48550/arxiv.2402.02207},
  eprint = {2402.02207},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2402.02207}
}

References

39 references, 28 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Albert, A. com/. URL https://www.jailbreakchat. Gurari, D., Li, Q., Stangl, A. J., Guo, A., Lin, C., Grauman, K., Luo, J., and Bigham, J. P. Vizwiz grand challenge: Answering visual questions from blind people. In CVPR, pp. 3608–3617, 2018. ↗
  2. Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., and Zhou, J. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966, 2023.
  3. Ha, E., Kim, H., Hong, S. C., and Na, D. Hod: A benchmark dataset for harmful object detection. arXiv preprint arXiv:2310.05192, 2023. — [arXiv:2310.05192](https://arxiv.org/abs/2310.05192) find
  4. Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., Das-Sarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022. find
  5. Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020.
  6. Barak, B. Another jailbreak for gpt4: Talk to it in morse code, 2023. URL https://twitter.com/boazbaraktcs/status/ 1637657623100096513. ↗
  7. Honovich, O., Scialom, T., Levy, O., and Schick, T. Unnatural instructions: Tuning language models with (almost) no human labor. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Long Papers), 2023.
  8. Bianchi, F., Suzgun, M., Attanasio, G., Röttger, P., Jurafsky, D., Hashimoto, T., and Zou, J. Safety-tuned llamas: Hu, E. J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. LoRA: Low-rank adaptation of large language models. In ICLR, 2022.
  9. Jailbreaking black-box llms automatically. arXiv preprint arXiv:2312.02119, 2023.
  10. OpenAI, R. Gpt-4 technical report. arXiv, pp. 2303–08774, 2023.
  11. Inan, H., Upasani, K., Chi, J., Rungta, R., Iyer, K., Mao, Y., Tontchev, M., Hu, Q., Fuller, B., Testuggine, D., et al. Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674, 2023.
  12. Pramanick, S., Sharma, S., Dimitrov, D., Akhtar, M. S., Nakov, P., and Chakraborty, T. MOMENTA: A multimodal framework for detecting harmful memes and their targets. In Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 4439–4455, 2021.
  13. Kiela, D., Firooz, H., Mohan, A., Goswami, V., Singh, A., Ringshia, P., and Testuggine, D. The hateful memes challenge: Detecting hate speech in multimodal memes. NeurIPS, 33, 2020.
  14. Qi, X., Huang, K., Panda, A., Wang, M., and Mittal, P. Visual adversarial examples jailbreak large language models. arXiv preprint arXiv:2306.13213, 2023a.
  15. Li, B., Zhang, Y., Chen, L., Wang, J., Yang, J., and Liu, Z. Otter: A multi-modal model with in-context instruction tuning. arXiv preprint arXiv:2305.03726, 2023a.
  16. Qi, X., Zeng, Y., Xie, T., Chen, P.-Y., Jia, R., Mittal, P., and Henderson, P. Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693, 2023b.
  17. Li, X., Zhang, T., Dubois, Y., Taori, R., Gulrajani, I., Guestrin, C., Liang, P., and Hashimoto, T. B. Alpacaeval: An automatic evaluator of instruction-following models. https://github.com/tatsu-lab/ alpaca_eval, 2023b. ↗
  18. Röttger, P., Kirk, H. R., Vidgen, B., Attanasio, G., Bianchi, F., and Hovy, D. Xstest: A test suite for identifying exaggerated safety behaviours in large language models. arXiv preprint arXiv:2308.01263, 2023.
  19. Lin, J., Yin, H., Ping, W., Lu, Y., Molchanov, P., Tao, A., Mao, H., Kautz, J., Shoeybi, M., and Han, S. Vila: On pre-training for visual language models. arXiv preprint arXiv:2312.07533, 2023.
  20. ShareGPT, 2023. URL https://sharegpt.com/. ↗
  21. Shen, X., Chen, Z., Backes, M., Shen, Y., and Zhang, Y. ” do anything now”: Characterizing and evaluating in-the-wild jailbreak prompts on large language models. arXiv preprint arXiv:2308.03825, 2023.
  22. Liu, H., Li, C., Li, Y., and Lee, Y. J. Improved baselines with visual instruction tuning. arXiv preprint arXiv:2310.03744, 2023a.
  23. Liu, H., Li, C., Wu, Q., and Lee, Y. J. Visual instruction tuning. NeurIPS, 2023b.
  24. Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/ stanford_alpaca, 2023. ↗
  25. Liu, X., Zhu, Y., Lan, Y., Yang, C., and Qiao, Y. Query-relevant images jailbreak large multi-modal models. arXiv preprint arXiv:2311.17600, 2023c.
  26. Team, G., Anil, R., Borgeaud, S., Wu, Y., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023.
  27. Liu, Y., Deng, G., Xu, Z., Li, Y., Zheng, Y., Zhang, Y., Zhao, L., Zhang, T., and Liu, Y. Jailbreaking chatgpt via prompt engineering: An empirical study. arXiv preprint arXiv:2305.13860, 2023d.
  28. Liu, Y., Duan, H., Zhang, Y., Li, B., Zhang, S., Zhao, W., Yuan, Y., Wang, J., He, C., Liu, Z., et al. Mmbench: Is your multi-modal model an all-around player? arXiv, 2023e.
  29. Wang, W., Lv, Q., Yu, W., Hong, W., Qi, J., Wang, Y., Ji, J., Yang, Z., Zhao, L., Song, X., et al. Cogvlm: Visual expert for pretrained language models. arXiv preprint arXiv:2311.03079, 2023.
  30. Lu, P., Mishra, S., Xia, T., Qiu, L., Chang, K.-W., Zhu, S.-C., Tafjord, O., Clark, P., and Kalyan, A. Learn to explain: Multimodal reasoning via thought chains for science question answering. In NeurIPS, 2022. find
  31. Wei, A., Haghtalab, N., and Steinhardt, J. Jailbroken: How does llm safety training fail? NeurIPS, 2023.
  32. Ye, Q., Xu, H., Ye, J., Yan, M., Liu, H., Qian, Q., Zhang, J., Huang, F., and Zhou, J. mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration. arXiv preprint arXiv:2311.04257, 2023.
  33. Mehrotra, A., Zampetakis, M., Kassianik, P., Nelson, B., Anderson, H., Singer, Y., and Karbasi, A. Tree of attacks: Yin, S., Fu, C., Zhao, S., Li, K., Sun, X., Xu, T., and Chen, E. A survey on multimodal large language models. arXiv preprint arXiv:2306.13549, 2023.
  34. Yu, W., Yang, Z., Li, L., Wang, J., Lin, K., Liu, Z., Wang, X., and Wang, L. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.02490, 2023.
  35. Zeng, E., Kohno, T., and Roesner, F. Bad news: Clickbait and deceptive ads on news and misinformation websites. In Workshop on Technology and Consumer Protection, pp. 1–11, 2020. find
  36. Zhang, P., Wang, X. D. B., Cao, Y., Xu, C., Ouyang, L., Zhao, Z., Ding, S., Zhang, S., Duan, H., Yan, H., et al. Internlm-xcomposer: A vision-language large model for advanced text-image comprehension and composition. arXiv preprint arXiv:2309.15112, 2023.
  37. Zhao, C., Mangat, J., Koujalgi, S., Squicciarini, A., and Caragea, C. Privacyalert: a dataset for image privacy prediction. In Proceedings of the International AAAI Conference on Web and Social Media, volume 16, 2022. find
  38. Zong, Y., Mac Aodha, O., and Hospedales, T. Self-supervised multimodal learning: A survey. arXiv preprint arXiv:2304.01008, 2023. — [arXiv:2304.01008](https://arxiv.org/abs/2304.01008) find
  39. Zou, A., Wang, Z., Kolter, J. Z., and Fredrikson, M. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zong-2024-safety-fine-tuning-almost-no/