Elamur Research⌕
☰
home / papers / 2025 / Attacking Vision-Language Computer Agents via Pop-ups

Attacking Vision-Language Computer Agents via Pop-ups

Paper · 2025 · Yanzhe Zhang, Tao Yu and Diyi Yang

Summary

The paper examines a simple but overlooked risk for computer-use agents built on vision-language models (VLMs): visible adversarial pop-ups that an experienced person would dismiss. The authors use a threat model in which an attacker controls a clickable image on screen, for example through a purchased ad slot, cross-site scripting or a phishing message. The goal is to get the agent to click that image instead of doing its task. Each pop-up has four parts: an attention hook, an instruction, an info banner and an ALT descriptor for the a11y tree. The pop-ups were injected into OSWorld and VisualWebArena. Screenshot agents and Set-of-Mark (SoM) agents were then tested on five closed-source VLM backbones. Agents clicked the pop-ups 86% of the time on average, and task success dropped by 47%. Basic defenses, such as prompts telling the agent to ignore pop-ups or an advertisement label, did little to help. The study shows that current agents cannot tell legitimate interface elements from malicious ones. This means they need safety training and human oversight before real-world use.

Key points

  • Backbones tested were GPT-4-Turbo, GPT-4o, Gemini 1.5, Claude 3.5 Sonnet and Claude 3.5 Sonnet v2. In the default full-knowledge setting, attack success rate (ASR) was above 80% on OSWorld and above 60% on VisualWebArena.
  • Among all actions taken by attacked agents, 92.7% (OSWorld) and 73.1% (VisualWebArena) were clicks on the pop-ups. Many tasks ran until the step limit.
  • The attention hook works best when it contains an LLM-written summary of the real user query. Replacing it with a fake virus alert or a guessed user intent lowered ASR by about 61% in relative terms. Knowing the pop-up position or the agent framework mattered much less.
  • Defensive system prompts and "ADVERTISEMENT" notices reduced ASR by at most 25% relative.
  • Manual analysis of successful attacks showed that screenshot agents mostly focused on the fake "OK" button (52%), while SoM agents mostly focused on the summarized query (62%). Code is released at github.com/SALT-NLP/PopupAttack.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksInformation Systems · 1,437 works

🧭 Topics 1Research subjects and languages the work is about.Vision & language · 657 works

Cite

Cite as Yanzhe Zhang, Tao Yu and Diyi Yang. 2025. Attacking Vision-Language Computer Agents via Pop-ups. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.411

@inproceedings{zhang2025attacking,
  title = {{Attacking Vision-Language Computer Agents via Pop-ups}},
  author = {Yanzhe Zhang, Tao Yu and Diyi Yang},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.411},
  url = {https://doi.org/10.18653/v1/2025.acl-long.411}
}

References

42 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sara Abdali, Jia He, CJ Barberan, and Richard Anarfi. 2024. Can llms be fooled? investigating vulnerabilities in llms. arXiv preprint arXiv:2407.20529. — [arXiv:2407.20529](https://arxiv.org/abs/2407.20529) find
  2. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  3. Saaket Agashe, Jiuzhou Han, Shuyu Gan, Jiachen Yang, Ang Li, and Xin Eric Wang. 2024. Agent s: An open agentic framework that uses computers like a human. Preprint, arXiv:2410.08164. — [arXiv:2410.08164](https://arxiv.org/abs/2410.08164) find
  4. Anthropic. 2024. The claude 3 model family: Opus, sonnet, haiku. Technical report, Anthropic. Available at: https://www.anthropic.com/news/claude-3-family. ↗
  5. Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu. 2024. Seeclick: Harnessing gui grounding for advanced visual gui agents. Preprint, arXiv:2401.10935.
  6. Federal Trade Commission et al. 2013. . com disclosures: how to make effective disclosures in digital advertising. March, http://www.ftc. gov/sites/default/files/attachments/press-releases/ftc-staff-revises-online-advertising-disclosureguidelines/130312dotcomdisclosures. pdf. ↗
  7. Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. 2023. Mind2web: Towards a generalist agent for the web. Preprint, arXiv:2306.06070.
  8. Isatou Hydara, Abu Bakar Md Sultan, Hazura Zulzalil, and Novia Admodisastro. 2015. Current state of research on cross-site scripting (xss)–a systematic literature review. Information and Software Technology, 58:170–186. find
  9. Yuta Imamura, Rintaro Orito, Kritsana Chaikaew, Célia Manardo, Pattara Leelaprute, Masaya Sato, and Toshihiro Yamauchi. 2019. Threat analysis of fake virus alerts using webview monitor. In 2019 Seventh International Symposium on Computing and Networking (CANDAR), pages 28–36. find
  10. Jasleen Kaur, Urvashi Garg, and Gourav Bathla. 2023. Detection of cross-site scripting (xss) attacks using machine learning techniques: a review. Artificial Intelligence Review, 56(11):12725–12769. find
  11. Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, and Daniel Fried. 2024. Visualwebarena: Evaluating multimodal agents on realistic visual web tasks. Preprint, arXiv:2401.13649.
  12. Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. 2024. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. Advances in Neural Information Processing Systems, 36.
  13. Ponnurangam Kumaraguru, Yong Rhee, Alessandro Acquisti, Lorrie Faith Cranor, Jason Hong, and Elizabeth Nunge. 2007. Protecting people from phishing: the design and evaluation of an embedded training email system. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 90… find
  14. Zeyi Liao, Lingbo Mo, Chejian Xu, Mintong Kang, Jiawei Zhang, Chaowei Xiao, Yuan Tian, Bo Li, and Huan Sun. 2024. Eia: Environmental injection attack on generalist web agents for privacy leakage. Preprint, arXiv:2409.11295. — [arXiv:2409.11295](https://arxiv.org/abs/2409.11295) find
  15. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2023. Autodan: Generating stealthy jailbreak prompts on aligned large language models. Preprint, arXiv:2310.04451.
  16. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2023. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. Preprint, arXiv:2310.02255.
  17. Xinbei Ma, Yiting Wang, Yao Yao, Tongxin Yuan, Aston Zhang, Zhuosheng Zhang, and Hai Zhao. 2024. Caution for the environment: Multimodal agents are susceptible to environmental distractions. Preprint, arXiv:2408.02544. — [arXiv:2408.02544](https://arxiv.org/abs/2408.02544) find
  18. OpenAI. 2024. Gpt-4o system card. https://openai.com/index/gpt-4o-system-card/. ↗
  19. OpenAI. 2025. A practical guide to building agents. Technical report, OpenAI. find
  20. Pooja Patel, Dawn M Sarno, Joanna E Lewis, Mindy Shoss, Mark B Neider, and Corey J Bohil. 2019. Perceptual representation of spam and phishing emails. Applied cognitive psychology, 33(6):1296–1304. find
  21. Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, et al. 2025. Ui-tars: Pioneering automated gui interaction with native agents. arXiv preprint arXiv:2501.12326.
  22. Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean-baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, et al. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.…
  23. Yangjun Ruan, Honghua Dong, Andrew Wang, Silviu Pitis, Yongchao Zhou, Jimmy Ba, Yann Dubois, Chris J. Maddison, and Tatsunori Hashimoto. 2023. Identifying the risks of lm agents with an lm-emulated sandbox. Preprint, arXiv:2309.15817. find
  24. Yijia Shao, Vinay Samuel, Yucheng Jiang, John Yang, and Diyi Yang. 2024. Collaborative gym: A framework for enabling and evaluating human-agent collaboration. Preprint, arXiv:2412.15701. find
  25. Aditya K Sood and Richard J Enbody. 2011. Malvertising–exploiting web advertising. Computer Fraud & Security, 2011(4):11–16. find
  26. Eric Wallace, Kai Xiao, Reimar Leike, Lilian Weng, Johannes Heidecke, and Alex Beutel. 2024. The instruction hierarchy: Training llms to prioritize privileged instructions. Preprint, arXiv:2404.13208. find
  27. Chen Henry Wu, Jing Yu Koh, Ruslan Salakhutdinov, Daniel Fried, and Aditi Raghunathan. 2024. Adversarial attacks on multimodal agents. Preprint, arXiv:2406.12814. find
  28. Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu. 2024. Osworld: Benchmarking multimodal agents for open-ended tasks in…
  29. Xinyu Xing, Wei Meng, Byoungyoung Lee, Udi Weins-berg, Anmol Sheth, Roberto Perdisci, and Wenke Lee. 2015. Understanding malvertising through ad-injecting browser extensions. In Proceedings of the 24th international conference on world wide web, pages 1286–1295. find
  30. Chejian Xu, Mintong Kang, Jiawei Zhang, Zeyi Liao, Lingbo Mo, Mengqi Yuan, Huan Sun, and Bo Li. 2024a. Advweb: Controllable black-box attacks on vlm-powered web agents. Preprint, arXiv:2410.17401. — [arXiv:2410.17401](https://arxiv.org/abs/2410.17401) find
  31. Yiheng Xu, Zekun Wang, Junli Wang, Dunjie Lu, Tianbao Xie, Amrita Saha, Doyen Sahoo, Tao Yu, and Caiming Xiong. 2024b. Aguvis: Unified pure vision agents for autonomous gui interaction. Preprint, arXiv:2412.04454. — [arXiv:2412.04454](https://arxiv.org/abs/2412.04454) find
  32. Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. 2023. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v. Preprint, arXiv:2310.11441. find
  33. Wenkai Yang, Xiaohan Bi, Yankai Lin, Sishuo Chen, Jie Zhou, and Xu Sun. 2024. Watch out for your agents! investigating backdoor threats to llm-based agents. Preprint, arXiv:2402.11208. — [arXiv:2402.11208](https://arxiv.org/abs/2402.11208) find
  34. Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. 2024. tau-bench: A benchmark for tool-agent-user interaction in real-world domains. Preprint, arXiv:2406.12045.
  35. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. Preprint, arXiv:2210.03629.
  36. Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. 2023. Mmmu: A massive multi-disc…
  37. Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. 2024. How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms. Preprint, arXiv:2401.06373.
  38. Qiusi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. 2024. Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents. Preprint, arXiv:2403.02691.
  39. Zhuohao Jerry Zhang, Eldon Schoop, Jeffrey Nichols, Anuj Mahajan, and Amanda Swearngin. 2024. From interaction to impact: Towards safer ai agents through understanding and evaluating ui operation impacts. Preprint, arXiv:2410.09006. — [arXiv:2410.09006](https://arxiv.org/abs/2410.09006) find
  40. Boyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun, and Yu Su. 2024. Gpt-4v(ision) is a generalist web agent, if grounded. Preprint, arXiv:2401.01614.
  41. Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. 2023. Webarena: A realistic web environment for building autonomous agents. Preprint, arXiv:2307.13854.
  42. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. Preprint, arXiv:2307.15043.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2025-attacking-vision-language-computer-agents/