Teams of LLM Agents can Exploit Zero-Day Vulnerabilities
Paper · 2026 · Yuxuan Zhu, Antony Kellermann, Akul Gupta et al.
Summary
The paper asks whether LLM-based agents can exploit real-world zero-day vulnerabilities, meaning flaws whose description the attacker does not have. Earlier single ReAct-style agents did poorly here. They lose track of long-horizon plans as context grows, and they struggle to back out of one attack type and try another. The authors propose HPTSA, a multi-agent system with three layers. A hierarchical planner explores the target website and decides which pages and vulnerability classes to pursue. A team manager chooses expert subagents, collects their results and can re-run them. The task-specific expert agents each come with their own tools, documents and prompts. The system is built on LangChain/LangGraph and mainly uses GPT-4. It is evaluated on a new benchmark of 14 reproducible web vulnerabilities in open-source software, all disclosed after GPT-4's knowledge cutoff. HPTSA clearly beats single agents, MetaGPT and standard scanners. The work matters because it gives evidence that autonomous AI hacking is practical and affordable, which has consequences for both attackers and defenders.
Key points
- The benchmark has 14 real web vulnerabilities, all past GPT-4's cutoff. Examples include XSS in Travel Journal, CSRF/XSS in flusity-CMS, SQLi in Dolibarr, and privilege escalation in Zabbix and LedgerSMB.
- There are six expert agents: XSS, SQLi, CSRF, SSTI, ZAP and a generic web agent. All can use Playwright, a terminal and file tools. The SQLi agent also has sqlmap, and the ZAP agent has the ZAP scanner.
- HPTSA reaches 42% pass@5. This is up to 4.3× better than prior agent frameworks and within 1.8× of a GPT-4 agent that is given the vulnerability description. Open-source vulnerability scanners score 0%.
- Removing the expert agents lowers performance. Open-source LLMs accessed through Fireworks solved none of the tasks.
- With GPT-4, an average run costs $4.39, or about $24.4 per successful exploit at an 18% overall success rate. This is roughly 3.1× cheaper than an estimated $75 for a human expert. The findings were disclosed to OpenAI.
Institutions
United StatesFields, topics and concepts
📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works
Cite
Cite as Yuxuan Zhu, Antony Kellermann, Akul Gupta, Philip Li, Richard Fang, Rohan Bindu and Daniel Kang. 2026. Teams of LLM Agents can Exploit Zero-Day Vulnerabilities. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).
@inproceedings{zhu2026teams,
title = {{Teams of LLM Agents can Exploit Zero-Day Vulnerabilities}},
author = {Yuxuan Zhu, Antony Kellermann, Akul Gupta, Philip Li, Richard Fang, Rohan Bindu and Daniel Kang},
booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
year = {2026},
doi = {10.18653/v1/2026.eacl-long.2},
url = {https://doi.org/10.18653/v1/2026.eacl-long.2}
}References
- Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
- Anthropic. 2024. A new initiative for developing third-party model evaluations. find
- Simon Bennetts. 2013. AppSec USA. find
- Leyla Bilge and Tudor Dumitraş. 2012. Before we knew it: an empirical study of zero-day attacks in the real world. In Proceedings of the 2012 ACM conference on Computer and communications security, pages 833–844. \#\# Limitations find
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhu-2026-teams-llm-agents-can-exploit/