Elamur Research⌕
☰
home / papers / 2026 / MulVul: Retrieval-augmented Multi-Agent Code Vulnerability Detection via Cross-Model Prompt Evolution

MulVul: Retrieval-augmented Multi-Agent Code Vulnerability Detection via Cross-Model Prompt Evolution

Paper · 2026 · Zihan Wu, Jie Xu, Yun Peng et al.

Summary

Detecting many kinds of code vulnerabilities with one LLM is difficult. Different weakness types require very different reasoning, and hand-crafting prompts for hundreds of CWE classes does not scale.

MulVul addresses this with a two-stage multi-agent design that follows the CWE hierarchy. A Router agent selects the top-k broad categories, and only the matching category-level Detector agents are then called to name the exact CWE type. Both kinds of agents ground their decisions in evidence retrieved from a knowledge base built on SCALE structured code representations. Detectors use contrastive examples and run independently, so one agent's error cannot spread to the others.

All agent prompts are created automatically by Cross-Model Prompt Evolution. One LLM (Claude) proposes and mutates candidate prompts, while a separate LLM (GPT-4o) runs and scores them. This avoids the bias that arises when a model judges its own revisions.

On PrimeVul, covering 130 CWE types, MulVul reaches a new best Macro-F1. It is also cheaper than competing agent pipelines, which makes it practical for large-scale code auditing.

Key points

  • Main results: On PrimeVul (C/C++ code, 130 CWE types), MulVul scores 34.79% Macro-F1, a 41.5% relative gain over the strongest baseline.
  • Prompt evolution ablation: Evolved prompts improve performance by 51.6% compared with manually written prompts.
  • Efficiency (per sample):
    • MulVul uses about 1,631 tokens and 10.98 s.
    • Reflexion uses 4,026 tokens and 22.85 s (27.40 F1).
    • MAD uses 5,915 tokens and 50.01 s (12.33 F1).
    • Single-pass GPT-4o scores only 3.86 F1.
  • Low-data behaviour: MulVul's F1 levels off near 63% at roughly 300 training samples per type. Fine-tuned methods (LLM×CPG, VISION) stall at about 35–38%.
  • Limitations:
    • Evaluation covers only C/C++ on PrimeVul.
    • Online detection needs 1+k API calls per sample.
    • Transfer of evolved prompts to open-weight models (Llama, Qwen, DeepSeek) has not been tested.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Chinese University of Hong KongHong Kong · China126 works · 14,073 citations🎓City University of Hong KongHong Kong · China45 works · 3,014 citations🎓Monash University MalaysiaSubang Jaya · Malaysia4 works · 10 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksInformation Systems · 1,437 worksSoftware · 92 works

Cite

Cite as Zihan Wu, Jie Xu, Yun Peng, Chun Yong Chong and Xiaohua Jia. 2026. MulVul: Retrieval-augmented Multi-Agent Code Vulnerability Detection via Cross-Model Prompt Evolution. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.391

@inproceedings{wu2026mulvul,
  title = {{MulVul: Retrieval-augmented Multi-Agent Code Vulnerability Detection via Cross-Model Prompt Evolution}},
  author = {Zihan Wu, Jie Xu, Yun Peng, Chun Yong Chong and Xiaohua Jia},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.391},
  url = {https://doi.org/10.18653/v1/2026.acl-long.391}
}

References

29 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Saikat Chakraborty, Rahul Krishna, Yangruibo Ding, and Baishakhi Ray. 2021. Deep learning based vulnerability detection: Are we there yet? IEEE Transactions on Software Engineering, 48(9):3280–3296. find
  2. Yangruibo Ding, Yanjun Fu, Omniyyah Ibrahim, Chawin Sitawarin, Xinyun Chen, Basel Alomair, David Wagner, Baishakhi Ray, and Yizheng Chen. 2024. Vulnerability detection with code language models: How far are we? In 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE), pages 469–… find
  3. David Egea, Barproda Halder, and Sanghamitra Dutta. 2025. Vision: Robust and interpretable code vulnerability detection leveraging counterfactual augmentation. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, volume 8, pages 812–823. find
  4. Jun Gao, Yun Peng, and Xiaoxue Ren. 2025. ReMind: Understanding deductive code reasoning in LLMs. arXiv preprint arXiv:2511.00488. — [arXiv:2511.00488](https://arxiv.org/abs/2511.00488) find
  5. Seyed Mohammad Ghaffarian and Hamid Reza Shahriari. 2017. Software vulnerability analysis and discovery using machine-learning and data-mining techniques: A survey. ACM computing surveys (CSUR), 50(4):1–36. find
  6. Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, and Jian Yin. 2022. Unixcoder: Unified cross-modal pre-training for code representation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7212–7225.
  7. Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie LIU, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, and 1 others. 2021. Graphcodebert: Pre-training code representations with data flow. In International Conference on Learning Representations.
  8. Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang. 2024. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. In The Twelfth International Conference on Learning Representations. find
  9. Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, and 1 others. 2023. Metagpt: Meta programming for a multi-agent collaborative framework. In The Twelfth International Conference on Learning Representations.
  10. Ahmed Lekssays, Hamza Mouhcine, Khang Tran, Ting Yu, and Issa Khalil. 2025. LLMxCPG: Context-Aware vulnerability detection through code property Graph-Guided large language models. In 34th USENIX Security Symposium (USENIX Security 25), pages 489–507. find
  11. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, and 1 others. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33:9459–…
  12. Zhen Li, Deqing Zou, Shouhuai Xu, Hai Jin, Yawei Zhu, and Zhaoxuan Chen. 2021. Sysevr: A framework for using deep learning to detect software vulnerabilities. IEEE Transactions on Dependable and Secure Computing, 19(4):2244–2258. find
  13. Zhen Li, Deqing Zou, Shouhuai Xu, Xinyu Ou, Hai Jin, Sujuan Wang, Zhijun Deng, and Yuyi Zhong. 2018. Vuldeepecker: A deep learning-based system for vulnerability detection. In 25th Annual Network and Distributed System Security Symposium, NDSS 2018, San Diego, California, USA, February 18-21, 2018.… find
  14. Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. 2024. Encouraging divergent thinking in large language models through multi-agent debate. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17…
  15. Jie Lin and David Mohaisen. 2025. From large to mammoth: A comparative evaluation of large language models in vulnerability detection. In Proceedings of the 2025 Network and Distributed System Security Symposium (NDSS). find
  16. Shuai Lu, Nan Duan, Hojae Han, Daya Guo, Seung-won Hwang, and Alexey Svyatkovskiy. 2022. Reacc: A retrieval-augmented code completion framework. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6227–6240.
  17. Qiheng Mao, Zhenhao Li, Xing Hu, Kui Liu, Xin Xia, and Jianling Sun. 2025. Towards explainable vulnerability detection with large language models. IEEE Transactions on Software Engineering. find
  18. MITRE. 2024. CWE List Version 4.19. https://cwe.mitre.org/data/index.html. Page last updated: November 19, 2024. Accessed: 2026-01-01. ↗
  19. Yun Peng, Shuzheng Gao, Cuiyun Gao, Yintong Huo, and Michael Lyu. 2024. Domain knowledge matters: Improving prompts with fix templates for repairing python type errors. In Proceedings of the 46th ieee/acm international conference on software engineering, pages 1–13. find
  20. Yun Peng, Kisub Kim, Linghan Meng, and Kui Liu. 2025. icodereviewer: Improving secure code review with mixture of prompts. In Proceedings of the 40th IEEE/ACM International Conference on Automated Software Engineering. find
  21. Ze Sheng, Zhicheng Chen, Shuning Gu, Heqing Huang, Guofei Gu, and Jeff Huang. 2025. LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insights. ACM Computing Surveys, 58(5):1–35. find
  22. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, volume 36.
  23. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  24. Xin-Cheng Wen, Cuiyun Gao, Shuzheng Gao, Yang Xiao, and Michael R Lyu. 2024. Scale: Constructing structured natural language comment trees for software vulnerability detection. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 235–247. find
  25. Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, and 1 others. 2024. Autogen: Enabling next-gen llm applications via multi-agent conversations. In First Conference on Language Modeling. find
  26. Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. 2023. Large language models as optimizers. In The Twelfth International Conference on Learning Representations.
  27. Xin Zhou, Sicong Cao, Xiaobing Sun, and David Lo. 2025. Large language model for vulnerability detection and repair: Literature review and the road ahead. ACM Transactions on Software Engineering and Methodology, 34(5):1–31. find
  28. Yaqin Zhou, Shangqing Liu, Jingkai Siow, Xiaoning Du, and Yang Liu. 2019. Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks. Advances in neural information processing systems, 32. find
  29. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2022. Large language models are human-level prompt engineers. In The eleventh international conference on learning representations.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2026-mulvul-retrieval-augmented-multi-agent/