Elamur Research⌕
☰
home / papers / 2026 / RedCoder: Automated Multi-Turn Red Teaming for Code LLMs

RedCoder: Automated Multi-Turn Red Teaming for Code LLMs

Paper · 2026 · Wenjie Jacky Mo, Qin Liu, Xiaofei Wen et al.

Summary

Code LLMs can be manipulated into producing insecure code. Earlier red-teaming work in this area mostly depends on hand-crafted prompts or partial code snippets and tests only one exchange at a time. Real programming assistance, by contrast, usually happens over several turns. The authors present RedCoder, an automated agent that holds multi-turn dialogues with a target model. It typically opens with harmless requests and gradually pushes the model toward code containing CWE-classified weaknesses.

To build it, four LLM agents play a simulated game:

  • an attacker,
  • a defender protected by a multi-turn guardrail,
  • an evaluator that checks for vulnerabilities,
  • a strategy analyst that compares successful and failed attempts.

This process yields two outputs: a set of successful "prototype conversations" and a reusable strategy arsenal. An LLM is fine-tuned on the prototype conversations to become RedCoder's backbone. At deployment, the agent uses retrieval-augmented generation to pull relevant tactics from the arsenal.

RedCoder outperforms established single-turn and multi-turn attack methods on several Code LLMs. It also shows that conventional per-turn guardrails do not stop attacks whose harm builds up across a conversation. This makes RedCoder a scalable, realistic tool for security auditing of code assistants.

Key points

  • The agents in the gaming loop are an attacker, a defender, an evaluator and a strategy analyst. The loop produces prototype conversations, used for supervised fine-tuning, and a strategy arsenal, used via RAG at deployment.
  • Vulnerable code was induced in 61.18% of conversations with CodeGemma-7B and 65.29% with Qwen2.5-Coder-7B.
  • Baselines include the single-turn methods GCG and AutoDAN and the multi-turn methods ActorAttack and CoA. RedCoder outperforms all of them.
  • Standard single-turn guardrails fail against these attacks. Only context-aware multi-turn guardrails trained on the prototype conversations give partial mitigation.
  • The scope covers 43 CWE types. The work was published at ACL 2026 (Long Papers) and supported by the Amazon Nova AI Challenge 2024–2025.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of California, DavisDavis · United States28 works · 41,916 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

🧭 Topics 3Research subjects and languages the work is about.Language models & pretraining · 3,320 worksBias, fairness & safety · 517 worksCode & programming languages · 129 works

Cite

Cite as Wenjie Jacky Mo, Qin Liu, Xiaofei Wen, Dongwon Jung, Hadi Askari, Wenxuan Zhou, Zhe Zhao and Muhao Chen. 2026. RedCoder: Automated Multi-Turn Red Teaming for Code LLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1531

@inproceedings{mo2026redcoder,
  title = {{RedCoder: Automated Multi-Turn Red Teaming for Code LLMs}},
  author = {Wenjie Jacky Mo, Qin Liu, Xiaofei Wen, Dongwon Jung, Hadi Askari, Wenxuan Zhou, Zhe Zhao and Muhao Chen},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1531},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1531}
}

References

54 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hojjat Aghakhani, Wei Dai, Andre Manoel, Xavier Fernandes, Anant Kharkar, Christopher Kruegel, Giovanni Vigna, David Evans, Ben Zorn, and Robert Sim. 2024. Trojanpuzzle: Covertly poisoning code-suggestion models. In 2024 IEEE Symposium on Security and Privacy (SP), pages 1122–1140. IEEE. find
  2. AWS. 2020. Coursera uses codeguru profiler to optimize performance. Available at AWS Blog, accessed 2025-06-27. find
  3. AWS. 2022. Atlassian uses codeguru profiler for production monitoring. Available at AWS Blog, accessed 2025-06-27. find
  4. AWS. 2023. How devfactory builds better applications with codeguru. Available at AWS Blog, accessed 2025-06-27. find
  5. AWS. 2025. Amazon CodeGuru Security—Reduce false-positive detections. https://aws.amazon.com/codeguru. Accessed: 2025-06-27. ↗
  6. Domenico Cotroneo, Cristina Improta, Pietro Liguori, and Roberto Natella. 2024. Vulnerabilities in ai code generators: Exploring targeted data poisoning attacks. In Proceedings of the 32nd IEEE/ACM International Conference on Program Comprehension, pages 280– 292. find
  7. Arghavan Moradi Dakhel, Vahid Majdinasab, Amin Nikanjam, Foutse Khomh, Michel C Desmarais, and Zhen Ming Jack Jiang. 2023. Github copilot ai pair programmer: Asset or liability? Journal of Systems and Software, 203:111734. find
  8. Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. 2024. Multilingual jailbreak challenges in large language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. Open-Review.net.
  9. Suyu Ge, Chunting Zhou, Rui Hou, Madian Khabsa, Yi-Chia Wang, Qifan Wang, Jiawei Han, and Yuning Mao. 2023. Mart: Improving llm safety with multi-round automatic red-teaming. arXiv preprint arXiv:2311.07689.
  10. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  11. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  12. Jingxuan He and Martin Vechev. 2023. Large language models for code: Security hardening and adversarial testing. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pages 1865–1879. find
  13. John Heibel and Daniel Lowd. 2024. Mapping your model: Assessing the impact of adversarial attacks on llm-based programming assistants. arXiv preprint arXiv:2407.11072. — [arXiv:2407.11072](https://arxiv.org/abs/2407.11072) find
  14. Sandy Huang, Nicolas Papernot, Ian Goodfellow, Yan Duan, and Pieter Abbeel. 2017. Adversarial attacks on neural network policies. arXiv preprint arXiv:1702.02284. — [arXiv:1702.02284](https://arxiv.org/abs/1702.02284) find
  15. Yujin Huang, Terry Yue Zhuo, Qiongkai Xu, Han Hu, Xingliang Yuan, and Chunyang Chen. 2023. Training-free lexical backdoor attacks on language models. In Proceedings of the ACM Web Conference 2023, pages 2198–2208. find
  16. Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186. — Qwen2.5-Coder Technical Report
  17. Cristina Improta. 2023. Poisoning programs by un-repairing code: Security concerns of ai-generated code. In 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshops (ISS-REW), pages 128–131. IEEE. find
  18. Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. 2023. Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674.
  19. Arnav Kumar Jain, Gonzalo Gonzalez-Pumariega, Wayne Chen, Alexander M Rush, Wenting Zhao, and Sanjiban Choudhury. 2025. Multi-turn code generation through single-step rewards. In ICLR 2025 Workshop: VerifAI: AI Verification in the Wild. find
  20. Slobodan Jenko, Jingxuan He, Niels Mündler, Mark Vero, and Martin T Vechev. 2024. Practical attacks against black-box code completion engines. CoRR. find
  21. Slobodan Jenko, Niels Mündler, Jingxuan He, Mark Vero, and Martin Vechev. 2025. Black-box adversarial attacks on llm-based code completion. In ICLR 2025 Workshop on Building Trust in Language Models and Applications. find
  22. Akshita Jha and Chandan K Reddy. 2023. Codeattack: Code-based adversarial attacks for pre-trained programming language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 14892–14900. find
  23. Fengqing Jiang, Zhangchen Xu, Luyao Niu, Zhen Xiang, Bhaskar Ramasubramanian, Bo Li, and Radha Poovendran. 2024a. ArtPrompt: ASCII art-based jailbreak attacks against aligned LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pag…
  24. Yifan Jiang, Kriti Aggarwal, Tanmay Laud, Kashif Munir, Jay Pujara, and Subhabrata Mukherjee. 2024b. RED QUEEN: safeguarding large language models against concealed multi-turn jailbreaking. CoRR, abs/2409.17458. — [arXiv:2409.17458](https://arxiv.org/abs/2409.17458) find
  25. Yanzhou Li, Shangqing Liu, Kangjie Chen, Xiaofei Xie, Tianwei Zhang, and Yang Liu. 2023. Multi-target backdoor attacks for code pre-trained models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7236–7254. find
  26. Xiaogeng Liu, Peiran Li, Edward Suh, Yevgeniy Vorobeychik, Zhuoqing Mao, Somesh Jha, Patrick Mc-Daniel, Huan Sun, Bo Li, and Chaowei Xiao. 2025. Autodan-turbo: A lifelong agent for strategy self-exploration to jailbreak llms. ICLR. find
  27. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2024. Autodan: Generating stealthy jailbreak prompts on aligned large language models. ICLR.
  28. Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, et al. 2024. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. In ICML.
  29. Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum S. Anderson, Yaron Singer, and Amin Karbasi. 2024. Tree of attacks: Jailbreaking black-box llms automatically. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2…
  30. Sida Peng, Eirini Kalliamvakou, Peter Cihon, and Mert Demirer. 2023. The impact of ai on developer productivity: Evidence from github copilot. arXiv preprint arXiv:2302.06590. — [arXiv:2302.06590](https://arxiv.org/abs/2302.06590) find
  31. Qibing Ren, Chang Gao, Jing Shao, Junchi Yan, Xin Tan, Wai Lam, and Lizhuang Ma. 2024a. Codeattack: Revealing safety generalization challenges of large language models via code completion. arXiv preprint arXiv:2403.07865. find
  32. Qibing Ren, Hao Li, Dongrui Liu, Zhanxu Xie, Xiaoya Lu, Yu Qiao, Lei Sha, Junchi Yan, Lizhuang Ma, and Jing Shao. 2024b. Derail yourself: Multi-turn llm jailbreak attack through self-discovered clues. arXiv preprint arXiv:2410.10700. — [arXiv:2410.10700](https://arxiv.org/abs/2410.10700) find
  33. Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. 2023. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950.
  34. The MITRE. 2025. Common weakness enumeration. https://cwe.mitre.org/. ↗
  35. Mark Russinovich, Ahmed Salem, and Ronen Eldan. 2024. Great, now write an article about that: The crescendo multi-turn LLM jailbreak attack. CoRR, abs/2404.01833. find
  36. Wenjie Mo, Jiashu Xu, Qin Liu, Jiongxiao Wang, Jun Yan, Chaowei Xiao, and Muhao Chen. 2025. Test-time backdoor mitigation for black-box large language models with defensive demonstrations. Findings of NAACL. find
  37. CodeGemma Team, Heri Zhao, Jeffrey Hui, Joshua Howland, Nam Nguyen, Siqi Zuo, Andrea Hu, Christopher A Choquette-Choo, Jingyue Shen, Joe Kelley, et al. 2024. Codegemma: Open code models based on gemma. arXiv preprint arXiv:2406.11409.
  38. Daye Nam, Andrew Macvean, Vincent Hellendoorn, Bogdan Vasilescu, and Brad Myers. 2024. Using an llm to help with code understanding. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, pages 1–13. find
  39. Thanh-Dat Nguyen, Yang Zhou, Xuan Bach D Le, Patanamon Thongtanunam, and David Lo. 2023. Adversarial attacks on code models with discriminative graph patterns. arXiv preprint arXiv:2308.11161. — [arXiv:2308.11161](https://arxiv.org/abs/2308.11161) find
  40. Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2022. Codegen: An open large language model for code with multi-turn program synthesis. arXiv preprint arXiv:2203.13474. find
  41. Mary Branscombe O’Neill. 2024. Ai-generated code can cause outages — and developers need better tools to prevent them. https://www.techrepublic.com/article/ai-generated-code-outages/. Accessed: 2025-04-29. ↗
  42. OpenAI. 2024. Gpt-4o: Openai’s newest model. https://openai.com/index/gpt-4o. ↗
  43. Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, and Ramesh Karri. 2025. Asleep at the keyboard? assessing the security of github copilot’s code contributions. Communications of the ACM, 68(2):96–105. find
  44. Jianxun Wang and Yixiang Chen. 2023. A review on code generation with llms: Application and evaluation. In 2023 IEEE International Conference on Medical Artificial Intelligence (MedAI), pages 284– 289. IEEE. find
  45. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. Multilingual e5 text embeddings: A technical report. arXiv preprint arXiv:2402.05672.
  46. Xiaofei Wen, Wenxuan Zhou, Wenjie Jacky Mo, and Muhao Chen. 2025. Thinkguard: Deliberative slow thinking leads to cautious guardrails. arXiv preprint arXiv:2502.13458. find
  47. Michel Wermelinger. 2023. Using github copilot to solve simple programming problems. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1, pages 172–178. find
  48. Fangzhou Wu, Xiaogeng Liu, and Chaowei Xiao. 2023. Deceptprompt: Exploiting llm-driven code generation via adversarial natural language instructions. arXiv preprint arXiv:2312.04730. — [arXiv:2312.04730](https://arxiv.org/abs/2312.04730) find
  49. Xilie Xu, Keyi Kong, Ning Liu, Lizhen Cui, Di Wang, Jingfeng Zhang, and Mohan S. Kankanhalli. 2024. An LLM can fool itself: A prompt-based adversarial attack. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. find
  50. Hao Yang, Lizhen Qu, Ehsan Shareghi, and Gholamreza Haffari. 2024a. Jigsaw puzzles: Splitting harmful questions to jailbreak large language models. CoRR, abs/2410.11459. — [arXiv:2410.11459](https://arxiv.org/abs/2410.11459) find
  51. Xikang Yang, Xuehai Tang, Songlin Hu, and Jizhong Han. 2024b. Chain of attack: a semantic-driven contextual multi-turn attacker for llm. arXiv preprint arXiv:2405.05610. — [arXiv:2405.05610](https://arxiv.org/abs/2405.05610) find
  52. Jinchuan Zhang, Yan Zhou, Yaxin Liu, Ziming Li, and Songlin Hu. 2024. Holistic automated red teaming for large language models through top-down test case generation and multi-turn interaction. arXiv preprint arXiv:2409.16783. find
  53. Kunhao Zheng, Juliette Decugis, Jonas Gehring, Taco Cohen, Benjamin Negrevergne, and Gabriel Synnaeve. 2024. What makes large language models reason in (multi-turn) code generation? arXiv preprint arXiv:2410.08105. — [arXiv:2410.08105](https://arxiv.org/abs/2410.08105) find
  54. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mo-2026-redcoder-automated-multi-turn-red/