Elamur Research⌕
☰
home / papers / 2026 / Gradient-Guided Multi-Judge Prompt Optimization

Gradient-Guided Multi-Judge Prompt Optimization

Paper · 2026 · ChenZhuo Zhao, Xinda Wang, Pu Zhao et al.

Summary

GMPO tackles a common trade-off in automatic prompt optimization. Feedback-driven methods give rich signals but are costly. Population search needs many noisy evaluations. Earlier loss-based approaches spend a lot on locating influential prompt parts and tend to overfit to one scoring model. GMPO splits the prompt into fixed-length token windows. It estimates each window's influence with a first-order Taylor approximation: the directional derivative of the loss as the window's embeddings move toward a mask embedding. One forward and one backward pass score all segments at once. A generator, which can be a closed black-box LLM, then rewrites the highest-impact segments. Several lightweight open-source judges compare the candidates using normalized cross-entropy losses, and the lowest aggregate loss wins. Experiments cover GSM8K, AQUA-RAT, Big-Bench Hard, AlpacaEval 2.0 and AdvBench. GMPO improves accuracy over CoT, OPRO and TextGrad, transfers across model families and uses far fewer tokens than most search-based baselines. This makes prompt tuning cheaper and less dependent on a single evaluator.

Key points

  • Cross-family transfer (BBH average accuracy): with gpt-4.1-nano as target, GMPO scores 86.30% (CoT 81.70%). It also reaches 93.03% on GSM8K and 85.04% on AQUA-RAT. With qwen2.5-7b-it, its BBH average is 74.53%, versus 72.85% for OPRO and 71.26% for CoT.
  • Token cost: optimization uses about 0.11M tokens in total. This compares with 1.1M for TextGrad, 4.1M for OPRO and 5.5M for EvoPrompt; only the PW baseline is lower, at 0.084M.
  • AdvBench safety stress test: on Qwen2.5-14B-Instruct with prompt-only attacks, GMPO reaches a 46.92% attack success rate. MSJ reaches 37.18% and GCG 23.85%. The authors withhold actionable jailbreak details for dual-use reasons.
  • Ablations (Math/BBH average 84.22): removing gradient attribution drops it to 82.95, a single judge gives 83.61 and random candidate selection gives 81.33. Dropping z-score normalization costs little (84.01).
  • Code and limitations: code is released at github.com/cyzcz/GMPO. The authors treat the attribution as a heuristic proxy, and running multiple judges adds engineering overhead.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢MicrosoftRedmond · United States378 works · 161,630 citations🎓Peking UniversityBeijing · China315 works · 43,268 citations🎓University of Hong KongPok Fu Lam · China118 works · 13,496 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

Cite

Cite as ChenZhuo Zhao, Xinda Wang, Pu Zhao, Yue Huang, Junting Lu, Ziqian Liu, Qingwei Lin, Saravan Rajmohan and Dongmei Zhang. 2026. Gradient-Guided Multi-Judge Prompt Optimization. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1089

@inproceedings{zhao2026gradient,
  title = {{Gradient-Guided Multi-Judge Prompt Optimization}},
  author = {ChenZhuo Zhao, Xinda Wang, Pu Zhao, Yue Huang, Junting Lu, Ziqian Liu, Qingwei Lin, Saravan Rajmohan and Dongmei Zhang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1089},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1089}
}

References

35 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Eshaan Agarwal, Raghav Magazine, Joykirat Singh, Vivek Dani, Tanuja Ganu, and Akshay Nambi. 2025. Promptwizard: Optimizing prompts via task-aware, feedback-driven self-evolution. In Findings of the Association for Computational Linguistics: ACL 2025, pages 19974–20003. find
  2. Cem Anil, Esin Durmus, Nina Panickssery, Mrinank Sharma, Joe Benton, Sandipan Kundu, Joshua Batson, Meg Tong, Jesse Mu, Daniel Ford, and 1 others. 2024. Many-shot jailbreaking. Advances in Neural Information Processing Systems, 37:129696–129742. find
  3. Bowen Cao, Deng Cai, Zhisong Zhang, Yuexian Zou, and Wai Lam. 2024. On the worst prompt performance of large language models. Advances in Neural Information Processing Systems, 37:69022–69042. find
  4. Anwoy Chatterjee, HSVNS Kowndinya Renduchintala, Sumit Bhatia, and Tanmoy Chakraborty. 2024. Posix: A prompt sensitivity index for large language models. arXiv preprint arXiv:2410.02185. find
  5. Yangyi Chen, Hongcheng Gao, Ganqu Cui, Fanchao Qi, Longtao Huang, Zhiyuan Liu, and Maosong Sun. 2022. Why should adversarial perturbations be imperceptible? rethink the research paradigm in adversarial nlp. arXiv preprint arXiv:2210.10683. find
  6. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.
  7. Sarkar Snigdha Sarathi Das, Ryo Kamoi, Bo Pang, Yusen Zhang, Caiming Xiong, and Rui Zhang. 2024. Greater: Gradients over reasoning makes smaller language models strong prompt optimizers. arXiv preprint arXiv:2412.09722. — [arXiv:2412.09722](https://arxiv.org/abs/2412.09722) find
  8. Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric Xing, and Zhiting Hu. 2022. Rlprompt: Optimizing discrete text prompts with reinforcement learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3369–339…
  9. Yihe Deng, Weitong Zhang, Zixiang Chen, and Quanquan Gu. 2024a. Rephrase and respond: Let large language models ask better questions for themselves. Preprint, arXiv:2311.04205.
  10. Yihe Deng, Weitong Zhang, Zixiang Chen, and Quanquan Gu. 2024b. Rephrase and respond: Let large language models ask better questions for themselves. Preprint, arXiv:2311.04205.
  11. Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B. Hashimoto. 2025. Length-controlled alpacaeval: A simple way to debias automatic evaluators. Preprint, arXiv:2404.04475. find
  12. Yassir Fathullah and Mark Gales. 2025. Efficient prompt optimization for comparative llm-as-a-judge through uncertainty estimation. In First International KDD Workshop on Prompt Optimization, 2025. find
  13. Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, and Tim Rocktäschel. 2023. Promptbreeder: Self-referential self-improvement via prompt evolution. arXiv preprint arXiv:2309.16797.
  14. Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. 2017. Program induction by rationale generation: Learning to solve and explain algebraic word problems. arXiv preprint arXiv:1705.04146. find
  15. OpenAI. 2025. gpt-oss-120b gpt-oss-20b model card. Preprint, arXiv:2508.10925.
  16. Kiran Ramnath, Kang Zhou, Sheng Guan, Soumya Smruti Mishra, Xuan Qi, Zhengyuan Shen, Shuai Wang, Sangmin Woo, Sullam Jeoung, Yawei Wang, Haozhu Wang, Han Ding, Yuzhe Lu, Zhichao Xu, Yun Zhou, Balasubramaniam Srinivasan, Qiaojing Yan, Yueyan Chen, Haibo Ding, and 2 others. 2025a. A systematic survey…
  17. Kiran Ramnath, Kang Zhou, Sheng Guan, Soumya Smruti Mishra, Xuan Qi, Zhengyuan Shen, Shuai Wang, Sangmin Woo, Sullam Jeoung, Yawei Wang, and 1 others. 2025b. A systematic survey of automatic prompt optimization techniques. arXiv preprint arXiv:2502.16923.
  18. Lennart Schneider, Martin Wistuba, Aaron Klein, Jacek Golebiowski, Giovanni Zappella, and Felice Antonio Merra. 2024. Hyperband-based bayesian optimization for black-box prompt selection. arXiv preprint arXiv:2412.07820. — [arXiv:2412.07820](https://arxiv.org/abs/2412.07820) find
  19. Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2023. Quantifying language models’ sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting. arXiv preprint arXiv:2310.11324. find
  20. Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. 2020. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980.
  21. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, and 1 others. 2022. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261.
  22. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, and 1 others. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. find
  23. Qwen Team and 1 others. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2(3). find
  24. Zeliang Tong, Zhuojun Ding, and Wei Wei. 2025. Evoprompt: Evolving prompts for enhanced zero-shot named entity recognition with large language models. In Proceedings of the 31st International Conference on Computational Linguistics, pages 5136–5153. find
  25. Yaxuan Wang, Quan Liu, Zhenting Wang, Zichao Li, Wei Wei, Yang Liu, and Yujia Bao. 2025. Prompt-bridge: Cross-model prompt transfer for large language models. arXiv preprint arXiv:2512.01420. — [arXiv:2512.01420](https://arxiv.org/abs/2512.01420) find
  26. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  27. Zeming Wei, Yifei Wang, Ang Li, Yichuan Mo, and Yisen Wang. 2023. Jailbreak and guard aligned language models with only few in-context demonstrations. arXiv preprint arXiv:2310.06387.
  28. Zhaoxuan Wu, Xiaoqiang Lin, Zhongxiang Dai, Wenyang Hu, Yao Shu, See-Kiong Ng, Patrick Jaillet, and Bryan Kian Hsiang Low. 2024. Prompt optimization with ease? efficient ordering-aware automated selection of exemplars. Advances in Neural Information Processing Systems, 37:122706–122740. find
  29. Jinyu Xiang, Jiayi Zhang, Zhaoyang Yu, Fengwei Teng, Jinhao Tu, Xinbing Liang, Sirui Hong, Chenglin Wu, and Yuyu Luo. 2025. Self-supervised prompt optimization. arXiv preprint arXiv:2502.06855. find
  30. Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. 2023. Large language models as optimizers. arXiv preprint arXiv:2309.03409.
  31. Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou. 2024. Textgrad: Automatic "differentiation" via text. Preprint, arXiv:2406.07496.
  32. Zheng Zhang, Peilin Zhao, Deheng Ye, and Hao Wang. 2025. Enhancing jailbreak attacks on llms via persona prompts. arXiv preprint arXiv:2507.22171. — [arXiv:2507.22171](https://arxiv.org/abs/2507.22171) find
  33. ChenZhuo Zhao, Ziqian Liu, Xinda Wang, Junting Lu, and Chaoyi Ruan. 2025. PMPO: Probabilistic metric prompt optimization for small and large language models. In Findings of the Association for Computational Linguistics: EMNLP 2025. find
  34. Huaixiu Steven Zheng, Swaroop Mishra, Xinyun Chen, Heng-Tze Cheng, Ed H Chi, Quoc V Le, and Denny Zhou. 2023. Take a step back: Evoking reasoning via abstraction in large language models. arXiv preprint arXiv:2310.06117.
  35. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhao-2026-gradient-guided-multi-judge-prompt/