Elamur Research⌕
☰
home / papers / 2026 / LLM Prompt Duel Optimizer: Efficient Label-Free Prompt Optimization

LLM Prompt Duel Optimizer: Efficient Label-Free Prompt Optimization

Paper · 2026 · Yuanchen Wu, Saurabh Verma, Justin Lee et al.

Summary

Automatic prompt optimization usually scores candidate instructions against labeled validation data. Such data is often missing early in a deployment. The authors, from Penn State and Meta, propose the Prompt Duel Optimizer (PDO), which needs no labels. An LLM judge sees the outputs of two prompts on the same unlabeled input and picks the better one. Prompt search becomes a dueling-bandit problem, where cost is the number of judge calls. The target is the Condorcet winner or, if none exists, the Copeland winner. PDO combines Double Thompson Sampling, which spends the budget on pairs whose outcome is still uncertain, with periodic mutation of the current leader. This grows the candidate pool while weak prompts are dropped. On 16 BIG-bench Hard tasks and four MS MARCO question categories, PDO found stronger prompts than label-free baselines under fixed comparison budgets. Its advantage held when judges from other model families re-checked the results. This suggests that pairwise LLM preferences can replace labels when bootstrapping prompts.

Key points

  • Two components. PDO pairs Double Thompson Sampling (D-TS), which has known O(K² log T) Copeland regret within a fixed pool, with top-performer guided mutation. Mutations use template edits, text-gradient changes or LLM rewrites.
  • Setup. Llama-3.3-70B-Instruct generated prompts, judged preferences and did the final evaluation. Data was split 50/50 into dev and test sets, and results are averaged over 10 runs. Metrics were accuracy for BBH and a 1–5 judge score against gold answers for MS MARCO.
  • Judge design. For multiple-choice tasks, a dual judge prefers the prompt with the correct answer and falls back to reasoning quality when both answers match. For open QA, it rates accuracy, completeness, relevance and clarity.
  • Bandit ablation. Posterior-sampling methods (D-TS, Self-Sparring) beat RMED, which in turn beat the UCB-style RUCB and MergeRUCB; random selection was worst. Self-Sparring led on the ENTITY task.
  • Limitations and code. PDO optimizes the judge's taste rather than the true task metric, so it can favor stylistic patterns. Code is in the meta-llama/prompt-ops repository on GitHub.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Pennsylvania State UniversityState College · United States67 works · 5,865 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksDecision Sciences · 249 worksManagement Science and Operations Research · 134 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Yuanchen Wu, Saurabh Verma, Justin Lee, Fangzhou Xiong, Poppy Zhang, Amel Awadelkarim, Xu Chen, Yubai Yuan and Shawndra Hill. 2026. LLM Prompt Duel Optimizer: Efficient Label-Free Prompt Optimization. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.490

@inproceedings{wu2026llm,
  title = {{LLM Prompt Duel Optimizer: Efficient Label-Free Prompt Optimization}},
  author = {Yuanchen Wu, Saurabh Verma, Justin Lee, Fangzhou Xiong, Poppy Zhang, Amel Awadelkarim, Xu Chen, Yubai Yuan and Shawndra Hill},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.490},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.490}
}

References

37 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shipra Agrawal and Navin Goyal. 2012. Analysis of thompson sampling for the multi-armed bandit problem. In Proceedings of the 25th Annual Conference on Learning Theory, volume 23 of Proceedings of Machine Learning Research, pages 39.1–39.26, Edinburgh, Scotland. PMLR. find
  2. Rin Ashizawa, Yoichi Hirose, Nozomu Yoshinari, Kento Uchida, and Shinichi Shirakawa. 2025. Bandit-based prompt design strategy selection improves prompt optimizers. In Findings of the Association for Computational Linguistics: ACL 2025, pages 20799–20817, Vienna, Austria. Association for Computation… find
  3. Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2016. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint a…
  4. Viktor Bengs, Róbert Busa-Fekete, Adil El Mesaoudi-Paul, and Eyke Hüllermeier. 2021. Preference-based online learning with dueling bandits: A survey. Journal of Machine Learning Research, 22(7):1–108. Open access. find
  5. Yongchao Chen, Jacob Arkin, Yilun Hao, Yang Zhang, Nicholas Roy, and Chuchu Fan. 2024. PRompt optimization in multi-step tasks (PROMST): Integrating human feedback and heuristic-based sampling. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 3859–3920… find
  6. Jiale Cheng, Xiao Liu, Kehan Zheng, Pei Ke, Hongning Wang, Yuxiao Dong, Jie Tang, and Minlie Huang. 2023. Black-box prompt optimization: Aligning large language models without model training. ArXiv, abs/2311.04155.
  7. Yihong Dong, Kangcheng Luo, Xue Jiang, Zhi Jin, and Ge Li. 2022. Pace: Improving prompt with actor-critic editing for large language model. In arXiv preprint arXiv:2212.XXXX. find
  8. Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, and Tim Rocktäschel. 2023. Promptbreeder: Self-referential self-improvement via prompt evolution. arXiv preprint arXiv:2309.16797.
  9. Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel M. Ni, and Jian Guo. 2024. A survey on llm-as-a-judge. arXiv preprint arXiv:2411.15594. find
  10. Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang. 2023. Evoprompt: Connecting llms with evolutionary algorithms yields powerful prompt optimizers. arXiv preprint arXiv:2309.08532.
  11. Horace He and Thinking Machines Lab. 2025. Defeating nondeterminism in llm inference. Thinking Machines Lab: Connectionism. Https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/. find
  12. Robert Kleinberg, Aleksandrs Slivkins, and Eli Upfal. 2008. Multi-armed bandits in metric spaces. In Proceedings of the 40th Annual ACM Symposium on Theory of Computing (STOC), pages 681–690. ACM. find
  13. Junpei Komiyama, Junya Honda, Hisashi Kashima, and Hiroshi Nakagawa. 2015. Regret lower bound and optimal algorithm in dueling bandit problem. In Proceedings of The 28th Conference on Learning Theory (COLT), pages 1141–1154. find
  14. Xiaoqiang Lin, Zhongxiang Dai, Arun Verma, See-Kiong Ng, Patrick Jaillet, and Bryan Kian Hsiang Low. 2025. Prompt optimization with human feedback. find
  15. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. G-eval: NLG evaluation using gpt-4 with better human alignment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511–2522, Singapore. Association for Computational Lin…
  16. Yinhong Liu, Han Zhou, Zhijiang Guo, Ehsan Shareghi, Ivan Vulić, Anna Korhonen, and Nigel Collier. 2024. Aligning with human judgement: The role of pairwise preference in large language model evaluators. In Proceedings of the First Conference on Language Modeling (COLM 2024), page –.
  17. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Sean Welleck, Bodhisattwa Prasad Majumder, Shashank Gupta, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-refine: Iterative refinement with self-feedback. ArX… find
  18. Krista Opsahl-Ong, Michael J. Ryan, Josh Purtell, David Broman, Christopher Potts, Matei Zaharia, and Omar Khattab. 2024. Optimizing instructions and demonstrations for multi-stage language model programs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pag…
  19. Konstantina Palla, José Luis Redondo García, Claudia Hauff, Francesco Fabbri, Henrik Lindström, Daniel R. Taber, Andreas Damianou, and Mounia Lalmas. 2025. Policy-as-prompt: Rethinking content moderation in the age of large language models. In Proceedings of the ACM Conference on Fairness, Accountab… find
  20. Reid Pryzant, Dan Iter, Jerry Li, Yin Lee, Chenguang Zhu, and Michael Zeng. 2023. Automatic prompt optimization with “gradient descent” and beam search. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7957–7968. Association for Computational Linguisti…
  21. Kiran Ramnath, Kang Zhou, Sheng Guan, Soumya Smruti Mishra, Xuan Qi, Zhengyuan Shen, Shuai Wang, Sangmin Woo, Sullam Jeoung, Yawei Wang, Haozhu Wang, Han Ding, Yuzhe Lu, Zhichao Xu, Yun Zhou, Balasubramaniam Srinivasan, Qiaojing Yan, Yueyan Chen, Haibo Ding, and 2 others. 2025. A systematic survey o…
  22. Alexander Ratner, Stephen H. Bach, Henry Ehrenberg, Jason Fries, Sen Wu, and Christopher Ré. 2017. Snorkel: Rapid training data creation with weak supervision. Proceedings of the VLDB Endowment, 11(3):269–282. find
  23. Samuel Schulhoff and 1 others. 2024. A systematic survey of prompting techniques. arXiv preprint arXiv:2406.06608.
  24. Chengshuai Shi, Kun Yang, Jing Yang, and Cong Shen. 2024. Efficient prompt optimization through the lens of best arm identification. In Neural Information Processing Systems. find
  25. Yanan Sui, Vincent Zhuang, Joel W. Burdick, and Yisong Yue. 2017. Multi-dueling bandits with dependent arms. In Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence (UAI). find
  26. Jiuding Sun, Chantal Shaib, and Byron C. Wallace. 2023. Evaluating the zero-shot robustness of instruction-tuned language models. arXiv preprint arXiv:2306.11270. — [arXiv:2306.11270](https://arxiv.org/abs/2306.11270) find
  27. Huasen Wu and Xin Liu. 2016. Double thompson sampling for dueling bandits. Advances in neural information processing systems, 29. find
  28. Jinyu Xiang, Jiayi Zhang, Zhaoyang Yu, Xinbing Liang, Fengwei Teng, Jinhao Tu, Fashen Ren, Xiangru Tang, Sirui Hong, Chenglin Wu, and Yuyu Luo. 2025. Self-supervised prompt optimization. arXiv preprint arXiv:2502.06855. Version v3, revised 21 August 2025. find
  29. Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. 2024. Large language models as optimizers. In ICLR.
  30. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. arXiv preprint arXiv:2306.05685.
  31. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2022. Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910.
  32. Masrour Zoghi, Shimon Whiteson, and Maarten de Rijke. 2015. Mergerucb: A method for large-scale online ranker evaluation. In Proceedings of the Eighth ACM International Conference on Web Search and Data Mining (WSDM), pages 17–26. ACM. find
  33. Masrour Zoghi, Shimon Whiteson, Rémi Munos, and Maarten de Rijke. 2013. Relative upper confidence bound for the k-armed dueling bandit problem. arXiv preprint arXiv:1312.3393. Version 2, submitted December 2013. — [arXiv:1312.3393](https://arxiv.org/abs/1312.3393) find
  34. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, , and Jason Wei. 2022. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261.
  35. Patrick Wagner. 2024. Using llms for text classification. Medium. find
  36. Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee, and Ee-Peng Lim. 2023. Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Lon… find
  37. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2026-llm-prompt-duel-optimizer-efficient/