Elamur Research⌕
☰
home / papers / 2026 / Think Earlier, Not Longer: Prompt Optimization via Reducing Unhealthy Exploration

Think Earlier, Not Longer: Prompt Optimization via Reducing Unhealthy Exploration

Paper · 2026 · Ling-I Wu, Minyu Chen, Jingyang Li et al.

Summary

Reasoning-oriented LLMs often spend many tokens on exploration. The authors argue that part of this exploration is "unhealthy": it comes from unclear prompts and lengthens outputs without improving answers. To test this, they took 1,000 MMLU-Pro questions and replaced technical terms with initialisms to make them more ambiguous, keeping the underlying task the same. They evaluated GPT-oss-120B and Qwen3-30B models on these questions. Greater ambiguity increased token counts and slightly lowered accuracy and self-consistency. Token-level entropy curves showed a higher and later early-stage peak, the phase where the model works out what is being asked. Near termination, the curves converged across ambiguity levels. Based on this, the authors trained a small prompt optimizer with multi-turn reinforcement learning to write short clarifications that are added to the prompt. Its reward is based on entropy peaks: it favors a lower, earlier early peak while leaving the late peak intact. The method raised reasoning efficiency by up to 52.9% in-domain and 38.54% out-of-domain without hurting accuracy. This suggests that clarifying the task can save compute better than encouraging longer thinking.

Key points

  • Controlled ambiguity setup: GPT-5 tagged technical terms in 1,000 MMLU-Pro items (e.g., page fault → PF). Mask ratios were 0/25/50/75/100%, with 16 rollouts per question at temperature 0.7. GPT-oss-120B was tested at low, medium and high reasoning effort.
  • Going from 0% to 100% masking had different costs depending on effort level:
    • At medium effort, the early entropy peak rose by 0.027 and outputs grew by 335 tokens.
    • At high effort, the peak gap fell to 0.006, but outputs grew by 437 tokens. Extra effort absorbs the ambiguity mostly by spending more tokens.
  • Late-stage peak differences shrank in a similar pattern:
    • At medium effort, the gap was 0.019, about 70% of the early-stage gap.
    • At high effort, it was 5.1 × 10⁻⁵, roughly 0.8% of the early-stage gap.
  • The authors compared their method with direct RLVR using a length-penalized reward on Qwen3-30B-Instruct:
    • RLVR traded accuracy against compression. At λ=0.75 it reached 69.80 accuracy on MMLU-Pro, versus 76.50 for the unoptimized baseline.
    • The proposed method reached 80.10 accuracy on MMLU-Pro. On the out-of-domain MedQA benchmark it had the best efficiency score (141.64).
  • Stated limitations:
    • Testing covered a limited set of architectures and datasets.
    • Entropy is only an indirect proxy for exploration.
    • Short clarifications may fall short when a prompt is missing too much information or is fundamentally badly constructed.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Shanghai Jiao Tong UniversityShanghai · China212 works · 19,544 citations🎓Shanghai Polytechnic UniversityShanghai · China2 works · 148 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

Cite

Cite as Ling-I Wu, Minyu Chen, Jingyang Li, Xi Chang and Guoqiang Li. 2026. Think Earlier, Not Longer: Prompt Optimization via Reducing Unhealthy Exploration. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.817

@inproceedings{wu2026think,
  title = {{Think Earlier, Not Longer: Prompt Optimization via Reducing Unhealthy Exploration}},
  author = {Ling-I Wu, Minyu Chen, Jingyang Li, Xi Chang and Guoqiang Li},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.817},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.817}
}

References

31 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, and 1 others. 2025a. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925.
  2. Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, and Hao Peng. 2025b. The unreasonable effectiveness of entropy minimization in llm reasoning. arXiv preprint arXiv:2505.15134. find
  3. Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, and 1 others. 2024. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI conference on a… find
  4. Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric Xing, and Zhiting Hu. 2022. Rlprompt: Optimizing discrete text prompts with reinforcement learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3369–339…
  5. Xinrun Du, Yifan Yao, Kaijing Ma, Bingli Wang, Tianyu Zheng, King Zhu, Minghao Liu, Yiming Liang, Xiaolong Jin, Zhenlin Wei, and 1 others. 2025. find
  6. Supergpqa: Scaling llm evaluation across 285 graduate disciplines. arXiv preprint arXiv:2502.14739.
  7. Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. 2024. Detecting hallucinations in large language models using semantic entropy. Nature, 630(8017):625–630. find
  8. Jiatong Han, Jannik Kossen, Muhammed Razzak, Lisa Schut, Shreshth A Malik, and Yarin Gal. 2024. Semantic entropy probes: Robust and cheap hallucination detection in llms. In ICML 2024 Workshop on Foundation Models in the Wild. find
  9. Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2021. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):6421.
  10. Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. arXiv preprint arXiv:2302.09664.
  11. Xiaoqiang Lin, Zhongxiang Dai, Arun Verma, See-Kiong Ng, Patrick Jaillet, and Bryan Kian Hsiang Low. 2024. Prompt optimization with human feedback. arXiv preprint arXiv:2405.17346. — [arXiv:2405.17346](https://arxiv.org/abs/2405.17346) find
  12. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori B Hashimoto. 2025. s1: Simple test-time scaling. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages…
  13. Dang Nguyen, Ali Payani, and Baharan Mirzasoleiman. 2025. Beyond semantic entropy: Boosting llm uncertainty quantification with pairwise semantic similarity. arXiv preprint arXiv:2506.00245. find
  14. Alexander Nikitin, Jannik Kossen, Yarin Gal, and Pekka Marttinen. 2024. Kernel language entropy: Fine-grained uncertainty quantification for llms from semantic similarities. Advances in Neural Information Processing Systems, 37:8901–8929. find
  15. Timo Schick and Hinrich Schütze. 2020. Few-shot text generation with pattern-exploiting training. arXiv preprint arXiv:2012.11926. — [arXiv:2012.11926](https://arxiv.org/abs/2012.11926) find
  16. Aman Sharma and Paras Chopra. 2025. Think just enough: Sequence-level entropy as a confidence signal for llm reasoning. arXiv preprint arXiv:2510.08146. find
  17. Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. 2020. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980.
  18. Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2025. Scaling llm test-time compute optimally can be more effective than scaling parameters for reasoning. In The Thirteenth International Conference on Learning Representations. find
  19. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2022. Large language models are human-level prompt engineers. In The eleventh international conference on learning representations.
  20. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and 1 others. 2023. Challenging big-bench tasks and whether chain-of-thought can solve them. In Findings of the Association for Computational Linguistics: AC…
  21. A Zeliang Tong, Zhuojun Ding, and Wei Wei. 2025. Evoprompt: Evolving prompts for enhanced zero-shot named entity recognition with large language models. In Proceedings of the 31st International Conference on Computational Linguistics, pages 5136–5153. find
  22. Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. Universal adversarial triggers for attacking and analyzing nlp. arXiv preprint arXiv:1908.07125. find
  23. Chen Wang, Zhaochun Li, Jionghao Bai, Yuzhi Zhang, Shisheng Cui, Zhou Zhao, and Yue Wang. 2025. Arbitrary entropy policy optimization: Entropy is controllable in reinforcement fine-tuning. arXiv preprint arXiv:2510.08141. — [arXiv:2510.08141](https://arxiv.org/abs/2510.08141) find
  24. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171.
  25. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, and 1 others. 2024. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. Advances in Neural Information Processing Systems, 37:95266–95290…
  26. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  27. Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. 2023. Large language models as optimizers. In The Twelfth International Conference on Learning Representations.
  28. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809–11822. find
  29. Jinghan Zhang, Xiting Wang, Fengran Mo, Yeyang Zhou, Wanfu Gao, and Kunpeng Liu. 2025. Entropy-based exploration conduction for multi-step reasoning. arXiv preprint arXiv:2503.15848. find
  30. Baseline Early Stop (Sharma and Chopra, 2025) is an inference-time efficiency method that monitors sequence-level entropy during generation and terminates reasoning once the entropy remains below a predefined threshold for a fixed number of steps. By truncating later-stage generation, it aims to red… find
  31. Evoprompt (Tong et al., 2025) is a prompt optimization framework that applies evolutionary algorithms to iteratively refine prompts based on task performance. It focuses on improving answer accuracy by exploring prompt variants through mutation and selection, and requires a development set to evalua… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2026-think-earlier-not-longer-prompt/