Elamur Research⌕
☰
home / papers / 2025 / Decoding Decoded: Understanding Hyperparameter Effects in Open-Ended Text Generation

Decoding Decoded: Understanding Hyperparameter Effects in Open-Ended Text Generation

Paper · 2025 · Esteban Garces Arias, Meimingwei Li, Christian Heumann et al.

Summary

LLMs output token probabilities, and the decoding step that turns them into text is often left at default settings even though it strongly shapes output quality. This study is a large-scale sensitivity analysis of how hyperparameters in deterministic (greedy, beam search), stochastic (temperature, top-k, nucleus sampling), and contrastive (contrastive search, adaptive contrastive search) decoding affect open-ended generation. Seven open-source models continued prompts from three datasets covering factual and creative domains (e.g., Wikinews). The authors scored outputs with diversity, MAUVE, and coherence (computed with OPT 2.7B). They also introduced QText, the harmonic mean of these three metrics, and compared the automatic scores with human ratings. The main finding is that the decoding method and its settings can affect quality about as much as model size, or more. Configurations that favor either coherence or diversity at the expense of the other perform worse overall. Contrastive search with moderate α and k, and the hyperparameter-free adaptive variant, produce the most balanced results. Sampling methods can do well but are more sensitive to tuning. These results give practitioners concrete guidance for setting decoding hyperparameters.

Key points

  • Models: GPT2-XL, Mistral 7B v0.1/v0.3, Llama 3.1 8B, Deepseek 7B, Qwen2 7B, and Falcon 2 11B.
  • Scale: 60 hyperparameter combinations plus adaptive contrastive search were tested, totalling 1242 experiments and about 2.2 million generated continuations. The continuations and code are released at github.com/YecanLee/Decoding-Decoded.
  • Best scores: Among all configurations, adaptive contrastive search achieved the top scores on both diversity (93.95%) and QText (87.37%). However, its outputs were more coherent than human-written text.
  • Human vs. automatic evaluation: Human judgments correlate only moderately with the automatic metrics, which sometimes reward diversity over coherence.
  • Limitations: The study covers English only and does not include SFT models. The metrics may inherit bias from the models used to compute them (OPT for coherence, GPT-2 embeddings for MAUVE).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Ludwig-Maximilians-Universität MünchenMunich · Germany107 works · 56,478 citations🎓Munich Center for Machine LearningMunich · Germany57 works · 1,094 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Text generation · 202 works

Cite

Cite as Esteban Garces Arias, Meimingwei Li, Christian Heumann and Matthias Assenmacher. 2025. Decoding Decoded: Understanding Hyperparameter Effects in Open-Ended Text Generation. Proceedings of the 31st International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{arias2025decoding,
  title = {{Decoding Decoded: Understanding Hyperparameter Effects in Open-Ended Text Generation}},
  author = {Esteban Garces Arias, Meimingwei Li, Christian Heumann and Matthias Assenmacher},
  booktitle = {Proceedings of the 31st International Conference on Computational Linguistics},
  year = {2025},
  url = {https://aclanthology.org/2025.coling-main.668/}
}

References

26 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David H Ackley, Geoffrey E Hinton, and Terrence J Sejnowski. 1985. A learning algorithm for boltzmann machines. Cognitive science, 9(1):147–169. find
  2. Afra Amini, Ryan Cotterell, John Hewitt, Luca Malagutti, Clara Meister, and Tiago Pimentel. 2023. Generating text from language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 6: Tutorial Abstracts), pages 27–31, Toronto, Canada. Association… find
  3. DeepSeek-AI, :, Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, Huazuo Gao, Kaige Gao, Wenjun Gao, Ruiqi Ge, Kang Guan, Daya Guo, Jianzhong Guo, Guangbo Hao, Zhewen Hao, Ying He, Wenjie Hu, Panpan Huang, Erhang Li, Guowei Li, Jia… find
  4. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez,… — The Llama 3 Herd of Models
  5. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. Preprint, arXiv:1805.04833.
  6. Markus Freitag and Yaser Al-Onaizan. 2017. Beam search strategies for neural machine translation. In Proceedings of the First Workshop on Neural Machine Translation. Association for Computational Linguistics.
  7. Esteban Garces-Arias, Julian Rodemann, Meimingwei Li, Christian Heumann, and Matthias Aßenmacher. 2024. Adaptive contrastive search: Uncertainty-guided decoding for open-ended text generation. Preprint, arXiv:2407.18698. find
  8. Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, I… — Llama 2: Open Foundation and Fine-Tuned Chat Models
  9. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751.
  10. Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. 2023. Contrastive decoding: Open-ended text generation as optimization. Preprint, arXiv:2210.15097.
  11. Quentin Malartic, Nilabhra Roy Chowdhury, Ruxandra Cojocaru, Mugariya Farooq, Giulia Campesan, Yasser Abdelaziz Dahou Djilali, Sanath Narayan, Ankit Singh, Maksim Velikanov, Basma El Amel Boussaha, Mohammed Al-Yafeai, Hamza Alobeidli, Leen Al Qadi, Mohamed El Amine Seddik, Kirill Fedyanin, Reda Alam… — [arXiv:2407.14885](https://arxiv.org/abs/2407.14885) find
  12. Clara Meister, Tiago Pimentel, Gian Wiher, and Ryan Cotterell. 2023. Locally typical sampling. Preprint, arXiv:2202.00666.
  13. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. Pointer sentinel mixture models. Preprint, arXiv:1609.07843.
  14. Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. 2021. Mauve: Measuring the gap between neural text and human text using divergence frontiers. Advances in Neural Information Processing Systems, 34:4816–4828.
  15. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. find
  16. Chufan Shi, Haoran Yang, Deng Cai, Zhisong Zhang, Yifan Wang, Yujiu Yang, and Wai Lam. 2024. A thorough examination of decoding methods in the era of llms. Preprint, arXiv:2402.06925.
  17. Yixuan Su and Nigel Collier. 2023. Contrastive search is what you need for neural text generation. Preprint, arXiv:2210.14140. — [arXiv:2210.14140](https://arxiv.org/abs/2210.14140) find
  18. Yixuan Su, Tian Lan, Yan Wang, Dani Yogatama, Lingpeng Kong, and Nigel Collier. 2022. A contrastive framework for neural text generation. Preprint, arXiv:2202.06417. — [arXiv:2202.06417](https://arxiv.org/abs/2202.06417) find
  19. Yixuan Su and Jialu Xu. 2022. An empirical study on contrastive search and contrastive decoding for open-ended text generation. Preprint, arXiv:2211.10797. — [arXiv:2211.10797](https://arxiv.org/abs/2211.10797) find
  20. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Gian Wiher, Clara Meister, and Ryan Cotterell. 2022. On Decoding Strategies for Neural Text Generators. Transactions of the Association for Computational…
  21. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Juny… find
  22. Haoran Yang, Deng Cai, Huayang Li, Wei Bi, Wai Lam, and Shuming Shi. 2024b. A frustratingly simple decoding method for neural text generation. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 536– 557,… find
  23. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022. Opt: Open pre-trai…
  24. Yuxuan Zhou, Margret Keuper, and Mario Fritz. 2024. Balancing diversity and risk in llm sampling: How to select your method and parameter for open-ended text generation. Preprint, arXiv:2408.13586.
  25. Wenhong Zhu, Hongkun Hao, Zhiwei He, Yiming Ai, and Rui Wang. 2024. Improving open-ended text generation via adaptive decoding. Preprint, arXiv:2402.18223. — [arXiv:2402.18223](https://arxiv.org/abs/2402.18223) find
  26. Yukun Zhu, Ryan Kiros, Richard Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. Preprint, arXiv:1506.06724.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/arias-2025-decoding-decoded-understanding-hyperparameter-effects/