Elamur Research
☰
home / papers / 2022 / Contrastive Decoding: Open-ended Text Generation as Optimization

Contrastive Decoding: Open-ended Text Generation as Optimization

Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/2210.15097

Summary

Open-ended text generation with large language models faces a tradeoff. Picking the most probable continuation tends to yield short, looping text, while sampling often wanders off topic or loses coherence. The authors introduce contrastive decoding (CD), a training-free, search-based method that pairs two frozen models from the same family: a large "expert" (e.g., OPT-13B, GPT2-XL) and a tiny "amateur" (e.g., OPT-125M, GPT2-small). Small models make the typical generation mistakes, such as repetition and drift, even more often than large ones. CD therefore uses beam search to favor tokens where the expert's log-probability most exceeds the amateur's. An adaptive plausibility constraint limits candidates to tokens the expert itself rates reasonably likely, which blocks rare junk tokens from winning the score. In tests on Wikipedia, news and story continuations, CD beat nucleus, top-k, typical decoding and SimCTG on coherence and matched or improved fluency, by both automatic metrics and human judges. The practical appeal is better generations at almost no extra cost, with no fine-tuning required.

Key points

  • The token score is the difference of log-probabilities, log p_EXP − log p_AMA. Only tokens whose expert probability reaches at least α = 0.1 times the top token's probability are allowed (the V_head constraint).
  • The amateur works best when it is the smallest model of the expert's family, and a larger size gap helps. n-gram amateurs are too weak to be useful. Amateur temperature τ in roughly 0.5–1.5 is robust; the paper uses τ = 0.5 for GPT-2 and τ = 1.0 for OPT.
  • In the ablations, dropping V_head causes MAUVE to fall from 0.92 to 0.01, a severe loss of fluency. Sampling from the softmaxed CD score instead of searching gives MAUVE 0.85 versus 0.92.
  • The amateur is conditioned only on the last prompt token. Giving it the full prompt slightly lowered coherence and fluency.
  • Limitation: CD did not help task-oriented generation. For summarization, a small BART amateur scored lower ROUGE than plain beam search. Code: github.com/XiangLi1999/ContrastiveDecoding.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

237 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 230 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2210.15097 · arXiv 2210.15097

@misc{li2022contrastive,
  title = {{Contrastive Decoding: Open-ended Text Generation as Optimization}},
  author = {Li, Xiang Lisa and Holtzman, Ari and Fried, Daniel and Liang, Percy and Eisner, Jason and Hashimoto, Tatsunori and Zettlemoyer, Luke and Lewis, Mike},
  year = {2022},
  doi = {10.48550/arxiv.2210.15097},
  eprint = {2210.15097},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2210.15097}
}

References

29 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chen An, Jiangtao Feng, Kai Lv, Lingpeng Kong, Xipeng Qiu, and Xuanjing Huang. 2022. Cont: Contrastive neural text generation. ArXiv, abs/2205.14690. — [arXiv:2205.14690](https://arxiv.org/abs/2205.14690) find
  2. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… find
  3. Alexandra DeLucia, Aaron Mueller, Xiang Lisa Li, and João Sedoc. 2020. Decoding methods for neural narrative generation. CoRR, abs/2010.07375. find
  4. Bryan Eikema and Wilker Aziz. 2020. Is map decoding all you need? the inadequacy of the mode in neural machine translation. In COLING, pages 4506–4520. find
  5. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889–898, Melbourne, Australia. Association for Computational Linguistics. find
  6. Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE: Simple contrastive learning of sentence embeddings. In Empirical Methods in Natural Language Processing (EMNLP). find our card
  7. H. Paul Grice. 1975. Logic and conversation. In Peter Cole and Jerry L. Morgan, editors, Speech Acts, volume 3 of Syntax and Semantics. find
  8. He He, Nanyun Peng, and Percy Liang. 2019. Pun generation with surprise. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1734–1744, Minneapolis, Minnesota. Assoc… find
  9. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. In International Conference on Learning Representations. find
  10. Laurence Horn. 1984. Toward a new taxonomy for pragmatic inference: Q-based and r-based implicature. Meaning, form, and use in context: Linguistic applications, 11:42. find
  11. Alex M Lamb, Anirudh Goyal ALIAS PARTH GOYAL, Ying Zhang, Saizheng Zhang, Aaron C Courville, and Yoshua Bengio. 2016. Professor forcing: A new algorithm for training recurrent networks. In Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc. find
  12. Stephen C Levinson. 2000. Presumptive Meanings: The Theory of Generalized Conversational Implicature. MIT Press. find
  13. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A diversity-promoting objective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pag… find our card
  14. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4582–… find
  15. Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. 2021. DExperts: Decoding-time controlled text generation with experts and anti-experts. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th I… find our card
  16. Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906–1919, Online. Association for Computational Linguistics. find
  17. Clara Meister, Tiago Pimentel, Gian Wiher, and Ryan Cotterell. 2022. Typical decoding for natural language generation. CoRR, abs/2202.00666. — [arXiv:2202.00666](https://arxiv.org/abs/2202.00666) find
  18. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2017. Pointer sentinel mixture models. In International Conference on Learning Representations. find our card
  19. Romain Paulus, Caiming Xiong, and Richard Socher. 2018. A deep reinforced model for abstractive summarization. In International Conference on Learning Representations. find our card
  20. Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. 2021. MAUVE: Measuring the gap between neural text and human text using divergence frontiers. In Advances in Neural Information Processing Systems. find our card
  21. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. https://openai.com/blog/better-language-models/. ↗
  22. Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2016. Sequence level training with recurrent neural networks. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings. find our card
  23. Yixuan Su and Nigel Collier. 2022. Contrastive search is what you need for neural text generation. arXiv preprint arXiv:2210.14140. — [arXiv:2210.14140](https://arxiv.org/abs/2210.14140) find
  24. Yixuan Su, Tian Lan, Yan Wang, Dani Yogatama, Lingpeng Kong, and Nigel Collier. 2022. A contrastive framework for neural text generation. Neurips, abs/2202.06417. — [arXiv:2202.06417](https://arxiv.org/abs/2202.06417) find
  25. Arun Venkatraman, Martial Hebert, and J.. Bagnell. 2015. Improving multi-step prediction of learned time series models. Proceedings of the AAAI Conference on Artificial Intelligence, 29(1). find
  26. Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. 2020. Neural text generation with unlikelihood training. In International Conference on Learning Representations. find our card
  27. Sam Wiseman and Alexander M. Rush. 2016. Sequence-to-sequence learning as beam-search optimization. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1296–1306, Austin, Texas. Association for Computational Linguistics. find
  28. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144. find
  29. Yukun Zhu, Ryan Kiros, Richard Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In arXiv preprint arXiv:1506.06724. find our card