Elamur Research
☰
home / papers / 2019 / DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation

DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1911.00536

Summary

Neural chatbots for open-domain conversation often give dull, generic replies, lose track of earlier turns, or shift in style or content. DialoGPT addresses these problems by pre-training the GPT-2 transformer on conversations instead of general web text. The authors took Reddit comment threads from 2005–2017 and turned each root-to-leaf reply path into one multi-turn dialogue. They then filtered out URLs, markup, non-English text, offensive material, repetitive replies and overly generic replies. Each conversation is joined into a single sequence and modelled autoregressively, so predicting a turn from its history becomes plain language modelling. To reduce blandness, the system samples several candidate replies and reranks them with a backward model (MMI). This model scores how well a reply predicts its source. DialoGPT was tested on the DSTC-7 benchmark and on a new multi-reference Reddit set. It beat a production seq2seq baseline on both automatic metrics and human judgments. In single-turn settings it came close to human-written replies, and judges sometimes preferred it. The authors released the weights and training pipeline, which makes it a practical base for fine-tuning dialogue systems on smaller datasets.

Key points

  • Training corpus: 147,116,725 Reddit dialogue instances (about 1.8B words). Filters removed URLs, bracket markup, offensive content and blocklisted subreddits, replies longer than 200 words, and highly generic replies built from frequent tri-grams.
  • Model sizes: 117M, 345M and 762M parameters (12, 24 and 36 layers), with a 50,257-entry BPE vocabulary. Training ran on 16 NVIDIA V100 GPUs and used lazy-loading data storage and dynamic batching to go faster.
  • DSTC-7 evaluation: a 2,208-instance test set with 5 references per instance, scored with BLEU, METEOR, NIST, Entropy and Dist-n. The 345M model with beam search had the best automatic scores, ahead of the PersonalityChat baseline.
  • Human evaluation (relevance): judges picked DialoGPT 345M over PersonalityChat 72% to 19%. The MMI-reranked variant was rated slightly above actual human responses (48% vs 43%).
  • Released resources: github.com/microsoft/DialoGPT and Hugging Face (microsoft/DialoGPT-medium). Using policy-gradient RL to optimise the MMI reward collapsed into a degenerate solution where the model simply repeated the input. The authors also warn that the model may produce biased or offensive output.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

600 papers of our library cite it (of the 140673 whose bibliographies we read); 23 of them have a page here, 577 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1911.00536 · arXiv 1911.00536

@misc{zhang2019dialogpt,
  title = {{DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation}},
  author = {Zhang, Yizhe and Sun, Siqi and Galley, Michel and Chen, Yen-Chun and Brockett, Chris and Gao, Xiang and Gao, Jianfeng and Liu, Jingjing and Dolan, Bill},
  year = {2019},
  doi = {10.48550/arxiv.1911.00536},
  eprint = {1911.00536},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1911.00536}
}

References

30 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. M. Burtsev, A. Seliverstov, R. Airapetyan, M. Arkhipov, D. Baymurzina, N. Bushkov, O. Gureenkova, T. Khakhulin, Y. Kuratov, D. Kuznetsov, A. Litinsky, V. Logacheva, A. Lymar, V. Malykh, M. Petrov, V. Polulyakh, L. Pugachev, A. Sorokin, M. Vikhreva, and M. Zaynutdinov. 2018. DeepPavlov: Open-source l… find
  2. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. find
  3. E. Dinan, V. Logacheva, V. Malykh, A. Miller, K. Shuster, J. Urbanek, D. Kiela, A. Szlam, I. Serban, R. Lowe, S. Prabhumoye, A. W. Black, A. Rudnicky, J. Williams, J. Pineau, M. Burtsev, and J. Weston. 2019. The second conversational intelligence challenge (ConvAI2). find our card
  4. George Doddington. 2002. Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In Proceedings of the second international conference on Human Language Technology Research. Morgan Kaufmann Publishers Inc. find
  5. Michel Galley, Chris Brockett, Xiang Gao, Jianfeng Gao, and Bill Dolan. 2019. Grounded response generation task at DSTC7. In AAAI Dialog System Technology Challenges Workshop. find
  6. J. Gao, M. Galley, and L. Li. 2019a. Neural approaches to conversational AI. Foundations and Trends in Information Retrieval. find
  7. Xiang Gao, Sungjin Lee, Yizhe Zhang, Chris Brockett, Michel Galley, Jianfeng Gao, and Bill Dolan. 2019b. Jointly optimizing diversity and relevance in neural response generation. find
  8. Xiang Gao, Yizhe Zhang, Sungjin Lee, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2019c. Structuring latent spaces for stylized response generation. EMNLP-IJCNLP. find
  9. M. Gardner, J. Grus, M. Neumann, O. Tafjord, P. Dasigi, N. F. Liu, M. Peters, M. Schmitz, and L. S. Zettlemoyer. 2018. AllenNLP: A deep semantic natural language processing platform. In Proceedings of Workshop for NLP Open Source Software. find our card
  10. Prakhar Gupta, Shikib Mehri, Tiancheng Zhao, Amy Pavel, Maxine Eskenazi, and Jeffrey P Bigham. 2019. Investigating evaluation of open-domain dialogue systems with human generated multiple references. arXiv preprint arXiv:1907.10568. find
  11. Z. Hu, H. Shi, Z. Yang, B. Tan, T. Zhao, J. He, W. Wang, L. Qin, D. Wang, et al. 2018. Texar: A modularized, versatile, and extensible toolkit for text generation. ACL. find
  12. HuggingFace. 2019. PyTorch transformer repository. https://github.com/huggingface/pytorch-transformers. ↗
  13. Alon Lavie and Abhaya Agarwal. 2007. Meteor: An automatic metric for mt evaluation with high levels of correlation with human judgments. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 228–231. Association for Computational Linguistics. find our card
  14. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016a. A diversity-promoting objective function for neural conversation models. NAACL. find our card
  15. Jiwei Li, Michel Galley, Chris Brockett, Georgios P Spithourakis, Jianfeng Gao, and Bill Dolan. 2016b. A persona-based neural conversation model. ACL. find our card
  16. A. H. Miller, W. Feng, A. Fisch, J. Lu, D. Batra, A. Bordes, D. Parikh, and J. Weston. 2017. ParlAI: A dialog research software platform. In Proceedings of the 2017 EMNLP System Demonstration. find our card
  17. Oluwatobi Olabiyi and Erik T Mueller. 2019. Multi-turn dialogue response generation with autoregressive transformer models. arXiv preprint:1908.01841. — [arXiv:1908.01841](https://arxiv.org/abs/1908.01841) find
  18. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. ACL. find our card
  19. M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer. 2018. Deep contextualized word representations. NAACL. find
  20. Lianhui Qin, Michel Galley, Chris Brockett, Xiaodong Liu, Xiang Gao, Bill Dolan, Yejin Choi, and Jianfeng Gao. 2019. Conversing by reading: Contentful neural conversation with on-demand machine reading. ACL. find
  21. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. 2018. Language models are unsupervised multitask learners. Technical report, OpenAI. find
  22. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint:1910.10683. find
  23. R. Sennrich, B. Haddow, and A. Birch. 2016. Neural machine translation of rare words with subword units. ACL. find
  24. Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2017. A hierarchical latent variable encoder-decoder model for generating dialogues. AAAI. find our card
  25. Vighnesh Leonardo Shiv, Chris Quirk, Anshuman Suri, Xiang Gao, Khuram Shahid, Nithya Govindarajan, Yizhe Zhang, Jianfeng Gao, Michel Galley, Chris Brockett, et al. 2019. Microsoft icecaps: An open-source toolkit for conversation modeling. ACL. find
  26. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. NeurIPS. find
  27. Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning. find
  28. Thomas Wolf, Victor Sanh, Julien Chaumond, and Clement Delangue. 2019. TransferTransfo: A transfer learning approach for neural network based conversational agents. CoRR, abs/1901.08149. — [arXiv:1901.08149](https://arxiv.org/abs/1901.08149) find
  29. Yizhe Zhang, Michel Galley, Jianfeng Gao, Zhe Gan, Xiujun Li, Chris Brockett, and Bill Dolan. 2018. Generating informative and diverse conversational responses via adversarial information maximization. NeurIPS. find
  30. Yizhe Zhang, Xiang Gao, Sungjin Lee, Chris Brockett, Michel Galley, Jianfeng Gao, and Bill Dolan. 2019. Consistent dialogue generation with self-supervised feature learning. arXiv preprint:1903.05759. — [arXiv:1903.05759](https://arxiv.org/abs/1903.05759) find