Elamur Research⌕
☰
home / papers / 2017 / Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders

Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders

Summary

Neural encoder-decoder dialog models trained with MLE tend to produce bland, generic replies such as "I don't know." Earlier fixes mostly worked at the word level, for example through decoding tricks. The authors instead frame response generation at the level of discourse, where a single context can legitimately lead to many different replies.

They adapt conditional variational autoencoders (CVAE) to dialog. A latent Gaussian variable, conditioned on the dialog context, represents the space of plausible conversational intents. Sampling it lets even a greedy decoder produce varied answers. They also propose Knowledge-Guided CVAE (kgCVAE), which adds linguistic features such as dialog acts to the latent structure, and a bag-of-word (BOW) auxiliary loss that keeps the decoder from ignoring the latent variable.

On human-human conversation data, both models give noticeably more diverse yet appropriate responses than the baseline. In a Penn Treebank language-modelling test, the BOW loss clearly beats KL annealing alone. The work matters because it offers a practical recipe for training latent-variable text generators and links neural dialog models with classic dialog-manager ideas.

Key points

  • Architecture:
    • Each utterance is encoded by a bidirectional GRU.
    • A 1-layer GRU context encoder takes the preceding utterances plus speaker-floor flags and meta features such as topic.
    • Separate recognition and prior networks output diagonal Gaussians over z.
    • A 1-layer GRU decoder generates the response, using a context window of 10 utterances.
  • kgCVAE: dialog acts serve as the knowledge feature. The model predicts a dialog act alongside each response, which improves both interpretability and performance.
  • Training setups compared: a plain VAE, KL annealing (KLA, with the weight rising linearly from 0 to 1 over the first 5000 batches), the BOW loss, and BOW+KLA.
  • Penn Treebank VAE results:
SetupPerplexityKL
Standard122.00.05
KLA111.52.02
BOW97.727.41
BOW+KLA73.0415.94
  • Qualitative behaviour: for open questions, kgCVAE outputs span several dialog acts, while baseline samples only vary locally and share prefixes. For back-channel contexts it varies wording within the single fitting act.

Interesting ideas

  • I1. BOW auxiliary loss: the latent code must also predict the response's words regardless of order. This prevents an autoregressive decoder from bypassing z, which could be reused in any VAE-style text model suffering from posterior collapse.
  • I2. Knowledge-guided latents: cheap linguistic annotations (dialog acts, and potentially sentiment or entities) are injected as an extra conditioned variable. This steers the latent space toward interpretable, controllable factors and may reduce the amount of data needed.
  • I3. Monitoring the KL term: tracking KL over training shows whether a latent model actually uses its code. Watching for the KL dropping to zero once annealing ends is a simple diagnostic for other generative setups.
  • I4. Diversity from the latent, not the decoder: sampling diversity from the latent variable while decoding greedily separates "what to say" from "how to say it." This could be useful for data augmentation or for generating varied candidate texts.
  • I5. Behaviour across context types: checking whether the model shows lexical variety in constrained contexts but intent-level variety in open ones is a useful evaluation lens for any diverse-generation system.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Discourse analysis · Speech act theory

Cited by

261 papers of our library cite it (of the 140884 whose bibliographies we read); 7 of them have a page here, 254 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p17-1061

@inproceedings{zhao2017learning,
  title = {{Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders}},
  author = {Zhao, Tiancheng and Zhao, Ran and Eskenazi, Maxine},
  booktitle = {Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2017},
  doi = {10.18653/v1/p17-1061},
  url = {https://doi.org/10.18653/v1/p17-1061}
}

References

28 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chia-Wei Liu, Ryan Lowe, Iulian V Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. arXiv preprint arXiv:1603.08023 . find
  2. Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2016. Fine-grained analysis of sentence embeddings using auxiliary prediction tasks. arXiv preprint arXiv:1608.04207 . find our card
  3. Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural language processing with Python. ” O’Reilly Media, Inc.”. find
  4. Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of Machine Learning Research 9(Nov):2579–2605. find
  5. David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. Latent dirichlet allocation. Journal of machine Learning research 3(Jan):993–1022. find our card
  6. Dan Bohus and Alexander I Rudnicky. 2003. Ravenclaw: Dialog management using hierarchical task decomposition and an expectation agenda . find
  7. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics. Association for Computational Linguistics, pages 311–318. find our card
  8. Samuel R Bowman, Luke Vilnis, Oriol Vinyals, Andrew M Dai, Rafal Jozefowicz, and Samy Bengio. 2015. Generating sentences from a continuous space. arXiv preprint arXiv:1511.06349 . find our card
  9. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In EMNLP. volume 14, pages 1532– 43. find
  10. Boxing Chen and Colin Cherry. 2014. A systematic comparison of smoothing techniques for sentence-level bleu. find our card
  11. Massimo Poesio and David Traum. 1998. Towards an axiomatization of dialogue acts. In Proceedings of the Twente Workshop on the Formal Semantics and Pragmatics of Dialogues (13th Twente Workshop on Language Technology. Citeseer. find
  12. Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 . find
  13. Antoine Raux, Brian Langner, Dan Bohus, Alan W Black, and Maxine Eskenazi. 2005. Lets go public! taking a spoken dialog system to the real world. In in Proc. of Interspeech 2005. Citeseer. find
  14. Gabriel Forgues, Joelle Pineau, Jean-Marie Larchevêque, and Réal Tremblay. 2014. Bootstrapping dialog systems with word embeddings. In NIPS, Modern Machine Learning and Natural Language Processing Workshop. find
  15. Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. arXiv preprint arXiv:1401.4082 . find our card
  16. John J Godfrey and Edward Holliman. 1997. Switchboard-1 release 2. Linguistic Data Consortium, Philadelphia . find
  17. Eugénio Ribeiro, Ricardo Ribeiro, and David Martins de Matos. 2015. The influence of context on dialogue act recognition. arXiv preprint arXiv:1506.00839 . find
  18. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 . find
  19. Diederik P Kingma and Max Welling. 2013. Autoencoding variational bayes. arXiv preprint arXiv:1312.6114 . find our card
  20. Gerard Salton and Christopher Buckley. 1988. Term-weighting approaches in automatic text retrieval. Information processing & management 24(5):513– 523. find
  21. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2015. A diversity-promoting objective function for neural conversation models. arXiv preprint arXiv:1510.03055 . find our card
  22. Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing 45(11):2673–2681. find
  23. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016a. A persona-based neural conversation model. arXiv preprint arXiv:1603.06155 . find our card
  24. Iulian V Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, and Joelle Pineau. 2016a. Building end-to-end dialogue systems using generative hierarchical neural network models. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI-16). find our card
  25. Jiwei Li, Will Monroe, Alan Ritter, and Dan Jurafsky. 2016b. Deep reinforcement learning for dialogue generation. arXiv preprint arXiv:1606.01541 . find our card
  26. Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2016b. A hierarchical latent variable encoder-decoder model for generating dialogues. arXiv preprint arXiv:1605.06069 . find our card
  27. Diane J Litman and James F Allen. 1987. A plan recognition model for subdialogues in conversations. Cognitive science 11(2):163–200. find
  28. A Kihyuk Sohn, Honglak Lee, and Xinchen Yan. 2015. Learning structured output representation using deep conditional generative models. In Advances in Neural Information Processing Systems. pages 3483–3491. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhao-2017-learning-discourse-level-diversity-neural/