Elamur Research
☰
home / papers / 2016 / A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues

A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1605.06069

Summary

Standard RNN-based dialogue generators, including RNNLM and HRED, introduce randomness only when each output token is sampled. The authors argue that this pushes models toward local, next-token prediction and makes them lose the broader conversational thread, which often yields bland, generic replies. They propose VHRED, which extends HRED's encoder, context and decoder RNNs with a continuous Gaussian latent variable sampled once per utterance. Both the prior and the approximate posterior over this variable are produced by feed-forward networks on top of the context RNN, and training maximizes a variational lower bound in the style of the VAE. The latent variable is meant to capture high-level choices such as topic or sentiment, so the decoder's hidden state can concentrate on word-level detail. The model is tested on response generation for Twitter and Ubuntu dialogues against LSTM and HRED baselines, using a human study on Amazon Mechanical Turk, embedding-based similarity metrics and entropy analysis. VHRED stays closer to the topic of the context, especially across three generated turns, and its responses carry more information. The design applies to any hierarchically structured sequence generation task.

Key points

  • Architecture: VHRED adds one diagonal-Gaussian latent variable per utterance to HRED. It is drawn from the approximate posterior during training and from the conditional prior at test time, then concatenated with the context RNN output before decoding. The authors retained a direct deterministic connection feeding the context RNN's output straight into the decoder, since doing so reduced the variational bound.
  • Evaluation setup: Two dialogue corpora, Twitter and Ubuntu, were used, with LSTM and HRED as baselines. Evaluation combined human judgments on Amazon Mechanical Turk with word2vec embedding-based metrics for 1-turn and 3-turn generation.
  • Topical coherence: VHRED matched the topic of the ground-truth response better than LSTM and HRED, with the clearest gains when generating three consecutive utterances.
  • Information content on Twitter (Table 4): VHRED produced the longest responses (12.29 tokens vs. 11.64 for HRED and 11.21 for LSTM). It also had the highest utterance entropy (84.56 bits vs. 78.35 and 75.61). Human responses remain far richer, at 166.57 bits.
  • Information content on Ubuntu (Table 4): VHRED had the highest per-word entropy (7.70 vs. 7.53 for HRED and 6.50 for LSTM). However, its responses were shorter than HRED's (9.22 vs. 11.05 tokens), so its utterance entropy was lower (71.00 vs. 83.16).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

284 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 277 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1605.06069 · arXiv 1605.06069

@misc{serban2016hierarchical,
  title = {{A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues}},
  author = {Serban, Iulian Vlad and Sordoni, Alessandro and Lowe, Ryan and Charlin, Laurent and Pineau, Joelle and Courville, Aaron and Bengio, Yoshua},
  year = {2016},
  doi = {10.48550/arxiv.1605.06069},
  eprint = {1605.06069},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1605.06069}
}

References

29 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bayer, J. and Osendorfer, C. (2014). Learning stochastic recurrent networks. In NIPS Workshop on Advances in Variational Inference. find
  2. Boulanger-Lewandowski, N., Bengio, Y., and Vincent, P. (2012). Modeling temporal dependencies in high-dimensional sequences: Application to polyphonic music generation and transcription. In ICML. find
  3. Bowman, S. R., Vilnis, L., Vinyals, O., Dai, A. M., Jozefowicz, R., and Bengio, S. (2015). Generating sentences from a continuous space. arXiv:1511.06349. find our card
  4. Chelba, C., Mikolov, T., Schuster, M., Ge, Q., Brants, T., Koehn, P., and Robinson, T. (2014). One billion word benchmark for measuring progress in statistical language modeling. In INTERSPEECH. find our card
  5. Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using RNN encoder–decoder for statistical machine translation. In EMNLP, pages 1724–1734. find
  6. Chung, J., Kastner, K., Dinh, L., Goel, K., Courville, A., and Bengio, Y. (2015). A recurrent latent variable model for sequential data. In NIPS, pages 2962–2970. find
  7. Fabius, O. and van Amersfoort, J. R. (2014). Variational recurrent auto-encoders. arXiv:1412.6581. — [arXiv:1412.6581](https://arxiv.org/abs/1412.6581) find
  8. Galley, M., Brockett, C., Sordoni, A., Ji, Y., Auli, M., Quirk, C., Mitchell, M., Gao, J., and Dolan, B. (2015). deltaBLEU: A discriminative metric for generation tasks with intrinsically diverse targets. In ACL. find
  9. Goodfellow, I., Courville, A., and Bengio, Y. (2015). Deep Learning. MIT Press. find
  10. Graves, A. (2012). Sequence transduction with recurrent neural networks. In ICML RLW. find our card
  11. Gregor, K., Danihelka, I., Graves, A., and Wierstra, D. (2015). DRAW: A recurrent neural network for image generation. In ICLR. find
  12. Hochreiter, S. and Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8). find
  13. Kingma, D. and Ba, J. (2015). Adam: A Method for Stochastic Optimization. In ICLR. find
  14. Kingma, D. P. and Welling, M. (2014). Auto-encoding variational bayes. In ICLR. find our card
  15. Li, J., Galley, M., Brockett, C., Gao, J., and Dolan, B. (2016). A diversity-promoting objective function for neural conversation models. In NAACL. find our card
  16. Liu, C.-W., Lowe, R., Serban, I. V., Noseworthy, M., Charlin, L., and Pineau, J. (2016). How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. arXiv:1603.08023. find
  17. Lowe, R., Pow, N., Serban, I., and Pineau, J. (2015). The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems. In SIGDIAL. find our card
  18. Mikolov, T., Karafiát, M., Burget, L., Cernockỳ, J., and Khudanpur, S. (2010). Recurrent neural network based language model. In INTERSPEECH, pages 1045–1048. find
  19. Mitchell, J. and Lapata, M. (2008). Vector-based models of semantic composition. In ACL, pages 236–244. find
  20. Pietquin, O. and Hastie, H. (2013). A survey on metrics for the evaluation of user simulations. The knowledge engineering review, 28(01), 59–73. find
  21. Rezende, D. J., Mohamed, S., and Wierstra, D. (2014). Stochastic backpropagation and approximate inference in deep generative models. In ICML. find our card
  22. Ritter, A., Cherry, C., and Dolan, W. B. (2011). Data-driven response generation in social media. In EMNLP, pages 583–593. find
  23. Rus, V. and Lintean, M. (2012). A comparison of greedy and optimal assessment of natural language student input using word-to-word similarity metrics. In Building Educational Applications Workshop, ACL. find
  24. Serban, I. V., Sordoni, A., Bengio, Y., Courville, A. C., and Pineau, J. (2016). Building end-to-end dialogue systems using generative hierarchical neural network models. In AAAI, pages 3776–3784. find our card
  25. Shaikh, S., Strzalkowski, T., Taylor, S., and Webb, N. (2010). VCA: an experiment with a multiparty virtual chat agent. In ACL Workshop on Companionable Dialogue Systems, pages 43–48. find
  26. Sordoni, A., Bengio, Y., Vahabi, H., Lioma, C., Simonsen, J. G., and Nie, J.-Y. (2015a). A hierarchical recurrent encoder-decoder for generative context-aware query suggestion. In CIKM. find
  27. Sordoni, A., Galley, M., Auli, M., Brockett, C., Ji, Y., Mitchell, M., Nie, J.-Y., Gao, J., and Dolan, B. (2015b). A neural network approach to context-sensitive generation of conversational responses. In NAACL-HLT. find our card
  28. Sutskever, I., Vinyals, O., and Le, Q. V. (2014). Sequence to sequence learning with neural networks. In NIPS, pages 3104–3112. find
  29. Young, S., Gasic, M., Thomson, B., and Williams, J. D. (2013). POMDP-based statistical spoken dialog systems: A review. IEEE, 101(5), 1160–1179. find