Elamur Research
☰
home / papers / 2015 / Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models

Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1507.04808

Summary

Open-domain chatbots of this era either retrieved existing responses or relied on hand-built state and action representations, which made them expensive to build and tied to narrow domains. This paper treats dialogue as a generative modelling problem. A model learns the probability of each utterance, token by token, given the conversation so far, and it can then produce new replies. The authors adapt the hierarchical recurrent encoder-decoder (HRED), first built for web query suggestion, to conversation. One RNN encodes each utterance, a second context RNN follows how the dialogue unfolds across turns, and a decoder RNN generates the next utterance. To test it they release MovieTriples, a set of three-turn exchanges taken from movie scripts. When scored by perplexity, HRED matches or beats n-gram backoff models and standard RNN language models. Pretraining on the large SubTle question-answer corpus and starting from Word2Vec embeddings give bigger gains than the architecture changes. The paper also shows that the most probable (MAP) replies tend to be bland and generic, which is an early and influential observation about neural dialogue models.

Key points

  • New resource: the MovieTriples dataset, made of three consecutive utterances from movie scripts and built with help from the Movie-DiC data. Speech acts such as pauses and end of turn are encoded as tokens.
  • Model variants: the baseline HRED and an HRED-Bidirectional version. The bidirectional encoder held on to information from the first two turns better than a plain RNN or the original HRED.
  • Bootstrapping: pretraining every parameter on SubTle cut perplexity by almost 10 points compared with HRED trained without it. Word2Vec initialization helped less because it only sets the word embeddings.
  • Generic responses: beam-search MAP outputs were often replies like "I don't know". Random sampling produced more varied, on-topic text. The authors argue that metrics computed on MAP outputs, such as BLEU, cosine similarity and Levenshtein distance, are biased in this setting.
  • Proposed next steps: model full-length dialogues, use longer and multimodal context (audio, video), and find better evaluation measures than perplexity.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

421 papers of our library cite it (of the 140673 whose bibliographies we read); 13 of them have a page here, 408 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1507.04808 · arXiv 1507.04808

@misc{serban2015building,
  title = {{Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models}},
  author = {Serban, Iulian V. and Sordoni, Alessandro and Bengio, Yoshua and Courville, Aaron and Pineau, Joelle},
  year = {2015},
  doi = {10.48550/arxiv.1507.04808},
  eprint = {1507.04808},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1507.04808}
}

References

40 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ameixa, D.; Coheur, L.; Fialho, P.; and Quaresma, P. 2014. Luke, I am your father: dealing with out-of-domain requests by using movies subtitles. In Intelligent Virtual Agents , 13– 21. find
  2. Auli, M.; Galley, M.; Quirk, C.; and Zweig, G. 2013. Joint language and translation modeling with recurrent neural networks. In Conference on Empirical Methods in Natural Language Processing , 1044–1054. find
  3. Bahdanau, D.; Cho, K.; and Bengio, Y. 2015. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations . find our card
  4. Banchs, R. E., and Li, H. 2012. IRIS: a chat-oriented dialogue system based on the vector space model. In Proceedings of the Association for Computational Linguistics, System Demonstrations , 37–42. find
  5. Banchs, R. E. 2012. Movie-DiC: A movie dialogue corpus for research and development. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics , 203–207. find
  6. Bastien, F.; Lamblin, P.; Pascanu, R.; Bergstra, J.; Goodfellow, I. J.; Bergeron, A.; Bouchard, N.; and Bengio, Y. 2012. Theano: new features and speed improvements. Conference on Neural Information Processing Systems, Workshop on Deep Learning and Unsupervised Feature Learning. find
  7. Bengio, Y.; Ducharme, R.; Vincent, P.; and Jauvin, C. 2003. A neural probabilistic language model. Journal of Machine Learning Research 3:1137–1155. find
  8. Bengio, Y. 2012. Practical recommendations for gradient-based training of deep architectures. In Tricks of the Trade, Second Edition , volume 7700 of Theoretical Computer Science and General Issues . Springer-Verlag. 437–478. find
  9. Bird, S.; Klein, E.; and Loper, E. 2009. Natural Language Processing with Python . O’Reilly Media. find
  10. Cantrell, R.; Talamadupula, K.; Schermerhorn, P.; Benton, J.; Kambhampati, S.; and Scheutz, M. 2012. Tell me when and why to do it! Run-time planner model updates via natural language instruction. In ACM/IEEE International Conference on Human-Robot Interaction , 471–478. find
  11. Cho, K.; van Merrienboer, B.; Gulcehre, C.; Bahdanau, D.; Bougares, F.; Schwenk, H.; and Bengio, Y. 2014. Learning phrase representations using RNN Encoder-Decoder for statistical machine translation. In Conference on Empirical Methods in Natural Language Processing , 1724–1734. find
  12. Clark, H. H., and Brennan, S. E. 1991. Grounding in communication. Perspectives on socially shared cognition 13(1991):127–149. find
  13. Deng, L., and Li, X. 2013. Machine learning paradigms for speech recognition: An overview. Audio, Speech, and Language Processing, IEEE Transactions on 21(5):1060–1089. find
  14. Forchini, P. 2009. Spontaneity reloaded: American face-to-face and movie conversation compared. In Corpus Linguistics 2009. Abstracts of the Corpus Linguistics Conference , 118. find
  15. Galley, M.; Brockett, C.; Sordoni, A.; Ji, Y.; Auli, M.; Quirk, C.; Mitchell, M.; Gao, J.; and Dolan, B. 2015. deltaBLEU: A discriminative metric for generation tasks with intrinsically diverse targets. In Proceedings of the Annual Meeting of the Association for Computational Linguistics and the Int… find
  16. Gasic, M.; Breslin, C.; Henderson, M.; Kim, D.; Szummer, M.; Thomson, B.; Tsiakoulis, P.; and Young, S. 2013. Online policy optimisation of Bayesian spoken dialogue systems via human interaction. In IEEE International Conference on Acoustics, Speech and Signal Processing , 8367–8371. find
  17. Goodfellow, I. J.; Warde-Farley, D.; Mirza, M.; Courville, A.; and Bengio, Y. 2013. Maxout networks. Proceedings of the International Conference on Machine Learning 1319– 1327. find
  18. Goodman, J. T. 2001. A bit of progress in language modeling extended version. Machine Learning and Applied Statistics Group Microsoft Research. Technical Report, MSR-TR-2001-72 . find
  19. Graves, A. 2012. Sequence transduction with recurrent neural networks. In Proceedings of the International Conference on Machine Learning, Representation Learning Workshop . find our card
  20. Henderson, M.; Thomson, B.; and Young, S. 2014. Word-based dialog state tracking with recurrent neural networks. In Annual Meeting of the Special Interest Group on Discourse and Dialogue , 292–299. find
  21. Hinton, G.; Deng, L.; Yu, D.; Dahl, G. E.; Mohamed, A.-r.; Jaitly, N.; Senior, A.; Vanhoucke, V.; Nguyen, P.; Sainath, T. N.; et al. 2012. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. Signal Processing Magazine, IEEE 29(6):82–97. find
  22. Kingma, D., and Ba, J. 2014. Adam: A method for stochastic optimization. In International Conference for Learning Representations . find
  23. Kiros, R.; Salakhutdinov, R.; and Zemel, R. 2014. Multimodal neural language models. In Proceedings of the International Conference on Machine Learning , 595–603. find
  24. Langley, P.; Meadows, B.; Gabaldon, A.; and Heald, R. 2014. Abductive understanding of dialogues about joint activities. Interaction Studies 15(3):426–454. find
  25. Li, J.; Galley, M.; Brockett, C.; Gao, J.; and Dolan, B. 2015. A diversity-promoting objective function for neural conversation models. CoRR abs/1510.03055. Mikolov, T.; Karafiát, M.; Burget, L.; Cernockỳ, J.; and Khudanpur, S. 2010. Recurrent neural network based language model. In Annual Confere… find our card
  26. Mikolov, T.; Sutskever, I.; Chen, K.; Corrado, G. S.; and Dean, J. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems , 3111–3119. find
  27. Mohan, S., and Laird, J. 2014. Learning goal-oriented hierarchical tasks from situated interactive instruction. In AAAI Conference on Artificial Intelligence , 387–394. find
  28. Pietquin, O., and Hastie, H. 2013. A survey on metrics for the evaluation of user simulations. The Knowledge Engineering Review 28(01):59–73. find
  29. Ritter, A.; Cherry, C.; and Dolan, B. 2010. Unsupervised modeling of Twitter conversations. In The Annual Conference of the North American Chapter of the Association for Computational Linguistics, Human Language Technologies , 172–180. find
  30. Ritter, A.; Cherry, C.; and Dolan, W. B. 2011. Data-driven response generation in social media. In Proceedings of the Empiricial Methods in Natural Language , 583–593. find
  31. Schatzmann, J.; Georgila, K.; and Young, S. 2005. Quantitative evaluation of user simulation techniques for spoken dialogue systems. In Special Interest Group on Discourse and Dialogue Workshop on Discourse and Dialogue . find
  32. Shang, L.; Lu, Z.; and Li, H. 2015. Neural responding machine for short-text conversation. In Association for Computational Linguistics , 1577–1586. find our card
  33. Shawar, B. A., and Atwell, E. 2007. Chatbots: are they really useful? In LDV Forum , volume 22, 29–49. find
  34. Singh, S.; Litman, D.; Kearns, M.; and Walker, M. 2002. Optimizing dialogue management with reinforcement learning: Experiments with the NJFun system. Journal of Artificial Intelligence Research 105–133. find
  35. Sordoni, A.; Bengio, Y.; Vahabi, H.; Lioma, C.; Simonsen, J. G.; and Nie, J.-Y. 2015a. A hierarchical recurrent encoder-decoder for generative context-aware query suggestion. In Proceedings of the ACM International Conference on Information and Knowledge Management , 553–562. find
  36. Sordoni, A.; Galley, M.; Auli, M.; Brockett, C.; Ji, Y.; Mitchell, M.; Nie, J.-Y.; Gao, J.; and Dolan, B. 2015b. A neural network approach to context-sensitive generation of conversational responses. In Conference of the North American Chapter of the Association for Computational Linguistics , 196–2… find our card
  37. Sutskever, I.; Vinyals, O.; and Le, Q. V. 2014. Sequence to sequence learning with neural networks. In Advances In Neural Information Processing Systems , 3104–3112. find
  38. Vinyals, O., and Le, Q. 2015. A neural conversational model. Proceedings of the International Conference on Machine Learning, Deep Learning Workshop . find our card
  39. Vinyals, O.; Toshev, A.; Bengio, S.; and Erhan, D. 2015. Show and tell: A neural image caption generator. In The IEEE Conference on Computer Vision and Pattern Recognition , 2155–2162. find our card
  40. Young, S.; Gasic, M.; Thomson, B.; and Williams, J. D. 2013. POMDP-based statistical spoken dialog systems: A review. IEEE 101(5):1160–1179. find