Elamur Research⌕
☰
home / papers / 2019 / Importance of Search and Evaluation Strategies in Neural Dialogue Modeling

Importance of Search and Evaluation Strategies in Neural Dialogue Modeling

Paper · 2019 · Ilia Kulikov, Alexander Miller, Kyunghyun Cho et al.

Summary

Neural dialogue research has mostly focused on architectures and training objectives. Decoding, meaning how a response is actually generated from a trained model, has received far less attention. This paper isolates the effect of the search algorithm. The authors keep a single attention-based recurrent sequence-to-sequence model trained on PersonaChat fixed and compare three decoders: greedy search, standard beam search, and a newly proposed iterative beam search. The new method reruns beam search under constraints so that it explores regions of the response space the earlier passes missed, which yields more varied candidates.

The decoders are judged in two ways:

  • Human evaluation: annotators hold full multi-turn conversations with the model. A model-based Bayesian calibration corrects for individual annotator bias.
  • Automatic metrics: model log-probability and distinct-n diversity.

Greedy decoding, the usual default, clearly underperforms both beam variants, even though the model and its weights are unchanged. Iterative beam search produces more diverse candidates and higher-likelihood outputs, but humans rate it about the same as plain beam search. This points to final response selection as an unresolved bottleneck.

Key points

  • Setup: one attention-based RNN seq2seq model (Bahdanau et al.) trained on PersonaChat. Only the decoding strategy varies: greedy, beam, or iter-beam (each beam variant returns up to 15 hypotheses).
  • Human evaluation: 201 full conversations with 41 unique annotators. Scores are calibrated with a Bayesian model of annotator bias, and posterior means and variances are reported. Greedy lags behind both beam variants on all metrics.
  • Diversity of final responses (distinct-1, post-selection): greedy 0.47, beam 0.56, iter-beam 0.59, versus 0.66 for human utterances. On distinct-2 and distinct-3, humans (0.85 / 0.82) remain far ahead of every decoder (at most 0.69 / 0.63).
  • Candidates vs. final output: iter-beam's candidate pools are noticeably more varied than beam's. However, the selected output is not much more diverse, and human scores for beam and iter-beam are statistically indistinguishable.
  • Conclusions: the authors argue that the selection mechanism matters as much as the search itself. They also note that the model assigns low probability to reference responses, suggesting the model itself is suboptimal. They plan to release code, trained models, and conversation transcripts.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢MetaMenlo Park · United States214 works · 241,492 citations🎓New York UniversityNew York · United States219 works · 103,487 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 2Research subjects and languages the work is about.Evaluation & benchmarks · 1,341 worksDialogue systems · 654 works

Cite

Cite as Ilia Kulikov, Alexander Miller, Kyunghyun Cho and Jason Weston. 2019. Importance of Search and Evaluation Strategies in Neural Dialogue Modeling. Proceedings of the 12th International Conference on Natural Language Generation.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-8609

@inproceedings{kulikov2019importance,
  title = {{Importance of Search and Evaluation Strategies in Neural Dialogue Modeling}},
  author = {Ilia Kulikov, Alexander Miller, Kyunghyun Cho and Jason Weston},
  booktitle = {Proceedings of the 12th International Conference on Natural Language Generation},
  year = {2019},
  doi = {10.18653/v1/w19-8609},
  url = {https://doi.org/10.18653/v1/w19-8609}
}

References

42 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
  2. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent continuous translation models. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1700–1709.
  3. Dhruv Batra, Payman Yadollahpour, Abner Guzman-Rivera, and Gregory Shakhnarovich. 2012. Diverse m-best solutions in markov random fields. In European Conference on Computer Vision, pages 1–16. Springer. find
  4. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
  5. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M. Rush. 2017. Open-NMT: Open-source toolkit for neural machine translation. In Proc. ACL. find
  6. Eli Bingham, Jonathan P. Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, and Noah D. Goodman. 2018. Pyro: Deep Universal Probabilistic Programming. arXiv preprint arXiv:1810.09538. find
  7. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2015. A diversity-promoting objective function for neural conversation models. arXiv preprint arXiv:1510.03055.
  8. Yun Chen, Victor OK Li, Kyunghyun Cho, and Samuel R Bowman. 2018. A stable and effective learning strategy for trainable greedy decoding. arXiv preprint arXiv:1804.07915. find
  9. Jiwei Li, Will Monroe, and Dan Jurafsky. 2016. A simple, fast diverse decoding algorithm for neural generation. arXiv preprint arXiv:1611.08562. find
  10. Jiwei Li, Will Monroe, and Dan Jurafsky. 2017a. Learning to decode for future success. arXiv preprint arXiv:1701.06549. find
  11. Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078.
  12. Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter, and Dan Jurafsky. 2017b. Adversarial learning for neural dialogue generation. arXiv preprint arXiv:1701.06547.
  13. Dev K Dalal, Nathan T Carter, and Christopher J Lake. 2014. Middle response scale options are inappropriate for ideal point scales. Journal of Business and Psychology, 29(3):463–478. find
  14. Chia-Wei Liu, Ryan Lowe, Iulian V Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. arXiv preprint arXiv:1603.08023.
  15. Yann N Dauphin, Angela Fan, Michael Auli, and David Grangier. 2016. Language modeling with gated convolutional networks. arXiv preprint arXiv:1612.08083.
  16. Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015. Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025.
  17. Emily Dinan, Varvara Logacheva, Valentin Malykh, Alexander Miller, Kurt Shuster, Jack Urbanek, Douwe Kiela, Arthur Szlam, Iulian Serban, Ryan Lowe, et al. 2019. The second conversational intelligence challenge (convai2). arXiv preprint arXiv:1902.00098.
  18. Pierre-Emmanuel Mazaré, Samuel Humeau, Martin Raison, and Antoine Bordes. 2018. Training millions of personalized dialogue agents. arXiv preprint arXiv:1809.01984.
  19. Tomáš Mikolov, Martin Karafiát, Lukáš Burget, Jan Černockỳ, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In Eleventh Annual Conference of the International Speech Communication Association. find
  20. Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. 2017. Convolutional sequence to sequence learning. arXiv preprint arXiv:1705.03122.
  21. Ashwin K Vijayakumar, Michael Cogswell, Ramprasaath R Selvaraju, Qing Sun, Stefan Lee, David J Crandall, and Dhruv Batra. 2018. Diverse beam search for improved description of complex scenes. In AAAI. find
  22. Alexander H Miller, Will Feng, Adam Fisch, Jiasen Lu, Dhruv Batra, Antoine Bordes, Devi Parikh, and Jason Weston. 2017. ParlAI: A dialog research software platform. arXiv preprint arXiv:1705.06476.
  23. Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784. find
  24. Oriol Vinyals and Quoc Le. 2015. A neural conversational model. arXiv preprint arXiv:1506.05869.
  25. Romain Paulus, Caiming Xiong, and Richard Socher. 2017. A deep reinforced model for abstractive summarization. arXiv preprint arXiv:1705.04304.
  26. Jason Weston, Emily Dinan, and Alexander H Miller. 2018. Retrieve and refine: Improved sequence generation models for dialogue. arXiv preprint arXiv:1808.04776.
  27. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543.
  28. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144. find
  29. David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. 1985. Learning internal representations by error propagation. Technical report, California Univ San Diego La Jolla Inst for Cognitive Science. find
  30. Zhen Xu, Bingquan Liu, Baoxun Wang, Chengjie SUN, Xiaolong Wang, Zhuoran Wang, and Chao Qi. 2017. Neural response generation via gan with an approximate embedding layer. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 617–626. Association for Computat… find
  31. Iulian Vlad Serban, Ryan Lowe, Peter Henderson, Laurent Charlin, and Joelle Pineau. 2015. A survey of available corpora for building data-driven dialogue systems. arXiv preprint arXiv:1512.05742. find
  32. Iulian Vlad Serban, Alessandro Sordoni, Yoshua Bengio, Aaron C Courville, and Joelle Pineau. 2016. Building end-to-end dialogue systems using generative hierarchical neural network models. In AAAI, volume 16, pages 3776–3784.
  33. Yinfei Yang, Steve Yuan, Daniel Cer, Sheng-yi Kong, Noah Constant, Petr Pilar, Heming Ge, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. 2018. Learning semantic textual similarity from conversations. arXiv preprint arXiv:1804.07754. find
  34. Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C Courville, and Yoshua Bengio. 2017. A hierarchical latent variable encoder-decoder model for generating dialogues. In AAAI, pages 3295–3301.
  35. Yury Zemlyanskiy and Fei Sha. 2018. Aiming to know you better perhaps makes me a more engaging dialogue partner. arXiv preprint arXiv:1808.07104. find
  36. Xiaoyu Shen, Hui Su, Shuzi Niu, and Vera Demberg. 2018. Improving variational encoder-decoders in dialogue generation. arXiv preprint arXiv:1802.02032. — [arXiv:1802.02032](https://arxiv.org/abs/1802.02032) find
  37. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? arXiv preprint arXiv:1801.07243.
  38. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):1929–1958.
  39. Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. arXiv preprint arXiv:1703.10960.
  40. Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, et al. 2015. End-to-end memory networks. In Advances in neural information processing systems, pages 2440–2448. find
  41. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112.
  42. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, pages 5998–6008.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kulikov-2019-importance-search-evaluation-strategies-neural/