Elamur Research
☰
home / papers / 2016 / A Diversity-Promoting Objective Function for Neural Conversation Models

A Diversity-Promoting Objective Function for Neural Conversation Models

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/N16-1014.pdf

Summary

Neural SEQ2SEQ conversation models often give bland replies such as "I don't know" no matter what the input is. These replies are very common in training data, so they score well under the usual maximum-likelihood decision rule, log p(T|S). Li et al. argue that the objective itself causes the problem. They replace it with Maximum Mutual Information (MMI) between message and response, which favours replies that fit the specific input. They derive two variants. MMI-antiLM subtracts a weighted language-model term, λ·log p(T). MMI-bidi combines p(T|S) with a reverse model, p(S|T). The LSTM models are trained with ordinary likelihood, and MMI is applied only at decoding time, so changing λ does not require retraining. Two extra fixes make decoding practical: a position-dependent weight on the anti-LM term prevents ungrammatical output, and reranking N-best lists keeps bidirectional search tractable. On Twitter and OpenSubtitles data, MMI decoding beats SEQ2SEQ and SMT baselines in BLEU, output diversity and human judgments. This was an early, influential treatment of blandness in neural dialogue generation.

Key points

  • The baseline is a 4-layer SEQ2SEQ LSTM trained on about 20M OpenSubtitles (OSDb) conversation pairs. About 0.45% of its training sentences contain "I don't know", and generic replies dominated its top beam outputs (beam size 200).
  • The general objective is log p(T|S) − λ·log p(T). Using Bayes' rule it can be rewritten as (1−λ)·log p(T|S) + λ·log p(S|T), which gives the MMI-antiLM and MMI-bidi variants.
  • MMI-antiLM replaces p(T) with U(T). U(T) multiplies each token's probability by a weight g(k) that decreases as the token position k grows, so the penalty applies mainly to the first words of a reply. This avoids the ungrammatical outputs that the plain anti-LM term produced.
  • In pairwise human evaluation of 1,000 items, MMI-bidi significantly beat the greedy and beam SEQ2SEQ, SMT and SMT+RNN baselines (α = 0.05). Rated on its own, its mean overall-quality score was 3.84 on a 5-point scale, with 7 judges per item.
  • The authors present this as the first work to address output diversity in neural generation. They expect the same approach to help image captioning, question answering and other generation tasks.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Machine translation

Cited by

999 papers of our library cite it (of the 140673 whose bibliographies we read); 38 of them have a page here, 961 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n16-1014 · arXiv 1510.03055

@inproceedings{li2016diversity,
  title = {{A Diversity-Promoting Objective Function for Neural Conversation Models}},
  author = {Li, Jiwei and Galley, Michel and Brockett, Chris and Gao, Jianfeng and Dolan, Bill},
  booktitle = {Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2016},
  doi = {10.18653/v1/n16-1014},
  eprint = {1510.03055},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/n16-1014}
}

References

32 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David Ameixa, Luisa Coheur, Pedro Fialho, and Paulo Quaresma. 2014. Luke, I am your father: dealing with out-of-domain requests by using movies subtitles. In Intelligent Virtual Agents, pages 13–21. find
  2. Springer. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proc. of the International Conference on Learning Representations (ICLR). find our card
  3. L. Bahl, P. Brown, P. de Souza, and R. Mercer. 1986. Maximum mutual information estimation of hidden Markov model parameters for speech recognition. Acoustics, Speech, and Signal Processing, IEEE International Conference on ICASSP ’86., pages 49–52. find
  4. Rafael E Banchs and Haizhou Li. 2012. IRIS: a chat-oriented dialogue system based on the vector space model. In Proc. of the ACL 2012 System Demonstrations, pages 37–42. find
  5. Peter F. Brown. 1987. The Acoustic-modeling Problem in Automatic Speech Recognition. Ph.D. thesis, Carnegie Mellon University. find
  6. Jaime Carbonell and Jade Goldstein. 1998. The use of mmr, diversity-based reranking for reordering documents and producing summaries. In In Research and Development in Information Retrieval, pages 335–336. find
  7. Yun-Nung Chen, Wei Yu Wang, and Alexander Rudnicky. 2013. An empirical investigation of sparse log-linear models for improved dialogue act classification. In Proc. of ICASSP, pages 8317–8321. find
  8. Michel Galley, Chris Brockett, Alessandro Sordoni, Yangfeng Ji, Michael Auli, Chris Quirk, Margaret Mitchell, Jianfeng Gao, and Bill Dolan. 2015. ∆B LEU : A discriminative metric for generation tasks with intrinsically diverse targets. In Proc. of ACL-IJCNLP, pages 445–450, Beijing, China, July. find
  9. Jianfeng Gao, Xiaodong He, Wen-tau Yih, and Li Deng. 2014. Learning continuous phrase representations for translation modeling. In Proc. of ACL, pages 699–709. find
  10. Kevin Gimpel, Dhruv Batra, Chris Dyer, and Gregory Shakhnarovich. 2013. A systematic exploration of diversity in machine translation. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1100–1111. find
  11. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735– 1780. find
  12. Xuedong Huang, Alex Acero, Hsiao-Wuen Hon, and Raj Foreword By-Reddy. 2001. Spoken language processing: A guide to theory, algorithm, and system development. find
  13. Prentice Hall. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine trans… find
  14. Esther Levin, Roberto Pieraccini, and Wieland Eckert. 2000. A stochastic model of human-machine interaction for learning dialog strategies. IEEE Transactions on Speech and Audio Processing, 8(1):11–23. find
  15. Thang Luong, Ilya Sutskever, Quoc Le, Oriol Vinyals, and Wojciech Zaremba. 2015. Addressing the rare word problem in neural machine translation. In Proc. of ACL-IJCNLP, pages 11–19, Beijing, China. find our card
  16. Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, Zhiheng Huang, and Alan Yuille. 2015. Deep captioning with multimodal recurrent neural networks (m-RNN). find
  17. ICLR. Lasguido Nio, Sakriani Sakti, Graham Neubig, Tomoki Toda, Mirna Adriani, and Satoshi Nakamura. 2014. Developing non-goal dialog system based on examples of drama television. In Natural Interaction with Robots, Knowbots and Smartphones, pages 355–361. find
  18. Springer. Franz Josef Och. 2003. Minimum error rate training in statistical machine translation. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 160–167, Sapporo, Japan, July. Association for Computational Linguistics. find
  19. Alice H Oh and Alexander I Rudnicky. 2000. Stochastic language generation for spoken dialogue systems. In Proc. of the 2000 ANLP/NAACL Workshop on Conversational systems-Volume 3, pages 27–32. Association for Computational Linguistics. find
  20. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. B LEU : a method for automatic evaluation of machine translation. In Proc. of ACL. find
  21. Roberto Pieraccini, David Suendermann, Krishna Dayanidhi, and Jackson Liscombe. 2009. Are we there yet? research in commercial spoken dialog systems. In Text, Speech and Dialogue, pages 3–13. find
  22. Springer. Adwait Ratnaparkhi. 2002. Trainable approaches to surface natural language generation and their application to conversational dialog systems. Computer Speech & Language, 16(3):435–455. find
  23. Alan Ritter, Colin Cherry, and William Dolan. 2011. Data-driven response generation in social media. In Proc. of EMNLP, pages 583–593. find
  24. Iulian V Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, and Joelle Pineau. 2015. Hierarchical neural network generative models for movie dialogues. arXiv preprint arXiv:1507.04808. find our card
  25. Lifeng Shang, Zhengdong Lu, and Hang Li. 2015. Neural responding machine for short-text conversation. In ACL-IJCNLP, pages 1577–1586. find our card
  26. Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Meg Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan. 2015. A neural network approach to context-sensitive generation of conversational responses. In Proc. of NAACL-HLT. find our card
  27. Ilya Sutskever, Oriol Vinyals, and Quoc Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112. Jörg Tiedemann. 2009. News from OPUS – a collection of multilingual parallel corpora with tools and interfaces. In Recent adva… find
  28. Oriol Vinyals and Quoc Le. 2015. A neural conversational model. In Proc. of ICML Deep Learning Workshop. find our card
  29. Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2015. Grammar as a foreign language. In Proc. of NIPS. find
  30. Marilyn A Walker, Rashmi Prasad, and Amanda Stent. 2003. A trainable generator for recommendations in multimodal dialog. In INTERSPEECH. find
  31. William Yang Wang, Ron Artstein, Anton Leuski, and David Traum. 2011. Improving spoken dialogue understanding using phonetic mixture models. In FLAIRS. Tsung-Hsien Wen, Milica Gasic, Nikola Mrkšić, Pei-Hao Su, David Vandyke, and Steve Young. 2015. Semantically conditioned LSTM-based natural langua… find
  32. Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. 2015. Show, attend and tell: Neural image caption generation with visual attention. In David Blei and Francis Bach, editors, Proc. of the 32nd International Conference on Machine Lear… find our card