Elamur Research⌕
☰
home / papers / 2019 / Personalizing Dialogue Agents via Meta-Learning

Personalizing Dialogue Agents via Meta-Learning

Paper · 2019 · Andrea Madotto, Zhaojiang Lin, Chien-Sheng Wu et al.

Summary

Personalized chit-chat models usually make responses consistent by conditioning them on short, hand-written persona profiles. Such profiles are costly to produce and hard to extract from real conversations. Madotto et al. avoid profiles entirely. They treat each persona as a separate task and apply Model-Agnostic Meta-Learning (MAML). Their variant, Persona-Agnostic Meta-Learning (PAML), learns initial weights for a Transformer response generator so that a few dialogues from one user are enough to fine-tune it toward that user's style.

To build per-persona dialogue sets, the authors group Persona-chat conversations by their persona description. Evaluation combines four signals:

  • perplexity and BLEU;
  • a new consistency score C, computed with a BERT model fine-tuned on Dialog NLI;
  • crowd-sourced human ratings.

PAML is the most persona-consistent system in both C and human judgments, and humans rate its fluency as comparable to the baselines. It does have weaker perplexity and BLEU. It also outperforms a model that sees the persona description directly. The result suggests that personalization can come from a user's past conversations rather than from manually written descriptions.

Key points

  • Data: On average, each persona in Persona-chat has 8.3 unique dialogues. At test time, the model is fine-tuned on all of a persona's dialogues except one, and the held-out dialogue is used for evaluation.

  • Consistency metric C: Each generated utterance is scored +1 if it entails a persona sentence, 0 if neutral, and −1 if it contradicts one, summed over the persona sentences. The BERT classifier behind it reaches 88.43% accuracy on the DNLI test set, on par with ESIM's 88.20%.

  • Automatic results (C / PPL / BLEU):

    SystemCPPLBLEU
    PAML0.2041.640.74
    Dialogue+Persona0.0730.421.00
    Dialogue+Fine-tuning0.0032.960.90
    Human reference0.33——

  • Human evaluation: 300 responses from 10 personas were rated. PAML scored a fluency of 3.185 and a consistency of 0.197. Dialogue+Persona scored 3.053 fluency and 0.011 consistency.

  • Few-shot behaviour and code: PAML exceeds Dialogue+Persona in consistency with just 3 dialogues, and its consistency grows roughly linearly with k. Plain fine-tuning barely benefits from extra dialogues. The model uses GloVe embeddings, SGD in the inner loop (α=0.01), and Adam in the outer loop (β=0.0003). Code is available at github.com/HLTCHKUST/PAML.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Hong Kong University of Science and TechnologyHong Kong · China58 works · 23,058 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Dialogue systems · 401 works

Cite

Cite as Andrea Madotto, Zhaojiang Lin, Chien-Sheng Wu and Pascale Fung. 2019. Personalizing Dialogue Agents via Meta-Learning. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p19-1542

@inproceedings{madotto2019personalizing,
  title = {{Personalizing Dialogue Agents via Meta-Learning}},
  author = {Andrea Madotto, Zhaojiang Lin, Chien-Sheng Wu and Pascale Fung},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1542},
  url = {https://doi.org/10.18653/v1/p19-1542}
}

References

37 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando De Freitas. 2016. Learning to learn by gradient descent by gradient descent. In Advances in Neural Information Processing Systems, pages 3981–3989. find
  2. Chaitanya K Joshi, Fei Mi, and Boi Faltings. 2017. Personalization in goal-oriented dialog. arXiv preprint arXiv:1706.07503. — [arXiv:1706.07503](https://arxiv.org/abs/1706.07503) find
  3. Samy Bengio, Yoshua Bengio, Jocelyn Cloutier, and Jan Gecsei. 1992. On the optimization of a synaptic learning rule. In Preprints Conf. Optimality in Artificial and Biological Neural Networks, pages 6–8. Univ. of Texas. find
  4. Ilya Kulikov, Alexander H Miller, Kyunghyun Cho, and Jason Weston. 2018. Importance of a search strategy in neural dialogue modelling. arXiv preprint arXiv:1811.00907. — [arXiv:1811.00907](https://arxiv.org/abs/1811.00907) find
  5. Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. Enhanced lstm for natural language inference. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1657–1668. Association for Computational Linguisti… find
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
  7. Emily Dinan, Varvara Logacheva, Valentin Malykh, Alexander Miller, Kurt Shuster, Jack Urbanek, Douwe Kiela, Arthur Szlam, Iulian Serban, Ryan Lowe, et al. 2019. The second conversational intelligence challenge (convai2). arXiv preprint arXiv:1902.00098.
  8. Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1126–1135. JMLR. org. find
  9. Jianfeng Gao, Michel Galley, and Lihong Li. 2018. Neural approaches to conversational ai. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 1371–1374. ACM. find
  10. Jiatao Gu, Yong Wang, Yun Chen, Victor O. K. Li, and Kyunghyun Cho. 2018. Meta-learning for low-resource neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3622–3631. Association for Computational Linguistics.
  11. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. find
  12. Jiwei Li, Michel Galley, Chris Brockett, Georgios Spithourakis, Jianfeng Gao, and Bill Dolan. 2016. A persona-based neural conversation model. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 994–1003.
  13. Zhaojiang Lin, Genta Indra Winata, and Pascale Fung. 2019. Learning comment generation by leveraging user-generated data. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7225–7229. IEEE. find
  14. Chia-Wei Liu, Ryan Lowe, Iulian Serban, Mike Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural L…
  15. JM Lucas, F Fernández, J Salazar, J Ferreiros, and R San Segundo. 2009. Managing speaker identity and user profiles in a spoken dialogue system. Procesamiento del lenguaje natural, (43). find
  16. Andrea Madotto, Chien-Sheng Wu, and Pascale Fung. 2018. Mem2seq: Effectively incorporating knowledge bases into end-to-end task-oriented dialog systems. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1468–1478.
  17. Pierre-Emmanuel Mazare, Samuel Humeau, Martin Raison, and Antoine Bordes. 2018. Training millions of personalized dialogue agents. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2775–2779. Association for Computational Linguistics.
  18. Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. 2017. A simple neural attentive meta-learner. ICLR. find
  19. Devang K Naik and RJ Mammone. 1992. Meta-neural networks that learn by learning. In [Proceedings 1992] IJCNN International Joint Conference on Neural Networks, volume 1, pages 437–442. IEEE. find
  20. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics.
  21. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543.
  22. Sachin Ravi and Hugo Larochelle. 2016. Optimization as a model for few-shot learning. find
  23. Revanth Reddy, Danish Contractor, Dinesh Raghu, and Sachindra Joshi. 2018. Multi-level memory for task oriented dialogs. arXiv preprint arXiv:1810.10647. find
  24. Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. 2016. Meta-learning with memory-augmented neural networks. In International conference on machine learning, pages 1842–1850. find
  25. Jurgen Schmidhuber. 1987. Evolutionary principles in self-referential learning. on learning now to learn: The meta-meta-meta...-hook. Diploma thesis, Technische Universitat Munchen, Germany, 14 May. find
  26. Jürgen Schmidhuber. 1992. Learning to control fast-weight memories: An alternative to dynamic recurrent networks. Neural Computation, 4(1):131–139. find
  27. Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. 2016. Matching networks for one shot learning. In Advances in neural information processing systems, pages 3630–3638. find
  28. Chien-Sheng Wu, Andrea Madotto, Genta Winata, and Pascale Fung. 2017. End-to-end recurrent entity network for entity-value independent goal-oriented dialog learning. In Dialog System Technology Challenges Workshop, DSTC6. find
  29. Chien-Sheng Wu, Andrea Madotto, Genta Indra Winata, and Pascale Fung. 2018. End-to-end dynamic query memory network for entity-value independent task-oriented dialog. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6154–6158. IEEE. find
  30. Chien-Sheng Wu, Richard Socher, and Caiming Xiong. 2019. Global-to-local memory pointer networks for task-oriented dialogue. In International Conference on Learning Representations. find
  31. Semih Yavuz, Abhinav Rastogi, Guan-lin Chao, Dilek Hakkani-Tür, and Amazon Alexa AI. Deepcopy: Grounded response generation with hierarchical pointer networks. find
  32. Mo Yu, Xiaoxiao Guo, Jinfeng Yi, Shiyu Chang, Saloni Potdar, Yu Cheng, Gerald Tesauro, Haoyu Wang, and Bowen Zhou. 2018. Diverse few-shot text classification with multiple metrics. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: H…
  33. Yury Zemlyanskiy and Fei Sha. 2018. Aiming to know you better perhaps makes me a more engaging dialogue partner. find
  34. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204– 2213. Assoc…
  35. Welleck Sean, Jason Weston, Arthur Szlam, and Kyunghyun Cho. 2018. Dialogue natural language inference. arXiv preprint arXiv:1811.00671.
  36. Sebastian Thrun and Lorien Pratt, editors. 1998. Learning to Learn. Kluwer Academic Publishers, Norwell, MA, USA. find
  37. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Proces…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/madotto-2019-personalizing-dialogue-agents-meta-learning/