Elamur Research⌕
☰
home / papers / 2020 / MIME: MIMicking Emotions for Empathetic Response Generation

MIME: MIMicking Emotions for Empathetic Response Generation

Summary

The paper tackles empathetic response generation in dialogue. The authors argue that earlier systems, such as the state-of-the-art MoEL, treat all emotions as one flat, uniform set. MIME instead assumes that empathetic listeners partly copy the speaker's emotion, and that how much they copy depends on whether that emotion is positive or negative. Positive statements usually receive positive replies. Negative ones call for agreement mixed with some comfort or hope.

The model works in four steps:

  • A transformer encoder represents the dialogue context, and user-emotion classification is used as an auxiliary training signal.
  • The 32 emotion labels are split into a positive and a negative cluster.
  • For each cluster, a response-emotion distribution is sampled from a VHRED-style latent variable, which adds variety.
  • The two resulting emotion representations are fused to steer a transformer decoder.

The model was trained and evaluated on a large empathetic dialogue dataset. Automatic and human evaluations, plus ablations, show that MIME beats MoEL on empathy and contextual relevance, and that its replies are more emotionally varied. It is somewhat weaker in fluency. The work matters because it shows that structuring emotions by polarity and modelling mimicry improves empathetic dialogue agents.

Key points

  • The 32 emotion classes are divided into 13 positive and 19 negative groups. The authors admit the split is intuition-based, and that "surprised" (treated as positive) causes errors on negatively surprised users.

  • The stochastic sampling of the emotion mixture reduces the generic, repetitive replies typical of MoEL. A comparison with and without this stochasticity is provided.

  • Emotion classification accuracy (top-1/top-2/top-5):

    • MoEL: 38% / 63% / 74%
    • MIME: 34% / 58% / 77%

    The authors attribute this to MIME sharing emotion embeddings between the encoder and decoder.

  • Error analysis notes occasional incoherent output. The authors suggest that the model favoured empathy and relevance over fluency.

  • The implementation is publicly available at github.com/declare-lab/MIME.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

81 papers of our library cite it (of the 140884 whose bibliographies we read); 6 of them have a page here, 75 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.emnlp-main.721

@inproceedings{majumder2020mime,
  title = {{MIME: MIMicking Emotions for Empathetic Response Generation}},
  author = {Majumder, Navonil and Hong, Pengfei and Peng, Shanshan and Lu, Jiankun and Ghosal, Deepanway and Gelbukh, Alexander and Mihalcea, Rada and Poria, Soujanya},
  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.721},
  url = {https://doi.org/10.18653/v1/2020.emnlp-main.721}
}

References

44 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dario Bertero, Farhad Bin Siddique, Chien-Sheng Wu, Yan Wan, Ricky Ho Yin Chan, and Pascale Fung. 2016. Real-time speech emotion and sentiment recognition for interactive dialogue systems. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1042– 1047, Au… find
  2. A. Bohart and L. Greenberg. 1997. Empathy reconsidered: New directions in psychotherapy. American Psychological Association. find
  3. Samuel R Bowman, Luke Vilnis, Oriol Vinyals, Andrew M Dai, Rafal Jozefowicz, and Samy Bengio. 2015. Generating sentences from a continuous space. arXiv preprint arXiv:1511.06349. find our card
  4. S. Buechel, A. Buffone, B. Slaff, L. Ungar, and J. Sedoc. 2018. Modeling empathy and distress in reaction to news stories. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Belgium. find our card
  5. Deng Cai, Yan Wang, Victoria Bi, Zhaopeng Tu, Xiaojiang Liu, Wai Lam, and Shuming Shi. 2018. Skeleton-to-response: Dialogue generation guided by retrieval memory. find
  6. Laurie Carr, Marco Iacoboni, Marie-Charlotte Dubeau, John C. Mazziotta, and Gian Luigi Lenzi. 2003. Neural mechanisms of empathy in humans: A relay from neural systems for imitation to limbic areas. Proceedings of the National Academy of Sciences, 100(9):5497–5502. find
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  8. R. Dymond. 1950. Personality and empathy. Journal of Consulting Psychology, 14(5). find
  9. N. Eisenberg, T. Spinrad, and A. Morris. 2014. Empathy-related responding in children. Psychology Press. find
  10. Pascale Fung, Anik Dey, Farhad Bin Siddique, Ruixi Lin, Yang Yang, Yan Wan, and Ho Yin Ricky Chan. 2016. Zara the Supergirl: An empathetic personality recognition system. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrati… find
  11. Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial networks. find
  12. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735–1780. find
  13. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. find
  14. Diederik P Kingma and Max Welling. 2013. Autoencoding variational bayes. arXiv preprint arXiv:1312.6114. find our card
  15. D. Krebs. 1975. Empathy and altruism. Journal of Personality and Social psychology, 32(6). find
  16. Zhaojiang Lin, Andrea Madotto, Jamin Shin, Peng Xu, and Pascale Fung. 2019. MoEL: Mixture of empathetic listeners. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages… find our card
  17. Chia-Wei Liu, Ryan Lowe, Iulian Serban, Mike Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural L… find
  18. Minh-Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. find
  19. S. Mercer and W. Reynolds. 2002. Empathy and quality of care. British Journal of General Practice, 52. find
  20. Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Jason Weston. 2016. Key-value memory networks for directly reading documents. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1400–1409, Austin, Texas. Association for… find
  21. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational… find our card
  22. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group… find
  23. V. Pérez-Rosas, R. Mihalcea, K. Resnicow, S. Singh, and L. An. 2017. Understanding and predicting empathic behavior in counseling therapy. In Proceedings of the Association for Computational Linguistics, Vancouver, Canada. find
  24. Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2018. I know the feeling: Learning to converse with empathy. ArXiv, abs/1811.00207. find
  25. Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards empathetic open-domain conversation models: a new benchmark and dataset. In Proceedings of the Association for Computatinal Linguistics. find
  26. Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. arXiv preprint arXiv:1401.4082. - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7989–7993. find our card
  27. T. Singer and C. Lamm. 2009. The social neuroscience of empathy. Annals of the New York Academy of Sciences, 1156. find
  28. A. Smith. 2006. Cognitive empathy and emotional empathy in human behavior and evolution. The Psychological Record, 56(1). find
  29. Kihyuk Sohn, Honglak Lee, and Xinchen Yan. 2015. Learning structured output representation using deep conditional generative models. In Advances in neural information processing systems, pages 3483– 3491. find
  30. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. find
  31. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Proces… find
  32. Oriol Vinyals and Quoc Le. 2015. A neural conversational model. find our card
  33. J. Sedoc, S. Buechel, Yehonathan Y. Nachmany, A. Buffone, and L. Ungar. 2020. Learning word ratings for empathy and distress from document-level user responses. In Proceedings of The 12th Language Resources and Evaluation Conference, France. find
  34. Ke Wang and Xiaojun Wan. 2018. Sentigan: Generating sentimental texts via mixture adversarial networks. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pages 4446–4452. International Joint Conferences on Artificial Intelligence Organization. find
  35. Iulian Vlad Serban, Ryan Lowe, Laurent Charlin, and Joelle Pineau. 2016. Generative deep neural networks for dialogue: A short review. find
  36. Genta Indra Winata, Onno Kampman, Yang Yang, Anik Dey, and Pascale Fung. 2017. Nora the empathetic psychologist. In INTERSPEECH. find
  37. Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2017. A hierarchical latent variable encoder-decoder model for generating dialogues. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI’17, page 329… find our card
  38. Thomas Wolf, Victor Sanh, Julien Chaumond, and Clement Delangue. 2019. Transfertransfo: A transfer learning approach for neural network based conversational agents. find
  39. S. Shen, C. Welch, R. Mihalcea, and V. Perez-Rosas. 2020. Counseling-style reflection generation using generative pretrained transformers with augmented context. In Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL 2020), Boise, Idaho. find
  40. Jamin Shin, Peng Xu, Andrea Madotto, and Pascale Fung. 2020. Generating empathetic responses by looking ahead the user’s sentiment. In ICASSP 2020 find
  41. Xiuyu Wu and Yunfang Wu. 2019. A simple dual-decoder model for generating response with sentiment. arXiv preprint arXiv:1905.06597. — [arXiv:1905.06597](https://arxiv.org/abs/1905.06597) find
  42. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204– 2213, Melbo… find our card
  43. Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu. 2018. Emotional chatting machine: Emotional conversation generation with internal and external memory. In Thirty-Second AAAI Conference on Artificial Intelligence. find
  44. Xianda Zhou and William Yang Wang. 2018. MojiTalk: Generating emotional responses at scale. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1128–1137, Melbourne, Australia. Association for Computational Linguistics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/majumder-2020-mime-mimicking-emotions-empathetic-response/