Elamur Research⌕
☰
home / papers / 2019 / Knowledge-Enriched Transformer for Emotion Detection in Textual Conversations

Knowledge-Enriched Transformer for Emotion Detection in Textual Conversations

Summary

The paper addresses emotion recognition for individual utterances in text dialogues. This is hard because speakers often express feelings implicitly, so a reader must rely on earlier turns and on world knowledge. The authors propose KET (Knowledge-Enriched Transformer), which uses a Transformer in a new way for this task:

  • Context modelling: preceding utterances go into the encoder, which uses hierarchical self-attention. The target utterance goes into the decoder, which uses context-response cross-attention.
  • Knowledge enrichment: each non-stopword token is enriched with neighbouring concepts retrieved from ConceptNet. A dynamic, context-aware affective graph attention weighs these concepts, using NRC VAD scores to balance how related a concept is against how emotionally charged it is.

KET was evaluated on five conversational datasets from different domains: EC, DailyDialog, MELD, EmoryNLP and IEMOCAP. It beat the state-of-the-art models in F1 on most of them, and ablations confirm that both context and external knowledge help. Since ConceptNet is multilingual and comparable emotion lexicons exist for other languages, the approach could be adapted beyond English, Ukrainian included.

Key points

  • Knowledge sources: ConceptNet (English part: about 5.9M assertions, 3.1M concepts, 38 relations, with confidence scores) and NRC VAD (about 20K words scored for valence, arousal and dominance in [0, 1]).
  • Ablation (validation F1): removing context lowers IEMOCAP from 0.5389 to 0.4976 and DailyDialog from 0.5544 to 0.5282. Removing knowledge also lowers scores on every dataset, for example IEMOCAP to 0.5217.
  • Relatedness-affectiveness trade-off: the mixing weight λ matters. Intermediate values (0.3 or 0.7) beat either extreme (0 or 1) on every dataset. IEMOCAP's best intermediate gain is the smallest, while relying fully on affectiveness (λ=1) gives its lowest score.
  • Context length: adding context helps, but returns diminish, and longer histories sometimes hurt. Enlarging ConceptNet improves results until they plateau.
  • Error analysis: disgust and fear are hardest to detect, likely because they have similar VAD profiles and few training examples. Fear reaches an F1 of only 0.0667 on MELD.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

81 papers of our library cite it (of the 140884 whose bibliographies we read); 4 of them have a page here, 77 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d19-1016

@inproceedings{zhong2019knowledge,
  title = {{Knowledge-Enriched Transformer for Emotion Detection in Textual Conversations}},
  author = {Zhong, Peixiang and Wang, Di and Miao, Chunyan},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  year = {2019},
  doi = {10.18653/v1/d19-1016},
  url = {https://doi.org/10.18653/v1/d19-1016}
}

References

67 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. find
  2. Muhammad Abdul-Mageed and Lyle Ungar. 2017. Emonet: Fine-grained emotion detection with gated recurrent neural networks. In ACL, volume 1, pages 718–728. find our card
  3. Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2019. Wizard of wikipedia: Knowledge-powered conversational agents. In ICLR. find
  4. Nabiha Asghar, Pascal Poupart, Jesse Hoey, Xin Jiang, and Lili Mou. 2018. Affective neural response generation. In ECIR, pages 154–166. Springer. find
  5. Paul Ekman. 1992. An argument for basic emotions. Cognition & emotion, 6(3-4):169–200. find our card
  6. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. find our card
  7. Christiane Fellbaum. 2012. Wordnet. The Encyclopedia of Applied Linguistics. find
  8. Anil Bandhakavi, Nirmalie Wiratunga, Stewart Massie, and Deepak Padmanabhan. 2017. Lexicon generation for emotion detection from text. IEEE Intelligent Systems, 32(1):102–108. find
  9. Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, Bill Dolan, Jianfeng Gao, Wen-tau Yih, and Michel Galley. 2018. A knowledge-grounded neural conversation model. In AAAI. find
  10. Yoshua Bengio, Patrice Simard, Paolo Frasconi, et al. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 5(2):157–166. find
  11. Sangdo Han, Jeesoo Bang, Seonghan Ryu, and Gary Geunbae Lee. 2015. Exploiting knowledge base to generate responses for natural language dialog listening agents. In Proceedings of the 16th SIGDIAL, pages 129–133. find
  12. Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N Chang, Sungbok Lee, and Shrikanth S Narayanan. 2008. IEMOCAP: Interactive emotional dyadic motion capture database. Language Resources and Evaluation, 42(4):335. find
  13. Yanchao Hao, Yuanzhe Zhang, Kang Liu, Shizhu He, Zhanyi Liu, Hua Wu, and Jun Zhao. 2017. An end-to-end model for question answering over knowledge base with cross-attention combining global knowledge. In ACL, pages 221–231. find
  14. Ankush Chatterjee, Umang Gupta, Manoj Kumar Chinnakotla, Radhakrishnan Srikanth, Michel Galley, and Puneet Agrawal. 2019. Understanding emotions in text using deep learning and big data. Computers in Human Behavior, 93:309 – 317. find
  15. Devamanyu Hazarika, Soujanya Poria, Rada Mihalcea, Erik Cambria, and Roger Zimmermann. 2018a. Icon: Interactive conversational memory network for multimodal emotion detection. In EMNLP, pages 2594–2604. find our card
  16. Jianpeng Cheng, Li Dong, and Mirella Lapata. 2016. Long short-term memory-networks for machine reading. In EMNLP, pages 551–561. find our card
  17. Devamanyu Hazarika, Soujanya Poria, Amir Zadeh, Erik Cambria, Louis-Philippe Morency, and Roger Zimmermann. 2018b. Conversational memory network for emotion recognition in dyadic dialogue videos. In NAACL, volume 1, pages 2122–2132. find our card
  18. Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder–decoder for statistical machine translation. In EMNLP, pages 1724–1734. find
  19. Junqing He, Bing Wang, Mingming Fu, Tianqi Yang, and Xuemin Zhao. 2019. Hierarchical attention and knowledge matching networks with information enhancement for end-to-end task-oriented dialog systems. IEEE Access, 7:18871–18883. find
  20. Zihang Dai, Zhilin Yang, Yiming Yang, William W Cohen, Jaime Carbonell, Quoc V Le, and Ruslan Salakhutdinov. 2019. Transformer-xl: Attentive language models beyond a fixed-length context. arXiv preprint arXiv:1901.02860. find
  21. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735–1780. find
  22. Yajie Hu, Xiaoou Chen, and Deshun Yang. 2009. Lyric-based song emotion detection with affective lexicon and fuzzy clustering method. In ISMIR, pages 123–128. find
  23. Laurence Devillers, Ioana Vasilescu, and Lori Lamel. 2002. Annotation and detection of emotion in a task-oriented human-human dialog corpus. In Proceedings of ISLE Workshop. find
  24. Eva Hudlicka. 2011. Guidelines for designing computational models of emotions. International Journal of Synthetic Emotions, 2(1):26–79. find
  25. Laurence Devillers and Laurence Vidrascu. 2006. Real-life emotions detection with lexical and paralinguistic cues on human-human call center dialogs. In Ninth International Conference on Spoken Language Processing. find
  26. Hamed Khanpour and Cornelia Caragea. 2018. Fine-grained emotion detection in health-related online posts. In EMNLP, pages 1160–1166. find
  27. Prasanna Parthasarathi and Joelle Pineau. 2018. Extending neural generative conversational model using external knowledge sources. In EMNLP, pages 690–695. find
  28. Chloé Kiddon, Luke Zettlemoyer, and Yejin Choi. 2016. Globally coherent text generation with neural checklist models. In EMNLP, pages 329–339. find our card
  29. Yoon Kim. 2014. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882. find our card
  30. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In EMNLP, pages 1532–1543. find
  31. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. find
  32. Soujanya Poria, Erik Cambria, Devamanyu Hazarika, Navonil Majumder, Amir Zadeh, and Louis-Philippe Morency. 2017. Context-dependent sentiment analysis in user-generated videos. In ACL, volume 1, pages 873–883. find our card
  33. Abhishek Kumar, Daisuke Kawahara, and Sadao Kurohashi. 2018. Knowledge-enriched two-layered attention network for sentiment analysis. In NAACL, volume 2, pages 253–258. find
  34. Chul Min Lee and Shrikanth S Narayanan. 2005. Toward detecting emotions in spoken dialogs. IEEE Transactions on Speech and Audio Processing, 13(2):293–303. find
  35. Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. 2018. Meld: A multimodal multi-party dataset for emotion recognition in conversations. arXiv preprint arXiv:1810.02508. find our card
  36. Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. Dailydialog: A manually labelled multi-turn dialogue dataset. In IJC-NLP, volume 1, pages 986–995. find our card
  37. Soujanya Poria, Navonil Majumder, Rada Mihalcea, and Eduard Hovy. 2019. Emotion recognition in conversation: Research challenges, datasets, and recent advances. arXiv preprint arXiv:1905.02947. — [arXiv:1905.02947](https://arxiv.org/abs/1905.02947) find
  38. Shuman Liu, Hongshen Chen, Zhaochun Ren, Yang Feng, Qun Liu, and Dawei Yin. 2018. Knowledge diffusion for neural dialogue generation. In ACL, pages 1489–1498. find
  39. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. find
  40. Yi Luan Mirella Lapata Rik Koncel-Kedziorski, Dhanush Bekal and Hannaneh Hajishirzi. 2019. Text Generation from Knowledge Graphs with Graph Transformers. In NAACL. find our card
  41. Andrea Madotto, Chien-Sheng Wu, and Pascale Fung. 2018. Mem2seq: Effectively incorporating knowledge bases into end-to-end task-oriented dialog systems. In ACL, volume 1, pages 1468–1478. find our card
  42. Armin Seyeditabari, Narges Tabari, and Wlodek Zadrozny. 2018. Emotion detection in text: a review. arXiv preprint arXiv:1806.00674. — [arXiv:1806.00674](https://arxiv.org/abs/1806.00674) find
  43. Navonil Majumder, Soujanya Poria, Devamanyu Hazarika, Rada Mihalcea, Alexander Gelbukh, and Erik Cambria. 2019. Dialoguernn: An attentive rnn for emotion detection in conversations. In AAAI. find
  44. Dinghan Shen, Guoyin Wang, Wenlin Wang, Martin Renqiang Min, Qinliang Su, Yizhe Zhang, Chunyuan Li, Ricardo Henao, and Lawrence Carin. 2018. Baseline needs more love: On simple word-embedding-based models and associated pooling mechanisms. In ACL, pages 440–450. find our card
  45. Albert Mehrabian. 1996. Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament. Current Psychology, 14(4):261–292. find
  46. Todor Mihaylov and Anette Frank. 2018. Knowledgeable reader: Enhancing cloze-style reading comprehension with external commonsense knowledge. In ACL, pages 821–832. find
  47. Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. In AAAI. find
  48. Nikita Moghe, Siddhartha Arora, Suman Banerjee, and Mitesh M Khapra. 2018. Towards exploiting background knowledge for building conversation systems. In EMNLP, pages 2322–2332. find our card
  49. Shang-Yu Su, Pei-Chieh Yuan, and Yun-Nung Chen. 2018. How time matters: Learning time-decay attention for contextual spoken language understanding in dialogues. In NAACL, volume 1, pages 2133– 2142. find
  50. Saif Mohammad. 2018a. Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 english words. In ACL, pages 174–184. find our card
  51. Saif M. Mohammad. 2018b. Word affect intensities. In LREC. find our card
  52. Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Kathryn Mazaitis, Ruslan Salakhutdinov, and William Cohen. 2018. Open domain question answering using early fusion of knowledge bases and text. In EMNLP, pages 4231–4242. find
  53. Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up?: sentiment classification using machine learning techniques. In EMNLP, pages 79– 86. Association for Computational Linguistics. find our card
  54. Maite Taboada, Julian Brooke, Milan Tofiloski, Kimberly Voll, and Manfred Stede. 2011. Lexicon-based methods for sentiment analysis. Computational Linguistics, 37(2):267–307. find our card
  55. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS, pages 5998–6008. find
  56. Petar Velikovi, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Li, and Yoshua Bengio. 2018. Graph attention networks. In ICLR. find
  57. Sida Wang and Christopher D Manning. 2012. Baselines and bigrams: Simple, good sentiment and topic classification. In ACL, pages 90–94. Association for Computational Linguistics. find our card
  58. Chien-Sheng Wu, Richard Socher, and Caiming Xiong. 2019. Global-to-local memory pointer networks for task-oriented dialogue. In ICLR. find
  59. Tom Young, Erik Cambria, Iti Chaturvedi, Hao Zhou, Subham Biswas, and Minlie Huang. 2018. Augmenting end-to-end dialogue systems with commonsense knowledge. In AAAI. find
  60. Sayyed M Zahiri and Jinho D Choi. 2018. Emotion detection on tv show transcripts with sequence-based convolutional neural networks. In Workshops at AAAI. find
  61. Jiacheng Zhang, Huanbo Luan, Maosong Sun, Feifei Zhai, Jingfang Xu, Min Zhang, and Yang Liu. 2018a. Improving the transformer translation model with document-level context. In EMNLP, pages 533–542. find our card
  62. Yuxiang Zhang, Jiamei Fu, Dongyu She, Ying Zhang, Senzhang Wang, and Jufeng Yang. 2018b. Text emotion distribution learning via multi-task convolutional neural network. In IJCAI, pages 4595– 4601. find
  63. Peixiang Zhong and Chunyan Miao. 2019. ntuer at SemEval-2019 task 3: Emotion classification with word and sentence representations in RCNN. In SemEval, pages 282–286. find
  64. Peixiang Zhong, Di Wang, and Chunyan Miao. 2019. An affect-rich neural conversational model with biased attention and weighted cross-entropy loss. In AAAI, pages 7492–7500. find
  65. Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu. 2018a. Emotional chatting machine: Emotional conversation generation with internal and external memory. In AAAI. find
  66. Hao Zhou, Tom Young, Minlie Huang, Haizhou Zhao, Jingfang Xu, and Xiaoyan Zhu. 2018b. Commonsense knowledge aware conversation generation with graph attention. In IJCAI, pages 4623–4629. find
  67. Xiangyang Zhou, Lu Li, Daxiang Dong, Yi Liu, Ying Chen, Wayne Xin Zhao, Dianhai Yu, and Hua Wu. 2018c. Multi-turn response selection for chatbots with deep attention matching network. In ACL, volume 1, pages 1118–1127. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhong-2019-knowledge-enriched-transformer-emotion-detection/