Elamur Research
☰
home / papers / 2019 / DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation

DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/D19-1015.pdf

Summary

DialogueGCN targets emotion recognition in conversation (ERC). Earlier RNN-based approaches such as DialogueRNN struggle to carry context across distant utterances and do not explicitly encode who said what or where an utterance sits relative to the target. The authors represent each dialogue as a directed graph. Utterances are nodes, and typed edges link each utterance to neighbours within a past and future window. Edge types depend on the speaker pair and on direction in time, which captures both inter-speaker influence and a speaker's own emotional inertia. Utterance features come from a text CNN over GloVe embeddings. A bidirectional GRU adds sequential context, and a relational graph convolutional network adds speaker-level context. The two representations are combined for classification. On IEMOCAP, AVEC and MELD, the model beats strong baselines, including the previous state of the art, DialogueRNN. Ablations show that both the speaker-aware edges and the graph encoder matter. The work shows that graph-based context modelling is a practical alternative to purely recurrent ERC models.

Key points

  • Text features: Utterances are encoded with a CNN over 300-d GloVe 840B vectors, using filter sizes 3, 4 and 5 and a 100-d output. Only the text modality is used, although all three datasets are multimodal.
  • Graph design: The window covers 10 past and 10 future utterances. Edge weights come from attention. A conversation with M speakers has 2M² relation types. Removing the speaker and temporal edge distinctions lowers IEMOCAP F1 from 64.18 to 60.11.
  • Results vs. DialogueRNN:
    • IEMOCAP weighted F1 is 64.18.
    • MELD F1 is 58.10, up from 57.03.
    • AVEC MAE is lower on all four dimensions (e.g., valence 0.157 vs. 0.168).
  • Encoder ablation: IEMOCAP F1 is 55.30 with the sequential encoder alone, 56.71 with the speaker-level encoder alone, and 36.75 with neither.
  • Window size and errors: Shrinking the window to (8,8), (4,4) and (0,0) lowers F1 to 62.48, 59.41 and 55.80. Common errors include frustrated vs. angry and excited vs. happy. Code is released at github.com/SenticNet/conv-emotion.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

133 papers of our library cite it (of the 140673 whose bibliographies we read).

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d19-1015

@inproceedings{ghosal2019dialoguegcn,
  title = {{DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation}},
  author = {Ghosal, Deepanway and Majumder, Navonil and Poria, Soujanya and Chhaya, Niyati and Gelbukh, Alexander},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  year = {2019},
  doi = {10.18653/v1/d19-1015},
  url = {https://doi.org/10.18653/v1/d19-1015}
}

References

35 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Zimmermann. 2018b. Conversational Memory Network for Emotion Recognition in Dyadic Dialogue Videos. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2122–2132, New Orleans,… find
  2. James Bradbury, Stephen Merity, Caiming Xiong, and Richard Socher. 2017. Quasi-Recurrent Neural Networks. In International Conference on Learning Representations (ICLR 2017). find
  3. Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. 2013. Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203. — [arXiv:1312.6203](https://arxiv.org/abs/1312.6203) find
  4. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. find
  5. Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N Chang, Sungbok Lee, and Shrikanth S Narayanan. 2008. IEMOCAP: Interactive emotional dyadic motion capture database. Language resources and evaluation, 42(4):335–359. find
  6. Chenyang Huang, Amine Trabelsi, and Osmar R Zaı̈ane. 2019. Ana at semeval-2019 task 3: Contextual emotion detection in conversations through hierarchical lstms and bert. arXiv preprint arXiv:1904.00132. find
  7. Sidney K. D’Mello, Scotty Craig, Jeremiah Sullins, and Arthur Graesser. 2006. Predicting affective states expressed through an emote-aloud procedure from autotutor’s mixed-initiative dialogue. I. J. Artificial Intelligence in Education, 16:3–28. find
  8. Minghai Chen, Sen Wang, Paul Pu Liang, Tadas Bal-trušaitis, Amir Zadeh, and Louis-Philippe Morency. 2017. Multimodal sentiment analysis with word-level fusion and reinforcement learning. In Proceedings of the 19th ACM International Conference on Multimodal Interaction, pages 163–171. ACM. find
  9. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In EMNLP 2014, pages 1746–1751. find our card
  10. Sheng-Yeh Chen, Chao-Chun Hsu, Chuan-Chun Kuo, Lun-Wei Ku, et al. 2018. Emotionlines: An emotion corpus of multi-party conversations. arXiv preprint arXiv:1802.08379. find
  11. Diederik P. Kingma and Jimmy Ba. 2014. A Method for Stochastic Optimization. abs/1412.6980. find
  12. Junyoung Chung, Çaglar Gülçehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. CoRR, abs/1412.3555. find
  13. Adam: CoRR, Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907. find our card
  14. Niko Colneriĉ and Janez Demsar. 2018. Emotion recognition on twitter: comparative study and training a unison model. IEEE Transactions on Affective Computing. find
  15. Bernhard Kratzwald, Suzana Ilic, Mathias Kraus, Stefan Feuerriegel, and Helmut Prendinger. 2018. Decision support with text-based emotion recognition: Deep learning for affective computing. arXiv preprint arXiv:1803.06397. — [arXiv:1803.06397](https://arxiv.org/abs/1803.06397) find
  16. Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 3844–3852. Cur… find our card
  17. Navonil Majumder, Soujanya Poria, Devamanyu Hazarika, Rada Mihalcea, Alexander Gelbukh, and Erik Cambria. 2019. DialogueRNN: An Attentive RNN for Emotion Detection in Conversations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6818–6825. find
  18. Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. 2017. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1263–1272. JMLR. org. find our card
  19. G. McKeown, M. Valstar, R. Cowie, M. Pantic, and M. Schroder. 2012. The SEMAINE Database: Annotated Multimodal Records of Emotionally Colored Conversations between a Person and a Limited Agent. IEEE Transactions on Affective Computing, 3(1):5–17. find
  20. Devamanyu Hazarika, Soujanya Poria, Rada Mihalcea, Erik Cambria, and Roger Zimmermann. 2018a. Icon: Interactive conversational memory network for multimodal emotion detection. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2594–2604. find
  21. Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML-10), pages 807–814. find
  22. Costanza Navarretta, K Choukri, T Declerck, S Goggi, M Grobelnik, and B Maegaard. 2016. Mirroring facial expressions and emotions in dyadic conversations. In LREC. find
  23. Devamanyu Hazarika, Soujanya Poria, Amir Zadeh, Erik Cambria, Louis-Philippe Morency, and Roger Amir Zadeh, Paul Pu Liang, Soujanya Poria, Prateek Vij, Erik Cambria, and Louis-Philippe Morency. 2018b. Multi-attention recurrent network for human communication comprehension. In Proceedings of the AAAI… find
  24. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532– 1543. find
  25. Soujanya Poria, Erik Cambria, Devamanyu Hazarika, Navonil Majumder, Amir Zadeh, and Louis-Philippe Morency. 2017. Context-Dependent Sentiment Analysis in User-Generated Videos. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8… find
  26. Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu. 2018. Emotional chatting machine: Emotional conversation generation with internal and external memory. In Thirty-Second AAAI Conference on Artificial Intelligence. find
  27. Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. 2019a. MELD: A multimodal multi-party dataset for emotion recognition in conversations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 527– 536, Flor… find our card
  28. Soujanya Poria, Navonil Majumder, Rada Mihalcea, and Eduard Hovy. 2019b. Emotion recognition in conversation: Research challenges, datasets, and recent advances. IEEE Access, 7:100943–100953. find
  29. Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2008. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61–80. find
  30. Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In European Semantic Web Conference, pages 593–607. Springer. find
  31. Björn Schuller, Michel Valster, Florian Eyben, Roddy Cowie, and Maja Pantic. 2012. find
  32. AVEC 2012: The Continuous Audio/Visual Emotion Challenge. In Proceedings of the 14th ACM International Conference on Multimodal Interaction, ICMI ’12, pages 449–456, New York, NY, USA. ACM. find
  33. Carlo Strapparava and Rada Mihalcea. 2010. Annotating and identifying emotions in text. In Intelligent Information Access, pages 21–38. Springer. find
  34. Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, and Rob Fergus. 2015. End-to-end Memory Networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2, NIPS’15, pages 2440–2448, Cambridge, MA, USA. MIT Press. find
  35. Amir Zadeh, Paul Pu Liang, Navonil Mazumder, Soujanya Poria, Erik Cambria, and Louis-Philippe Morency. 2018a. Memory Fusion Network for Multi-view Sequential Learning. In AAAI Conference on Artificial Intelligence, pages 5634–5641. find