Elamur Research⌕
☰
home / papers / 2023 / Who is Speaking? Speaker-Aware Multiparty Dialogue Act Classification

Who is Speaking? Speaker-Aware Multiparty Dialogue Act Classification

Paper · 2023 · Ayesha Qamar, Adarsh Pyarelal and Ruihong Huang

Summary

The paper targets dialogue act (DA) classification in conversations with more than two participants. Earlier speaker-aware methods either assume a fixed, small set of speakers or reduce speaker information to a same/different flag, which effectively turns a meeting into a two-person exchange. The authors build on a RoBERTa plus bidirectional GRU utterance encoder. They prepend a token marking whether the speaker stayed the same or changed. They then add a small graph neural network in which each speaker in a chunk becomes a node linked only to that speaker's own utterances. The learned speaker vector is concatenated with each utterance vector before classification. On the multiparty MRDA corpus, this gives about one macro F1 point over the turn-aware baseline. It also beats denser graphs that add utterance-to-utterance edges. On the dyadic SwDA corpus, no gain appears. The work shows that explicitly modelling speakers, with a cheap sparse graph, helps most when many people talk.

Key points

  • Data: the ICSI MRDA meeting corpus, which is multiparty with up to 9 speakers and about 6.13 on average per full dialogue, and SwDA, which is dyadic. The code is released on GitHub.
  • Encoder: RoBERTa-base, with only the final layer fine-tuned, followed by a bidirectional GRU for context. Turn tokens ⟨same⟩/⟨switch⟩ add speaker-change information without new parameters.
  • Graph design:
    • The graph has |D| utterance nodes and |P| randomly initialised speaker nodes.
    • A single undirected utterance–speaker relation is used, with two GNN layers.
    • The best graph has 2|D| edges, while window-based utterance links would add 2|D|w more.
  • Results:
    • The graph model reaches 40.06 macro F1 on MRDA with chunks of 128.
    • Even chunks of 16, with about 3.3 speakers, still beat the turn-aware baseline that used 128-utterance context.
  • Concatenating speaker nodes worked better than concatenating graph utterance nodes or both (40.06 vs 39.46/39.47).

Interesting ideas

  • I1. Represent each participant as its own node fed only by that participant's utterances, so the model learns a role such as leader or backchanneler that works for any number of speakers.
  • I2. Mark speaker continuity with a special same/switch token instead of adding parameters, as a cheap first signal of turn-taking.
  • I3. Sparse, targeted links beat dense context graphs: extra utterance-to-utterance edges added noise and memory cost rather than accuracy.
  • I4. Using a dyadic corpus where the method shows no gain acts as a natural control that confirms the benefit comes specifically from multiparty structure.
  • I5. Analysing chunk size shows that short local windows keep most of the benefit, so long contexts are not always needed.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): crate prag could keep a per-speaker state built from that speaker's own lines and use it as evidence when choosing speech acts like acknowledgement or understanding check in multi-voice dialogue in fables.
  • I2 (tokens): the tokenizer could emit an explicit turn-change marker for quoted dialogue, giving later stages a transparent feature for whether a new voice has started.
  • I3 (world): when linking entities and events in the world of a text, prefer few typed, justified links, such as an entity connected to its own mentions, over windowed all-pairs links, which keeps the graph explainable and small.
  • I4 (learn): evaluations of speaker or coreference features could include a matched set without the phenomenon as a negative control, verifying that improvements come from the intended mechanism.
  • I5 (answer): QA readers over tales could test whether a local window of recent turns suffices for speaker-dependent questions before paying for whole-text reasoning.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of ArizonaTucson · United States38 works · 10,603 citations🎓Texas A&M UniversityCollege Station · United States26 works · 1,871 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Dialogue systems · 624 worksSpeech act theory · 229 works

Cite

Cite as Ayesha Qamar, Adarsh Pyarelal and Ruihong Huang. 2023. Who is Speaking? Speaker-Aware Multiparty Dialogue Act Classification. Findings of the Association for Computational Linguistics: EMNLP 2023.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.findings-emnlp.678

@inproceedings{qamar2023who,
  title = {{Who is Speaking? Speaker-Aware Multiparty Dialogue Act Classification}},
  author = {Ayesha Qamar, Adarsh Pyarelal and Ruihong Huang},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.findings-emnlp.678},
  url = {https://doi.org/10.18653/v1/2023.findings-emnlp.678}
}

References

51 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. John Aberdeen and Lisa Ferro. 2003. Dialogue patterns and misunderstandings. In Proceedings of the ISCA Workshop on Error Handling in Spoken Dialogue Systems, pages 17–23. find
  2. Jeremy Ang, Yang Liu, and Elizabeth Shriberg. 2005. Automatic dialog act segmentation and classification in multiparty meetings. In Proceedings.(ICASSP’05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005., volume 1, pages I– 1061. IEEE. find
  3. Mohamed Bekkar, Hassiba Kheliouane Djemaa, and Taklit Akrouf Alitouche. 2013. Evaluation measures for models assessment over imbalanced data sets. J Inf Eng Appl, 3(10). find
  4. Chandrakant Bothe, Cornelius Weber, Sven Magg, and Stefan Wermter. 2018. A context-based approach for dialogue act recognition using simple recurrent neural networks. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).
  5. Michael Buckland and Fredric Gey. 1994. The relationship between recall and precision. Journal of the American society for information science, 45(1):12– 19. ↗
  6. Harry Bunt. 2006. Dimensions in dialogue act annotation. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06).
  7. Dan Busbridge, Dane Sherburn, Pietro Cavallo, and Nils Y Hammerla. 2019. Relational graph attention networks. arXiv preprint arXiv:1904.05811. ↗
  8. Nitesh V Chawla. 2009. Data mining for imbalanced datasets: An overview. Data mining and knowledge discovery handbook, pages 875–886. ↗
  9. Zheqian Chen, Rongqin Yang, Zhou Zhao, Deng Cai, and Xiaofei He. 2018. Dialogue act recognition via crf-attentive structured network. In The 41st international acm sigir conference on research & development in information retrieval, pages 225–234. ↗
  10. Ta-Chung Chi, Po-Chun Chen, Shang-Yu Su, and Yun-Nung Chen. 2017. Speaker role contextual modeling for language understanding and dialogue policy learning. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 163–168, Taipei, Taiwan. Asian Federation of Natural Language Processing. ↗
  11. Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259. ↗
  12. Pierre Colombo, Emile Chapuis, Matteo Manica, Emmanuel Vignon, Giovanna Varni, and Chloe Clavel. 2020. Guiding attention in sequence-to-sequence models for dialogue act prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 7594–7601. ↗
  13. Rajdip Dhillon, Sonali Bhagat, Hannah Carvey, and Elizabeth Shriberg. 2004. Meeting recorder project: Dialog act labeling guide. Technical report, INTER-NATIONAL COMPUTER SCIENCE INST BERKE-LEY CA. ↗
  14. Matthias Fey and Jan Eric Lenssen. 2019. Fast graph representation learning with pytorch geometric. arXiv preprint arXiv:1903.02428. ↗
  15. Deepanway Ghosal, Navonil Majumder, Soujanya Poria, Niyati Chhaya, and Alexander Gelbukh. 2019. Dialoguegcn: A graph convolutional neural network for emotion recognition in conversation. arXiv preprint arXiv:1908.11540. ↗
  16. Chih-Wen Goo and Yun-Nung Chen. 2018a. Abstractive dialogue summarization with sentence-gated modeling optimized by dialogue acts. In 2018 IEEE Spoken Language Technology Workshop (SLT), pages 735–742. IEEE. find
  17. Chih-Wen Goo and Yun-Nung Chen. 2018b. Abstractive dialogue summarization with sentence-gated modeling optimized by dialogue acts. In 2018 IEEE Spoken Language Technology Workshop (SLT), pages 735–742. find
  18. Qiong Gu, Li Zhu, and Zhihua Cai. 2009. Evaluation measures of the classification performance of imbalanced data sets. volume 51, pages 461–471. find
  19. Zihao He, Leili Tavabi, Kristina Lerman, and Mohammad Soleymani. 2021. Speaker turn modeling for dialogue act classification. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2150–2157, Punta Cana, Dominican Republic. Association for Computational Linguistics. ↗
  20. Dou Hu, Lingwei Wei, and Xiaoyong Huai. 2021. Dia-logueCRN: Contextual reasoning networks for emotion recognition in conversations. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7042–7052, Online. Association for Computational Linguistics. find
  21. Taichi Ishiwatari, Yuki Yasuda, Taro Miyazaki, and Jun Goto. 2020. Relation-aware graph attention networks with relational position encodings for emotion recognition in conversations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7360–7370, Online. Association for Computational Linguistics. find
  22. Justin M Johnson and Taghi M Khoshgoftaar. 2019. Survey on deep learning with class imbalance. Journal of Big Data, 6(1):1–54. ↗
  23. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. ↗
  24. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations. find
  25. Navonil Majumder, Soujanya Poria, Devamanyu Haz-arika, Rada Mihalcea, Alexander Gelbukh, and Erik Cambria. 2019. Dialoguernn: An attentive rnn for emotion detection in conversations. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 6818–6825. ↗
  26. Dan Jurafsky. 1997. Switchboard swbd-damsl shallow-discourse-function annotation coders manual. Institute of Cognitive Science Technical Report. find
  27. Tatsuro Oya and Giuseppe Carenini. 2014. Extractive summarization and dialogue act modeling on email threads: An integrated probabilistic approach. In Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pages 133–140.
  28. Hamed Khanpour, Nishitha Guntakandla, and Rodney Nielsen. 2016. Dialogue act classification in domain-independent conversations using a deep recurrent neural network. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2012–2021, Osaka, Japan.
  29. The COLING 2016 Organizing Committee. find
  30. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32. ↗
  31. Sotiris Kotsiantis, Dimitris Kanellopoulos, Panayiotis Pintelas, et al. 2006. Handling imbalanced datasets: A review. GESTS international transactions on computer science and engineering, 30(1):25–36. find
  32. Harshit Kumar, Arvind Agarwal, Riddhiman Dasgupta, and Sachindra Joshi. 2018. Dialogue act sequence labeling using hierarchical encoder with crf. In Proceedings of the aaai conference on artificial intelligence, volume 32. ↗
  33. Bongseok Lee and Yong Suk Choi. 2021. Graph based network with contextualized representations of turns in dialogue. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 443–455, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. find
  34. Ji Young Lee and Franck Dernoncourt. 2016. Sequential short-text classification with recurrent and convolutional neural networks. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 515–520, San Diego, California. Association for Computational Linguistics.
  35. Yunlong Liang, Fandong Meng, Ying Zhang, Yufeng Chen, Jinan Xu, and Jie Zhou. 2021. Infusing multi-source knowledge with heterogeneous graph neural network for emotional conversation generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 13343–13352. find
  36. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543. ↗
  37. Vipul Raheja and Joel Tetreault. 2019. Dialogue Act Classification with Context-Aware Self-Attention. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3727–3733, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  38. Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2008. The graph neural network model. IEEE transactions on neural networks, 20(1):61–80. ↗
  39. John Rogers Searle. 1969. Speech acts: An essay in the philosophy of language, volume 626. Cambridge university press. find
  40. Guokan Shang, Antoine Tixier, Michalis Vazirgiannis, and Jean-Pierre Lorré. 2020. Speaker-change aware CRF for dialogue act classification. In Proceedings of the 28th International Conference on Computational Linguistics, pages 450–464, Barcelona, Spain (Online). International Committee on Computational Linguistics. ↗
  41. Weizhou Shen, Junqing Chen, Xiaojun Quan, and Zhix-ian Xie. 2021a. Dialogxl: All-in-one xlnet for multi-party conversation emotion recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 13789–13797. find
  42. Weizhou Shen, Siyue Wu, Yunyi Yang, and Xiaojun Quan. 2021b. Directed acyclic graph network for conversational emotion recognition. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1551–1560, Online. Association for Computational Linguistics. ↗
  43. Dongming Sheng, Dong Wang, Ying Shen, Haitao Zheng, and Haozhuang Liu. 2020. Summarize before aggregate: A global-to-local heterogeneous graph inference network for conversational emotion recognition. In Proceedings of the 28th International Conference on Computational Linguistics, pages 4153–4163, Barcelona, Spain (Online). International Committee on Computational Linguistics. find
  44. Elizabeth Shriberg, Raj Dhillon, Sonali Bhagat, Jeremy Ang, and Hannah Carvey. 2004. The ICSI meeting recorder dialog act (MRDA) corpus. In Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue at HLT-NAACL 2004, pages 97–100, Cambridge, Massachusetts, USA. Association for Computational Linguistics. ↗
  45. Rui Song, Fausto Giunchiglia, Lida Shi, Qiang Shen, and Hao Xu. 2023. Sunet: Speaker-utterance interaction graph neural network for emotion recognition in conversations. Engineering Applications of Artificial Intelligence, 123:106315. ↗
  46. Yang Sun, Nan Yu, and Guohong Fu. 2021. Integrating rich utterance features for emotion recognition in multi-party conversations. In International Conference on Neural Information Processing, pages 51–62. Springer. find
  47. Kai Wang, Junfeng Tian, Rui Wang, Xiaojun Quan, and Jianxing Yu. 2020. Multi-domain dialogue acts and response co-generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7125–7134, Online. Association for Computational Linguistics.
  48. Can Xu, Wei Wu, and Yu Wu. 2018. Towards explainable and controllable open domain dialogue generation with dialogue acts. arXiv preprint arXiv:1807.07255. ↗
  49. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural information processing systems, 32. ↗
  50. Piotr Żelasko, Raghavendra Pappagari, and Najim Dehak. 2021. What helps transformers recognize conversational structure? importance of context, punctuation, and labels in dialog act recognition. Transactions of the Association for Computational Linguistics, 9:1179–1195. ↗
  51. Dong Zhang, Liangqing Wu, Changlong Sun, Shoushan Li, Qiaoming Zhu, and Guodong Zhou. 2019. Modeling both context-and speaker-sensitive dependence for emotion detection in multi-speaker conversations. In IJCAI, pages 5415–5421. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/qamar-2023-who-is-speaking-speaker-aware/