Elamur Research
☰
home / papers / 2019 / Dialogue Act Classification with Context-Aware Self-Attention

Dialogue Act Classification with Context-Aware Self-Attention

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/N19-1373.pdf

Summary

This NAACL 2019 paper from Grammarly addresses Dialogue Act (DA) classification, which assigns each utterance in a conversation a communicative-function label such as Question, Statement or Backchannel. Many earlier neural approaches classify utterances one at a time and ignore the surrounding dialogue. The authors instead frame the task as sequence labeling and build a hierarchical model with three parts:

  • Utterance level: a Bi-GRU over word representations that combine GloVe, ELMo and CNN character embeddings.
  • Context-aware self-attention: an extension of the Lin et al. (2017) encoder that also receives the previous conversation-level hidden state, so each utterance vector reflects the dialogue so far.
  • Conversation level: a Bi-GRU topped by a linear-chain CRF that decodes all labels in a conversation jointly.

On SwDA, the model reaches 82.9% accuracy, a new state of the art, approaching the reported human agreement of 84.0%. On MRDA it scores 91.1%, slightly below the best prior system. The authors link the weaker MRDA result to its much longer conversations and noisier label dependencies. Ablations replacing the utterance encoder indicate that better lower-level utterance representations translate into higher conversation-level accuracy.

Key points

  • Datasets: SwDA has 43 classes and about 193k training utterances. MRDA, in its 5-class version, has about 78k. Both use the Lee and Dernoncourt (2016) splits, and both are heavily imbalanced.
  • Main results (averaged over ten training runs): 82.9% on SwDA versus 81.3% for Chen et al. (2018). On MRDA, 91.1% versus Chen et al.'s 91.7%. The TF-IDF GloVe baseline scores 66.5% on SwDA and 78.7% on MRDA.
  • Encoder ablation (SwDA / MRDA): Bi-RNN + Self-attention + Context performs best at 82.9 / 91.1. Plain Bi-RNN + Self-attention reaches 81.8 / 89.2. Pre-trained Skip-Thought Vectors and Paragraph Vectors trail at roughly 72.5 / 82.6–82.8.
  • Difficulty analysis:
    • Conversation length: MRDA conversations average 1493 utterances, compared with 271 in SwDA.
    • Label entropy: MRDA's normalized label entropy is about six times that of SwDA.
    • Correlation with accuracy: previous-label entropy correlates with per-class accuracy at −0.68 on MRDA and +0.22 on SwDA.
  • Novelty: the authors present this as the first use of self-attention for DA classification. Context is used at several levels: character embeddings, conversation-aware attention and CRF decoding.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Speech act theory

Cited by

46 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 42 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n19-1373 · arXiv 1904.02594

@inproceedings{raheja2019dialogue,
  title = {{Dialogue Act Classification with Context-Aware Self-Attention}},
  author = {Raheja, Vipul and Tetreault, Joel},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  year = {2019},
  doi = {10.18653/v1/n19-1373},
  eprint = {1904.02594},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/n19-1373}
}

References

30 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jeremy Ang, Yang Liu, and Elizabeth Shriberg. 2005. Automatic dialog act segmentation and classification in multiparty meetings. In 2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP ’05, pages 1061–1064. find
  2. John Langshaw Austin. 1975. How to do things with words. Oxford university press. find
  3. Zheqian Chen, Rongqin Yang, Zhou Zhao, Deng Cai, and Xiaofei He. 2018. Dialogue act recognition via crf-attentive structured network. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR ’18, pages 225–234. ACM. find
  4. Jason Chiu and Eric Nichols. 2016. Named entity recognition with bidirectional lstm-cnns. Transactions of the Association for Computational Linguistics, 4:357–370. find our card
  5. Yangfeng Ji, Gholamreza Haffari, and Jacob Eisenstein. 2016. A latent variable recurrent neural network for discourse-driven language models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 332–342. Association for Computational Linguistics. find
  6. Quoc V. Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In Proceedings of the 31th International Conference on Machine Learning, ICML 2014, pages 1188–1196. find our card
  7. Ji Young Lee and Franck Dernoncourt. 2016. Sequential short-text classification with recurrent and convolutional neural networks. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 515–520. Association for Computational Linguistics. find our card
  8. Ruizhe Li, Chenghua Lin, Matthew Collinson, Xiao Li, and Guanyi Chen. 2018. A dual-attention hierarchical recurrent neural network for dialogue act classification. CoRR, abs/1810.09154. ↗
  9. Wei Li and Yunfang Wu. 2016. Multi-level gated recurrent neural network for dialog act classification. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1970–1979. find
  10. The COLING 2016 Organizing Committee. find
  11. Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 2017. A structured self-attentive sentence embedding. In International Conference on Learning Representations 2017 (Conference Track). find our card
  12. Dan Jurafsky, Liz Shriberg, and Debra Biasca. 1997. Switchboard SWBD-DAMSL shallow-discourse-function annotation coders manual, draft 13. Technical report, University of Colorado at Boulder Technical Report 97-02. find
  13. Yang Liu, Kun Han, Zhao Tan, and Yun Lei. 2017. Using context information for dialog act classification in dnn framework. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2170–2178. Association for Computational Linguistics. find our card
  14. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent convolutional neural networks for discourse compositionality. In Proceedings of the Workshop on Continuous Vector Space Models and their Compositionality, pages 119–126. Association for Computational Linguistics. find our card
  15. Xuezhe Ma and Eduard Hovy. 2016. End-to-end sequence labeling via bi-directional lstm-cnns-crf. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1064–1074. Association for Computational Linguistics. find our card
  16. Hamed Khanpour, Nishitha Guntakandla, and Rodney Nielsen. 2016. Dialogue act classification in domain-independent conversations using a deep recurrent neural network. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2012– 2021. find our card
  17. The COLING 2016 Organizing Committee. find
  18. Daniel Ortega and Ngoc Thang Vu. 2017. Neural-based context representation learning for dialog act classification. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 247–252. Association for Computational Linguistics. find
  19. Ryan Kiros, Yukun Zhu, Ruslan R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Skip-thought vectors. In Advances in Neural Information Processing Systems 28, pages 3294–3302. find
  20. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. Association for Computational Linguistics. find
  21. Harshit Kumar, Arvind Agarwal, Riddhiman Dasgupta, and Sachindra Joshi. 2018. Dialogue act sequence labeling using hierarchical encoder with CRF. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), pages 3440– 3447. find our card
  22. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227– 2237. Association for Computational Linguistics. find
  23. Sujith Ravi and Zornitsa Kozareva. 2018. Self-governing neural networks for on-device short text classification. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 887–893. Association for Computational Linguistics. find
  24. John R Searle. 1969. Speech acts: An essay in the philosophy of language, volume 626. Cambridge university press. find
  25. Sheng-syun Shen and Hung-yi Lee. 2016. Neural attention models for sequence classification: Analysis and application to key term extraction and dialogue act detection. In INTERSPEECH 2016, 17th Annual Conference of the International Speech Communication Association, pages 2716–2720. find
  26. Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang, Shirui Pan, and Chengqi Zhang. 2018a. Disan: Directional self-attention network for rnn/cnn-free language understanding. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), pages 5446–5455. find
  27. Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang, and Chengqi Zhang. 2018b. Bi-directional block self-attention for fast and memory-efficient sequence modeling. In International Conference on Learning Representations. find
  28. Yao Wan, Wenqiang Yan, Jianwei Gao, Zhou Zhao, Jian Wu, and Philip S. Yu. 2018. Improved dynamic memory network for dialogue act classification with adversarial training. CoRR, abs/1811.05021. ↗
  29. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1480–1489. Association for Computational Linguistics. find our card
  30. Matthias Zimmermann. 2009. Joint segmentation and classification of dialog acts using conditional random fields. In INTERSPEECH 2009, 10th Annual Conference of the International Speech Communication Association, pages 864–867. find