Elamur Research
☰
home / papers / 2016 / Dialogue Act Classification in Domain-Independent Conversations Using a Deep Recurrent Neural Network

Dialogue Act Classification in Domain-Independent Conversations Using a Deep Recurrent Neural Network

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/C16-1189.pdf

Summary

The paper tackles dialogue act (DA) classification in open-domain, spontaneous conversation, where each utterance must be assigned a label that captures its communicative function. Most earlier work either targeted narrow task-oriented domains or relied on handcrafted lexical and prosodic features. The authors encode each utterance as a sequence of word vectors and feed it to a stacked (deep) LSTM. The word vectors are 300-dimensional embeddings pre-trained on a Wikipedia dump, and a pooling layer produces a single utterance representation for softmax classification. They tuned hyperparameters one at a time on a development split of the Switchboard (SwDA) data. Word-embedding settings, dropout, weight decay and network depth mattered most. Without retuning, they applied the same configuration to the five-class MRDA corpus. The model surpassed prior neural and probabilistic approaches on both benchmarks. It reached 80.1% on SwDA and 86.8% on MRDA, approaching the human agreement level reported for SwDA. A practical lesson is that dropout, even the RNN-adapted variant, can hurt LSTM accuracy and should be validated rather than applied by default.

Key points

  • Data: SwDA has 1,155 five-minute dialogues annotated with 42 DA tags, with 19 conversations in the test set. MRDA (ICSI Meeting Recorder, DAMSL-based) has 75 one-hour multi-party meetings and is used in its 5-tag version, with splits following Lee and Dernoncourt (2016).
  • SwDA results: 80.1% accuracy versus 77.0% for DRLM-conditional training (Ji et al., 2016), 73.9% for RCNN, 71.0% for HMM, a 47.3% tf-idf baseline and 84.0% inter-annotator agreement.
  • MRDA results: 86.8% accuracy versus 84.6% for the CNN of Lee and Dernoncourt (2016), achieved without tuning on any MRDA data.
  • Final configuration:
    • 10 stacked LSTM layers; development accuracy rose up to this depth and fell at 15.
    • Decay λ = 0.7 and learning rate η = 1e-3.
    • L2 regularization of 1e-5, last-pooling and tanh activation.
  • Dropout finding: Zaremba-style dropout lowered development accuracy at every tested probability (0.1–0.5), so the final model uses no dropout (73.29% vs. 71.95% at 0.5).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Speech act theory

Cited by

42 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 39 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{khanpour2016dialogue,
  title = {{Dialogue Act Classification in Domain-Independent Conversations Using a Deep Recurrent Neural Network}},
  author = {Khanpour, Hamed and Guntakandla, Nishitha and Nielsen, Rodney},
  booktitle = {Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers},
  year = {2016},
  url = {https://aclanthology.org/C16-1189/}
}

References

43 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jeremy Ang, Yang Liu, and Elizabeth Shriberg. 2005. Automatic dialog act segmentation and classification in multiparty meetings. In ICASSP (1), pages 1061–1064. find
  2. John Langshaw Austin. 1975. How to do things with words. Oxford university press. find
  3. Tao Lei Hrishikesh Joshi Regina Barzilay, Tommi Jaakkola, Katerina Tymoshenko, and Alessandro Moschitti Llu Marquez. 2016. Semi-supervised question retrieval with gated convolutions. Naacl. find
  4. Justin Bayer, Christian Osendorfer, Daniela Korhammer, Nutan Chen, Sebastian Urban, and Patrick van der Smagt. 2013. On fast dropout and its applicability to recurrent networks. arXiv preprint arXiv:1311.0701. ↗
  5. Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks, 5(2):157–166. find
  6. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. journal of machine learning research, 3(Feb):1137–1155. find
  7. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. The Journal of Machine Learning Research, 12:2493–2537. find our card
  8. Rajdip Dhillon, Sonali Bhagat, Hannah Carvey, and Elizabeth Shriberg. 2004. Meeting recorder project: Dialog act labeling guide. Technical report, DTIC Document. find
  9. John J Godfrey, Edward C Holliman, and Jane McDaniel. 1992. Switchboard: Telephone speech corpus for research and development. In Acoustics, Speech, and Signal Processing, 1992. ICASSP-92., 1992 IEEE International Conference on, volume 1, pages 517–520. IEEE. find
  10. Sergio Grau, Emilio Sanchis, Maria Jose Castro, and David Vilar. 2004. Dialogue act classification using a bayesian approach. In 9th Conference Speech and Computer. find
  11. Alex Graves, Navdeep Jaitly, and Abdel-rahman Mohamed. 2013. Hybrid speech recognition with deep bidirectional lstm. In Automatic Speech Recognition and Understanding (ASRU), 2013 IEEE Workshop on, pages 273–278. IEEE. find
  12. Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2012. Improving neural networks by preventing co-adaptation of feature detectors. CoRR, abs/1207.0580. ↗
  13. Sepp Hochreiter. 1991. Untersuchungen zu dynamischen neuronalen netzen. Diploma, Technische Universität München, page 91. find
  14. Edward Ivanovic. 2005. Dialogue act tagging for instant messaging chat sessions. In Proceedings of the ACL Student Research Workshop, pages 79–84. Association for Computational Linguistics. find
  15. Aaron Jaech, Larry Heck, and Mari Ostendorf. 2016. Domain adaptation of recurrent neural networks for natural language understanding. arXiv preprint arXiv:1604.00117. ↗
  16. Susanne Jekat, Alexandra Klein, Elisabeth Maier, Ilona Maleck, Marion Mast, and J Joachim Quantz. 1995. Dialogue acts in VERBMOBIL. Citeseer. find
  17. Gang Ji and Jeff Bilmes. 2006. Backoff model training using partially observed data: Application to dialog act tagging. In Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, pages 280–287. Association for Computational Linguistics. find
  18. Yangfeng Ji, Gholamreza Haffari, and Jacob Eisenstein. 2016. A latent variable recurrent neural network for discourse relation language models. arXiv preprint arXiv:1603.01913. ↗
  19. Daniel Jurafsky, Rebecca Bates, Noah Coccaro, Rachel Martin, Marie Meteer, Klaus Ries, Elizabeth Shriberg, Au-dreas Stolcke, Paul Taylor, Van Ess-Dykema, et al. 1997. Automatic detection of discourse structure for speech recognition and understanding. In Automatic Speech Recognition and Understanding, 1997. Proceedings., 1997 IEEE Workshop on, pages 88–95. IEEE. find
  20. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent convolutional neural networks for discourse compositionality. arXiv preprint arXiv:1306.3584. ↗ our card
  21. Yoon Kim. 2014. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882. ↗ our card
  22. Pavel Král and Christophe Cerisara. 2012. Dialogue act recognition approaches. Computing and Informatics, 29(2):227–250. find
  23. Ji Young Lee and Franck Dernoncourt. 2016. Sequential short-text classification with recurrent and convolutional neural networks. arXiv preprint arXiv:1603.03827. ↗ our card
  24. Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2015. Molding cnns for text: non-linear, non-consecutive convolutions. arXiv preprint arXiv:1508.04112. ↗
  25. Piroska Lendvai and Jeroen Geertzen. 2007. Token-based chunking of turn-internal dialogue act sequences. In Proceedings of the 8th SIGDIAL Workshop on Discourse and Dialogue, pages 174–181. find
  26. Liang Lu, Lingpeng Kong, Chris Dyer, Noah A Smith, and Steve Renals. 2016. Segmental recurrent neural networks for end-to-end speech recognition. arXiv preprint arXiv:1603.00223. ↗
  27. Christopher D Manning. 2016. Computational linguistics and deep learning. Computational Linguistics. find
  28. Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernockỳ, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In Interspeech, volume 2, page 3. find
  29. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. ↗
  30. Nicole Novielli and Carlo Strapparava. 2013. The role of affect analysis in dialogue act identification. Affective Computing, IEEE Transactions on, 4(4):439–451. find
  31. Marius Pachitariu and Maneesh Sahani. 2013. Regularization and nonlinearities for neural language models: when are they needed? arXiv preprint arXiv:1301.5650. ↗
  32. Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013. On the difficulty of training recurrent neural networks. ICML (3), 28:1310–1318. find
  33. James W Pennebaker, Roger J Booth, and Martha E Francis. 2007. Linguistic inquiry and word count: Liwc [computer software]. Austin, TX: liwc. net. find
  34. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In EMNLP, volume 14, pages 1532–1543. find
  35. Suman Ravuri and Andreas Stolcke. 2015. Recurrent neural network and lstm models for lexical utterance classification. Proc. Interspeech, Dresden. find
  36. Radim Řehůřek and Petr Sojka. 2010. Software Framework for Topic Modelling with Large Corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pages 45–50, Valletta, Malta, May. ELRA. http://is.muni.cz/publication/884893/en. ↗
  37. Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the conference on empirical methods in natural language processing (EMNLP), volume 1631, page 1642. Citeseer. find
  38. Andreas Stolcke, Noah Coccaro, Rebecca Bates, Paul Taylor, Carol Van Ess-Dykema, Klaus Ries, Elizabeth Shriberg, Daniel Jurafsky, Rachel Martin, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational linguistics, 26(3):339–373. find our card
  39. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112. find
  40. Swabha Swayamdipta, Miguel Ballesteros, Chris Dyer, and Noah A Smith. 2016. Greedy, joint syntactic-semantic parsing with stack lstms. arXiv preprint arXiv:1606.08954. ↗
  41. Kai Sheng Tai, Richard Socher, and Christopher D Manning. 2015. Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.00075. ↗
  42. Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: a simple and general method for semi-supervised learning. In Proceedings of the 48th annual meeting of the association for computational linguistics, pages 384–394. Association for Computational Linguistics. find our card
  43. Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. 2014. Recurrent neural network regularization. arXiv preprint arXiv:1409.2329. ↗ our card