Elamur Research⌕
☰
home / papers / 2017 / Neural-based Context Representation Learning for Dialog Act Classification

Neural-based Context Representation Learning for Dialog Act Classification

Summary

Short utterances such as "Right" are often ambiguous for dialog act (DA) classification. Their label depends on what came before them in the conversation. Ortega and Vu study how neural models can learn a useful representation of preceding utterances for this task. Each utterance and its context are encoded with a multi-window CNN over word2vec embeddings. The authors then compare several ways of combining these utterance vectors:

  • max-pooling,
  • plain attention,
  • an RNN with LSTM or GRU cells,
  • an RNN with attention placed either on its outputs or on its inputs (order-preserving).

Experiments use the MRDA and SwDA corpora with the splits of Lee and Dernoncourt (2016). Simply concatenating context, max-pooling, or attention alone brought little or no gain. Recurrent modeling of the context helped consistently, and attention gave further improvement. The best placement of attention differed by corpus: on the RNN inputs for multiparty meetings (MRDA), and on the RNN outputs for two-speaker calls (SwDA). The best systems are on par with the state of the art. The work shows that hierarchical context modeling matters for DA recognition and that attention design should fit the dialog type.

Key points

  • Datasets:
    • MRDA (ICSI Meeting Recorder, 5 tags, about 78k training utterances).
    • SwDA (Switchboard, 43 tags, about 193k training utterances).
    • Majority classes are 59.1% and 33.7% respectively.
  • Best accuracy on MRDA: 84.3% with RNN-Input-Attention (LSTM). This compares with 84.6% for CNN-FF (Lee and Dernoncourt, 2016) and 83.6% for the no-context Baseline I.
  • Best accuracy on SwDA: 73.8% with RNN-Output-Attention (LSTM). This beats CNN-FF at 73.1% and the no-context baseline at 71.3%.
  • Optimal context length depends on the corpus: three previous utterances for MRDA and two for SwDA.
  • Setup:
    • Kim (2014)-style CNN with filter widths 3–5 and 100 feature maps each.
    • Dropout 0.5.
    • Frozen 300-dimensional word2vec embeddings.
    • Averaged SGD for 30 epochs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Speech act theory

Cited by

17 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 14 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w17-5530 · arXiv 1708.02561

@inproceedings{ortega2017neural,
  title = {{Neural-based Context Representation Learning for Dialog Act Classification}},
  author = {Ortega, Daniel and Vu, Ngoc Thang},
  booktitle = {Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue},
  year = {2017},
  doi = {10.18653/v1/w17-5530},
  eprint = {1708.02561},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/w17-5530}
}

References

26 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Heike Adel and Hinrich Schütze. 2017. Exploring different dimensions of attention for uncertainty detection. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. Association for Computational Linguistics, Valencia, Spain, pages 22–34. http://www.aclweb.org/anthology/E17-1003. ↗
  2. Adam Janin, Don Baron, Jane Edwards, Dan Ellis, David Gelbart, Nelson Morgan, Barbara Peskin, Thilo Pfau, Elizabeth Shriberg, Andreas Stolcke, and Chuck Wooters. 2003. The icsi meeting corpus. pages 364–367. find
  3. Gang Ji and Jeff Bilmes. 2006. Backoff model training using partially observed data: Application to dialog act tagging. In Proceedings of the Main Conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics. Association for Computational Linguistics, Stroudsburg, PA, USA, HLT-NAACL ’06, pages 280–287. https://doi.org/10.3115/1220835.1220871. ↗
  4. Jeremy Ang, Yang Liu, and Elizabeth Shriberg. 2005. Automatic dialog act segmentation and classification in multiparty meetings. In in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP. pages 1061–1064. find
  5. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. CoRR abs/1409.0473. http://arxiv.org/abs/1409.0473. ↗ our card
  6. Yangfeng Ji, Gholamreza Haffari, and Jacob Eisenstein. 2016. A latent variable recurrent neural network for discourse relation language models. CoRR abs/1603.01913. http://arxiv.org/abs/1603.01913. ↗
  7. KyungHyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. CoRR abs/1409.1259. http://arxiv.org/abs/1409.1259. ↗ our card
  8. D. Jurafsky, E. Shriberg, and D. Biasca. 1997. Switchboard SWBD-DAMSL shallow-discourse-function annotation coders manual. Technical Report Draft 13, University of Colorado, Institute of Cognitive Science. find
  9. Jan Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, KyungHyun Cho, and Yoshua Bengio. 2015. Attention-based models for speech recognition. CoRR abs/1506.07503. http://arxiv.org/abs/1506.07503. ↗ our card
  10. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent convolutional neural networks for discourse compositionality. CoRR abs/1306.3584. http://arxiv.org/abs/1306.3584. ↗ our card
  11. Junyoung Chung, Çaglar Gülçehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR abs/1412.3555. http://arxiv.org/abs/1412.3555. ↗
  12. Yoon Kim. 2014. Convolutional neural networks for sentence classification. CoRR abs/1408.5882. http://arxiv.org/abs/1408.5882. ↗ our card
  13. Ji Young Lee and Franck Dernoncourt. 2016. Sequential short-text classification with recurrent and convolutional neural networks. CoRR abs/1603.03827. http://arxiv.org/abs/1603.03827. ↗ our card
  14. Rajdip Dhillon, Sonali Bhagat, Hannah Carvey, and Elizabeth Shriberg. 2004. Meeting Recorder Project: Dialog Act Labeling Guide. Technical report, ICSI Tech. Report. https://goo.gl/TtLJlE. ↗
  15. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. CoRR abs/1310.4546. http://arxiv.org/abs/1310.4546. ↗
  16. John J. Godfrey, Edward C. Holliman, and Jane McDaniel. 1992. Switchboard: Telephone speech corpus for research and development. In Proceedings of the 1992 IEEE International Conference Matthias Zimmermann. 2009. Joint segmentation and classification of dialog acts using conditional random fields. In INTERSPEECH. pages 864–867. find
  17. B. T. Polyak and A. B. Juditsky. 1992. Acceleration of stochastic approximation by averaging. SIAM J. Control Optim. 30(4):838–855. https://doi.org/10.1137/0330046. ↗
  18. Eugénio Ribeiro, Ricardo Ribeiro, and David Martins de Matos. 2015. The influence of context on dialogue act recognition. CoRR abs/1506.00839. http://arxiv.org/abs/1506.00839. ↗
  19. Tim Rocktäschel, Edward Grefenstette, Karl Moritz Hermann, Tomás Kociský, and Phil Blunsom. 2015. Reasoning about entailment with neural attention. CoRR abs/1509.06664. http://arxiv.org/abs/1509.06664. ↗ our card
  20. Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. In Lluı́s Màrquez, Chris Callison-Burch, Jian Su, Daniele Pighin, and Yuval Marton, editors, Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015. The Association for Computational Linguistics, pages 379–389. http://aclweb.org/anthology/D/D15/D15-1044.pdf. ↗
  21. Sheng-syun Shen and Hung-yi Lee. 2016. Neural attention models for sequence classification: Analysis and application to key term extraction and dialogue act detection. CoRR abs/1604.00077. http://arxiv.org/abs/1604.00077. ↗
  22. Elizabeth Shriberg, Raj Dhillon, Sonali Bhagat, Jeremy Ang, and Hannah Carvey. 2004. The icsi meeting recorder dialog act (mrda) corpus. In Michael Strube and Candy Sidner, editors, Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue. Association for Computational Linguistics, Cambridge, Massachusetts, USA, pages 97–100. http://www.aclweb.org/anthology/W04-2319. ↗ our card
  23. Andreas Stolcke, Noah Coccaro, Rebecca Bates, Paul Taylor, Carol Van Ess-Dykema, Klaus Ries, Elizabeth Shriberg, Daniel Jurafsky, Rachel Martin, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Comput. Linguist. 26(3):339–373. https://doi.org/10.1162/089120100561737. ↗ our card
  24. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alexander J. Smola, and Eduard H. Hovy. 2016. Hierarchical attention networks for document classification. In HLT-NAACL. find our card
  25. Wenpeng Yin, Hinrich Schütze, Bing Xiang, and Bowen Zhou. 2015. ABCNN: attention-based convolutional neural network for modeling sentence pairs. CoRR abs/1512.05193. http://arxiv.org/abs/1512.05193. ↗ our card
  26. Ye Zhang and Byron C. Wallace. 2015. A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification. CoRR abs/1510.03820. http://arxiv.org/abs/1510.03820. ↗ our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/ortega-2017-neural-based-context-representation-learning/