Elamur Research
☰
home / papers / 2017 / Using Context Information for Dialog Act Classification in DNN Framework

Using Context Information for Dialog Act Classification in DNN Framework

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/D17-1231.pdf

Summary

The paper asks how much surrounding dialog context helps dialog act (DA) classification within neural models, and which way of adding it works best. Earlier DNN studies each tried a single context method, so the gains were unclear. The authors begin with a context-free baseline: a sentence-level CNN with pre-trained embeddings and a speaker-change feature. They then compare three families of context models. The first appends DA predictions for earlier utterances (hard labels or posterior probabilities) to the CNN vector and retrains. The second places a hierarchical RNN/LSTM, BLSTM or CNN over the sequence of sentence vectors. The third decodes the whole conversation by combining CNN posteriors with DA transition probabilities. Experiments use the Switchboard corpus with the 42-tag scheme and text transcripts only, without acoustic features. Every context method beats the baseline by a statistically significant margin, by about 2–3 points of accuracy. The methods differ very little from one another. The results set a new state of the art on Switchboard and suggest that context mostly helps through the preceding utterances, which also makes it usable in online settings.

Key points

  • Data: Switchboard, 42 DA tags (Jurafsky et al., 1997). Inputs are transcripts plus speaker information. Utterances are pre-segmented.
  • Accuracy on the two evaluation sets:
    • Baseline CNN: 74.73 / 77.12
    • CNN + DA predictions: 76.73 / 79.9
    • CNN + BLSTM: 76.91 / 79.7
    • Hierarchical CNN + CNN: 77.15 / 79.74
    • CNN posteriors + DA transition decoding: 76.70 / 79.69
  • Using system-predicted DA history in both training and testing works better than training on reference labels and testing on predictions. The immediately preceding utterance gives most of the gain, and longer histories of predicted labels add noise.
  • For comparison, Kalchbrenner and Blunsom (2013) reported 73.9% with a recurrent CNN, but only on 19 test conversations, so the numbers are not directly comparable.
  • Remaining frequent errors are statement vs. statement-opinion and backchannel vs. agreement (e.g., "right", "yeah"). Context corrects roughly 5% of errors that were backchannel-vs-answer confusions.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Speech act theory

Cited by

45 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 40 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d17-1231

@inproceedings{liu2017using,
  title = {{Using Context Information for Dialog Act Classification in DNN Framework}},
  author = {Liu, Yang and Han, Kun and Tan, Zhao and Lei, Yun},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1231},
  url = {https://doi.org/10.18653/v1/d17-1231}
}

References

21 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jeremy Ang, Yang Liu, and Elizabeth Shriberg. 2005. Automatic dialog act segmentation and classification in multiparty meetings. In Proc. of ICASSP. find
  2. Lin Chen and Barbara Di Eugenio. 2013. Multimodality and dialogue act classification in the robohelper project. In Proceddings of Sigdial. find
  3. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research 12(Aug):2493–2537. find our card
  4. Alfred Dielmann and Steve Renals. 2008. Recognition of dialogue acts in multiparty meetings using a switching dbn. IEEE Transactions on Audio, Speech and Language Processing 16. find
  5. Raul Fernandez and Rosalind Picard. 2002. Dialog act classification from prosodic features using support vector machines. In Proc. of Speech Prosody. find
  6. Gang Ji and Jeff Bilmes. 2005. Dialog act tagging using graphical models. In Proc. of ICASSP. find
  7. D. Jurafsky, L. Shriberg, and D. Biasca. 1997. Switchboard swbd-damsl shallow-discourse-function annotation coders manual. Technical report, University of Colorado at Boulder. find
  8. Silvia Quarteroni, Alexei V. Ivanov, and Giuseppe Riccardi. 2011. Simultaneous dialog segmentation and classification from human-human spoken conversations. In Proceedings of ICASSP. find
  9. Eugenio Ribeiro, Ricardo Ribeiro, and David Martins de Matos. 2015. The influence of context on dialogue act recognition . find
  10. Lina M. Rojas-Barahona, Milica Gasic, Nikola Mrksic, Pei-Hao Su, Stefan Ultes, Tsung-Hsien, and Steve Young. 2016. Exploiting sentence and context representation in deep neural models for spoken language understanding. find
  11. Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000. Dialog act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics . find
  12. Anand Venkataraman, Lucianna Ferrer, Andreas Stolcke, and Elizabeth Shriberg. 2003. Training a prosody based dialog act tagger from unlabeled data. In Proc. of ICASSP. find
  13. Nick Webb, Mark Hepple, and Yorick Wilks. 2005. Dialog act classification based on intra-utterances features. In AAAI workshop on Spoken Language Understanding. find
  14. Yucan Zhou, Qinghua Hu, Jie Liu, and Yuan Jia. 2015. Combining heterogeneious deep neural networks with conditional random fields for chinese dialogue act recognition. Neurocomputing 168. find
  15. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent convolutional neural networks for discourse compositionality. find our card
  16. Su Nam Kim, Lawrence Cavedon, and Timothy Baldwin. 2010. Classifying dialog acts in one-on-one live chats. In Proceedings of EMNLP. find
  17. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proc. of EMNLP. pages 1746–1751. find our card
  18. Pavel Kral and Chrisophe Cerisara. 2014. Automatic dialogue act recognition with syntactic features. In Proceedings of LREC. find
  19. Piroska Lendvai. 2007. Token-based chunking of turn-internal dialogue act sequences. find
  20. Yang Liu. 2006. Using svm and error-correcting codes for multiclass dialog act classification in meeting corpus. In Interspeech. find
  21. Marion Mast, Ralf Kompe, Stefan Harbeck, Andreas Kiebling, Heinrich Niemann, and Elmar Noth. 1996. Dialog act classification with the help of prosody. In Proc. of ICSLP. find