Elamur Research⌕
☰
home / papers / 2017 / Preserving Distributional Information in Dialogue Act Classification

Preserving Distributional Information in Dialogue Act Classification

Summary

The paper addresses online dialogue act (DA) classification, where each utterance must be labeled as it arrives, without seeing future turns. Many neural sequence labelers feed their single best guess for the current DA into the next prediction. A wrong guess then misleads later decisions, causing label bias and error propagation. The authors propose keeping the whole probability distribution over the current DA and passing it forward instead. Their base model encodes each utterance with an LSTM and uses DA-specific softmax parameters, similar to a discriminative maximum entropy Markov model. They test two variants. Uncertainty Propagation computes the posterior over labels with a forward-style dynamic programming step and uses it for both training and decoding. Average Embedding mixes the label-specific parameters, weighted by the predicted distribution. On MapTask and Switchboard, both variants beat greedy, gold-label and Scheduled Sampling baselines, as well as more complex published neural models. They also recover from errors more reliably. The idea is general and well suited to streaming sequence-labeling tasks.

Key points

  • Data:
    • MapTask has 128 conversations and 13 DAs. The split is 12 validation, 13 test and the remainder for training.
    • Switchboard has 1155 conversations and 42 DAs. It follows the Stolcke et al. (2000) setup of 1115 training and 19 test conversations.
  • Comparison of label-use strategies (Table 1): Uncertainty Propagation scored 75.61% on Switchboard and 65.87% on MapTask. Scheduled Sampling reached 74.43% and 64.50%, and the no-label model 72.93% and 61.27%.
  • Comparison with published systems:
    • On Switchboard, Uncertainty Propagation reached 75.6%, ahead of Tran et al. (2017) at 74.5% and Kalchbrenner and Blunsom (2013) at 73.9%.
    • On MapTask, with four-fold cross-validation, it reached 62.9%, while Tran et al. (2017) obtained 61.6%.
  • Ji et al. (2016): this work reports 77.0%, but the authors could only obtain 72.5% using its public code, because the experimental setup was underspecified.
  • Error recovery (Table 4): Uncertainty Propagation was most likely to predict correctly after n consecutive mistakes. Average Embedding degraded quickly as n grew, particularly on Switchboard.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Speech act theory

Cited by

10 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 8 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d17-1229

@inproceedings{tran2017preserving,
  title = {{Preserving Distributional Information in Dialogue Act Classification}},
  author = {Tran, Quan Hung and Zukerman, Ingrid and Haffari, Gholamreza},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1229},
  url = {https://doi.org/10.18653/v1/d17-1229}
}

References

15 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anne H Anderson, Miles Bader, Ellen Gurman Bard, Elizabeth Boyle, Gwyneth Doherty, Simon Garrod, Stephen Isard, Jacqueline Kowtko, Jan McAllister, Jim Miller, et al. 1991. The HCRC MapTask corpus. Language and speech 34(4):351–366. find
  2. Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. Scheduled sampling for sequence prediction with recurrent neural networks. In Advances in Neural Information Processing Systems. pages 1171–1179. ↗
  3. Björn Gambäck, Fredrik Olsson, and Oscar Täckström. 2011. Active learning for dialogue act classification. In Interspeech 2011 – Proceedings of the International Conference on Spoken Language Processing. pages 1329–1332. find
  4. John J Godfrey, Edward C Holliman, and Jane Mc-Daniel. 1992. Switchboard: Telephone speech corpus for research and development. In Acoustics, Speech, and Signal Processing, 1992. ICASSP-92., 1992 IEEE International Conference on. IEEE, volume 1, pages 517–520. find
  5. Alex Graves. 2013. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 . ↗
  6. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9(8):1735–1780. ↗
  7. Yangfeng Ji, Gholamreza Haffari, and Jacob Eisenstein. 2016. A latent variable recurrent neural network for discourse-driven language models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego, California, pages 332–342. http://www.aclweb.org/anthology/N16-1037. ↗
  8. Fatema N Julia, Khan M Iftekharuddin, and ATIQ U ISLAM. 2010. Dialog act classification using acoustic and discourse information of MapTask data. International Journal of Computational Intelligence and Applications 9(04):289–311. ↗
  9. Daniel Jurafsky, Elizabeth Shriberg, and Debra Bi-asca. 1997. Switchboard SWBD-DAMSL Shallow-Discourse-Function Annotation Coders Manual, Draft 13. Technical report, University of Colorado. find
  10. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent convolutional neural networks for discourse compositionality. arXiv preprint arXiv:1306.3584 . ↗ our card
  11. Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2015. Sequence level training with recurrent neural networks. arXiv preprint arXiv:1511.06732 . ↗ our card
  12. Sheng-syun Shen and Hung-yi Lee. 2016. Neural attention models for sequence classification: Analysis and application to key term extraction and dialogue act detection. arXiv preprint arXiv:1604.00077 . ↗
  13. Andreas Stolcke, Noah Coccaro, Rebecca Bates, Paul Taylor, Carol Van Ess-Dykema, Klaus Ries, Elizabeth Shriberg, Daniel Jurafsky, Rachel Martin, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational linguistics 26(3):339–373. ↗ our card
  14. Dinoj Surendran and Gina-Anne Levow. 2006. Dialog act tagging with support vector machines and hidden markov models. In Interspeech 2006 – Proceedings of the International Conference on Spoken Language Processing. pages 1950–1953. find
  15. Quan Hung Tran, Ingrid Zukerman, and Gholamreza Haffari. 2017. A hierarchical neural model for learning sequences of dialogue acts. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. Valencia, Spain, pages 428–437. http://www.aclweb.org/anthology/E17-1041. ↗ our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/tran-2017-preserving-distributional-information-dialogue-act/