Elamur Research⌕
☰
home / papers / 2018 / Deep Dialog Act Recognition using Multiple Token, Segment, and Context Information Representations

Deep Dialog Act Recognition using Multiple Token, Segment, and Context Information Representations

Paper · 2018 · Eugenio Ribeiro; Ricardo Ribeiro; and David Martins de Matos

Summary

The paper tackles automatic dialog act recognition: deciding what a speaker intends with each segment of a dialog, such as a question, a statement or an agreement. The authors break the problem into three representation levels and test options at each one. At the token level they combine word-level input (classic and contextualized BERT embeddings), character-level input aimed at morphological cues, and a functional abstraction that hides words and keeps segment structure. At the segment level they compare RNN- and CNN-based encoders. At the context level they encode the labels of earlier segments, condensed by a recurrent summarizer, together with turn-taking history. Experiments on SwDA and MRDA, including replications of earlier state-of-the-art systems on the same splits, show the combined system beating prior work. The margin is about 2 points on SwDA and over 10 points on MRDA. When future segments are also visible, as they are to a human annotator, the system roughly matches human agreement on SwDA and exceeds it on MRDA. The work shows that well-chosen inputs and context matter more than elaborate segment encoders.

Key points

  • Evaluated on two English corpora: Switchboard Dialog Act Corpus (SwDA) and ICSI Meeting Recorder Dialog Act Corpus (MRDA).
  • With BERT word embeddings, replacing the segment encoder with plain max pooling cost under one percentage point on SwDA and almost nothing on MRDA. The authors kept a CNN encoder because it is cheaper.
  • The character-level CNN uses windows of 3, 5 and 7 characters, aimed at affixes and lemmas.
  • Context works best as a recurrent summary of earlier segments' predicted labels. On SwDA, the gap between gold-label context and the classifier's own predictions was only 1.38 points. For turn-taking, a flat flag over the three previous segments worked best.
  • With past and future context, the system passed the 84% inter-annotator agreement on the SwDA validation set but stayed 1.66 points below it on the test set.

Interesting ideas

  • I1. Feed context to the model as the labels of earlier units, condensed into a running summary, instead of their raw words; this keeps context compact and order-aware.
  • I2. Measure the upper bound by giving the model gold context, then compare it with self-predicted context to see how much errors propagate.
  • I3. Choose character window sizes on purpose, matching typical affix lengths, rather than tuning them blindly, so the morphology cues are interpretable.
  • I4. Abstract words away into functional or syntactic units so the classifier attends to segment structure, which signals acts like questions or commands.
  • I5. Ablate down to the simplest encoder (max pooling) and replicate competing systems on identical splits before claiming gains.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): the speech-act layer in prag could use the sequence of already-assigned speech acts from earlier sentences as an explicit, explainable feature for labeling the current one.
  • I2 (learn): any stage that consumes its own earlier outputs (coreference, event order) can report a gold-vs-predicted gap as a standard diagnostic of error cascades.
  • I3 (tokens): the en tagger could expose fixed prefix/suffix windows as named features, making it explainable why a word got a given tag or mood.
  • I4 (frames): speech-act detection can key on UD tree shapes (aux inversion, imperative root, wh-word position) rather than on lexical items.
  • I5 (learn): every new parser or reader component should be benchmarked against a trivial baseline and re-run rivals on the same data, as a built-in negative control.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Instituto de Engenharia de Sistemas e Computadores Investigação e DesenvolvimentoLisbon · Portugal21 works · 3,723 citations🎓Instituto Superior TécnicoLisbon · Portugal28 works · 3,944 citations🎓Iscte – Instituto Universitário de LisboaLisbon · Portugal5 works · 34 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 2Research subjects and languages the work is about.Dialogue systems · 401 worksSpeech act theory · 160 works

Cite

Cite as Eugenio Ribeiro; Ricardo Ribeiro; and David Martins de Matos. 2018. Deep Dialog Act Recognition using Multiple Token, Segment, and Context Information Representations. arXiv:1807.08587.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1807.08587 · arXiv 1807.08587

@misc{ribeiro2018deep,
  title = {{Deep Dialog Act Recognition using Multiple Token, Segment, and Context Information Representations}},
  author = {Eugenio Ribeiro; Ricardo Ribeiro; and David Martins {de Matos}},
  year = {2018},
  doi = {10.48550/arxiv.1807.08587},
  eprint = {1807.08587},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1807.08587}
}

References

63 references, 29 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abadi, M., et al. (2015). TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/. ↗
  2. Alexandersson, J., Buschbeck-Wolf, B., Fujinami, T., Kipp, M., Koch, S., Maier, E., Reithinger, N., Schmitz, B., & Siegel, M. (1998). Dialogue Acts in VERBMOBIL-2 Second Edition. Tech. rep., DFKI. find
  3. Anderson, A. H., Bader, M., Bard, E. G., Boyle, E., Doherty, G., Garrod, S., Isard, S., Kowtko, J., McAllister, J., Miller, J., et al. (1991). The HCRC Map Task Corpus. Language and Speech, 34 (4), 351–366. ↗
  4. Ang, J., Liu, Y., & Shriberg, E. (2005). Automatic Dialog Act Segmentation and Classification in Multiparty Meetings. In ICASSP, Vol. 1, pp. 1061–1064. find
  5. Baum, L. E., & Petrie, T. (1966). Statistical Inference For Probabilistic Functions Of Finite State Markov Chains. Annals of Mathematical Statistics, 37 (6), 1554–1563. ↗
  6. Benedı́, J.-M., Lleida, E., Varona, A., Castro, M.-J., Galiano, I., Justo, R., de Letona, I. L., & Miguel, A. (2006). Design and Acquisition of a Telephone Spontaneous Speech Dialogue Corpus in Spanish: DIHANA. In LREC, pp. 1636–1639. find
  7. Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017). Enriching Word Vectors with Subword Information. Transactions of the Association for Computational Linguistics, 5, 135–146. ↗
  8. Camacho-Collados, J., & Pilehvar, M. T. (2018). From Word to Sense Embeddings: A Survey on Vector Representations of Meaning. Journal of Artificial Intelligence Research, 63, 743–788. ↗
  9. Carletta, J. (1996). Assessing Agreement on Classification Tasks: The Kappa Statistic. Computational Linguistics, 22 (2), 249–254. ↗
  10. Chollet, F., et al. (2015). Keras: The Python Deep Learning Library. https://keras.io/. ↗
  11. Chung, J., Gülcehre, C., Cho, K., & Bengio, Y. (2014). Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. In NIPS Deep Learning and Representation Learning Workshop. find
  12. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. (2011). Natural Language Processing (Almost) from Scratch. Journal of Machine Learning Research, 12, 2493–2537.
  13. Conneau, A., Schwenk, H., Barrault, L., & Lecun, Y. (2017). Very Deep Convolutional Networks for Text Classification. In EACL, Vol. 1, pp. 1107–1116.
  14. Cover, T., & Hart, P. (1967). Nearest Neighbor Pattern Classification. IEEE Transactions on Information Theory, 13 (1), 21–27. ↗
  15. Devlin, J., Chang, M.-W., Kenton, L., & Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. CoRR, abs/1810.04805. ↗
  16. Fukushima, K. (1980). Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position. Biological Cybernetics, 36 (4), 193–202. ↗
  17. Gambäck, B., Olsson, F., & Täckström, O. (2011). Active Learning for Dialogue Act Classification. In INTERSPEECH, pp. 1329–1332. find
  18. Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P., Liu, N. F., Peters, M., Schmitz, M., & Zettlemoyer, L. S. (2017). AllenNLP: A Deep Semantic Natural Language Processing Platform. CoRR, abs/1803.07640. ↗
  19. Godfrey, J. J., Holliman, E. C., & McDaniel, J. (1992). SWITCHBOARD: Telephone Speech Corpus for Research and Development. In ICASSP, Vol. 1, pp. 517–520. find
  20. Goldberg, Y. (2016). A Primer on Neural Network Models for Natural Language Processing. Journal of Artificial Intelligence Research, 57, 345–420. ↗
  21. Han, L., Kashyap, A. L., Finin, T., Mayfield, J., & Weese, J. (2013). UMBC EBIQUITY-CORE: Semantic Textual Similarity Systems. In *SEM, Vol. 1, pp. 44–52. ↗
  22. Hochreiter, S., & Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9 (8), 1735–1780. ↗
  23. Honnibal, M., & Montani, I. (2017). spaCy 2: Natural Language Understanding with Bloom Embeddings, Convolutional Neural Networks and Incremental Parsing. https://spacy.io/. ↗
  24. Janin, A., Baron, D., Edwards, J., Ellis, D., Gelbart, D., Morgan, N., Peskin, B., Pfau, T., Shriberg, E., Stolcke, A., & Wooters, C. (2003). The ICSI Meeting Corpus. In ICASSP, Vol. 1, pp. 364–367. find
  25. Ji, Y., Haffari, G., & Eisenstein, J. (2016). A Latent Variable Recurrent Neural Network for Discourse Relation Language Models. In NAACL-HLT, pp. 332–342. ↗
  26. Jurafsky, D., Shriberg, E., & Biasca, D. (1997). Switchboard SWBD-DAMSL Shallow-Discourse-Function Annotation Coders Manual. Tech. rep. Draft 13, University of Colorado, Institute of Cognitive Science. find
  27. Kalchbrenner, N., & Blunsom, P. (2013). Recurrent Convolutional Neural Networks for Discourse Compositionality. In Workshop on Continuous Vector Space Models and their Compositionality, pp. 119–126.
  28. Khanpour, H., Guntakandla, N., & Nielsen, R. (2016). Dialogue Act Classification in Domain-Independent Conversations Using a Deep Recurrent Neural Network. In COLING, pp. 2012–2021.
  29. Kim, S., D’Haro, L. F., Banchs, R. E., Williams, J., & Henderson, M. (2016). The Fourth Dialog State Tracking Challenge. In International Workshop on Spoken Dialog Systems. ↗
  30. Kim, Y. (2014). Convolutional Neural Networks for Sentence Classification. In EMNLP, pp. 1746–1751.
  31. Kingma, D. P., & Ba, J. (2015). Adam: A Method for Stochastic Optimization. In ICLR. ↗
  32. Král, P., & Cerisara, C. (2010). Dialogue Act Recognition Approaches. Computing and Informatics, 29 (2), 227–250. find
  33. Lai, S., Xu, L., Liu, K., & Zhao, J. (2015). Recurrent Convolutional Neural Networks for Text Classification. In AAAI, pp. 2267–2273. find
  34. Lavelli, A., Sebastiani, F., & Zanoli, R. (2004). Distributional Term Representations: An Experimental Comparison. In CIKM, pp. 615–624. find
  35. Lee, J. Y., & Dernoncourt, F. (2016). Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks. In NAACL-HLT, pp. 515–520.
  36. Levy, O., & Goldberg, Y. (2014). Dependency-Based Word Embeddings. In ACL, Vol. 2, pp. 302–308. find
  37. Liu, Y., Han, K., Tan, Z., & Lei, Y. (2017). Using Context Information for Dialog Act Classification in DNN Framework. In EMNLP, pp. 2160–2168.
  38. Manning, C. D. (2015). Computational Linguistics and Deep Learning. Computational Linguistics, 41 (4), 701–707. ↗
  39. Marcus, M. P., Santorini, B., & Marcinkiewicz, M. A. (1993). Building a Large Annotated Corpus of English: The Penn Treebank. Computational Linguistics, 19 (2), 313–330. ↗
  40. Mikolov, T., Grave, E., Bojanowski, P., Puhrsch, C., & Joulin, A. (2018). Advances in Pre-Training Distributed Word Representations. In LREC.
  41. Mikolov, T., Karafiát, M., Burget, L., Cernocký, J., & Khudanpur, S. (2010). Recurrent Neural Network Based Language Model. In INTERSPEECH, pp. 1045–1048. find
  42. Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed Representations of Words and Phrases and their Compositionality. In NIPS, pp. 3111–3119. ↗
  43. Pennington, J., Socher, R., & Manning, C. D. (2014). GloVe: Global Vectors for Word Representation. In EMNLP, pp. 1532–1543.
  44. Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep Contextualized Word Representations. In NAACL-HLT, Vol. 1, pp. 2227–2237.
  45. Petrov, S., Das, D., & McDonald, R. (2012). A Universal Part-of-Speech Tagset. In LREC.
  46. Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving Language Understanding by Generative Pre-Training. Preprint. find
  47. Ribeiro, E., Ribeiro, R., & de Matos, D. M. (2015). The Influence of Context on Dialogue Act Recognition. CoRR, abs/1506.00839. ↗
  48. Ribeiro, E., Ribeiro, R., & de Matos, D. M. (2018). A Study on Dialog Act Recognition using Character-Level Tokenization. In AIMSA, pp. 93–103. ↗
  49. Ribeiro, E., Ribeiro, R., & de Matos, D. M. (2019). A Multilingual and Multidomain Study on Dialog Act Recognition using Character-Level Tokenization. Information, 10 (3), 94. ↗
  50. Rotaru, M. (2002). Dialog Act Tagging Using Memory-Based Learning. Tech. rep., University of Pittsburgh. find
  51. Schmidhuber, J. (1990). Dynamische Neuronale Netze und das Fundamentale Raumzeitliche Lernproblem. Ph.D. thesis, TU München. find
  52. Schmitt, A., Ultes, S., & Minker, W. (2012). A Parameterized and Annotated Spoken Dialog Corpus of the CMU Let’s Go Bus Information System. In LREC. find
  53. Searle, J. R. (1969). Speech Acts: An Essay in the Philosophy of Language. Cambridge University Press. ↗
  54. Shriberg, E., Dhillon, R., Bhagat, S., Ang, J., & Carvey, H. (2004). The ICSI Meeting Recorder Dialog Act (MRDA) Corpus. In SIGDIAL, pp. 97—-100. ↗
  55. Sridhar, V. K. R., Bangalore, S., & Narayanan, S. (2009). Combining Lexical, Syntactic and Prosodic Cues for Improved Online Dialog Act Tagging. Computer Speech and Language, 23 (4), 407–422. ↗
  56. Stolcke, A., Coccaro, N., Bates, R., Taylor, P., Van Ess-Dykema, C., Ries, K., Shriberg, E., Jurafsky, D., Martin, R., & Meteer, M. (2000). Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech. Computational Linguistics, 26 (3), 339–373. ↗
  57. Tiedemann, J. (2012). Parallel Data, Tools and Interfaces in OPUS. In LREC, pp. 2214– 2218. find
  58. Tran, Q. H., Zukerman, I., & Haffari, G. (2017a). A Generative Attentional Neural Network Model for Dialogue Act Classification. In ACL, Vol. 2, pp. 524–529.
  59. Tran, Q. H., Zukerman, I., & Haffari, G. (2017b). A Hierarchical Neural Model for Learning Sequences of Dialogue Acts. In EACL, Vol. 1, pp. 428–437.
  60. Tran, Q. H., Zukerman, I., & Haffari, G. (2017c). Preserving Distributional Information in Dialogue Act Classification. In EMNLP, pp. 2151–2156.
  61. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is All You Need. In NIPS, pp. 5998–6008.
  62. Webb, N., & Ferguson, M. (2010). Automatic Extraction of Cue Phrases for Cross-corpus Dialogue Act Classification. In COLING, pp. 1310–1317.
  63. Zhang, X., Zhao, J., & LeCun, Y. (2015). Character-Level Convolutional Networks for Text Classification. In NIPS, Vol. 1, pp. 649–657. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ribeiro-2018-deep-dialog-act-recognition-using/