Elamur Research⌕
☰
home / papers / 2020 / Automatic Recognition of the General-Purpose Communicative Functions defined by the ISO 24617-2 Standard for Dialog Act Annotation

Automatic Recognition of the General-Purpose Communicative Functions defined by the ISO 24617-2 Standard for Dialog Act Annotation

Paper · 2020 · Eugenio Ribeiro; Ricardo Ribeiro; and David Martins de Matos

Summary

The paper tackles automatic labeling of dialog segments with the general-purpose communicative functions of ISO 24617-2, the dialog act annotation standard. Unlike the flat tag sets of most dialog act corpora, these functions form a hierarchy, and a segment's correct label may stop at an inner node rather than a leaf. The authors use the English part of the DialogBank, the only public resource fully annotated to the standard, and adapt a flat dialog act recognizer to this hierarchical setting. Their network has a specialization layer and an output layer for each hierarchy level. Each level also gets an extra "nothing deeper" label. Maximum a posteriori estimation then selects the most probable valid path through the hierarchy. Because the DialogBank is very small, the segment-representation layers are pre-trained on a large dialog act corpus and reused through transfer learning. The hierarchical model beats a flat baseline on both exact match ratio and hierarchical F-measure. The authors describe this as the first study to recognize the full general-purpose hierarchy, which makes it a useful baseline for standard-compliant intent recognition.

Key points

  • ISO 24617-2 annotates functional segments, not turns. Each annotation is a structure covering participants, relations to other segments, one of nine semantic dimensions, a communicative function and optional qualifiers.
  • The DialogBank holds only 15 English and 9 Dutch dialogs, which together amount to fewer than 3,000 segments.
  • Segment representations use BERT embeddings and context information. The layers that build these representations are first trained on the biggest available dialog act corpus, then specialized per hierarchy level.
  • The authors added LEGO corpus dialogs, labeled through a mapping into ISO functions, as extra training data. Combined with the pre-trained layers, this data hurt performance, because the many repetitive, domain-specific LEGO dialogs caused the per-level layers to overfit.
  • In ablations, every component (per-level outputs, per-level specialization, the stop label, MAP path decoding, transfer learning) contributed to the gains.

Interesting ideas

  • I1. Giving each level of a label hierarchy its own classifier keeps each classifier from confusing a label with its own ancestors or descendants. A per-level "stop here" label then lets the predicted path end at a general node when the evidence is too weak for a specific one.
  • I2. MAP decoding over valid parent-child paths, rather than taking each level's top choice independently, rules out inconsistent label chains and tolerates a mistake at one level.
  • I3. Pre-training generic representations on a large related corpus and training only light specialization layers on the tiny target set reduces overfitting.
  • I4. Adding a large, label-mapped, single-domain corpus can backfire, because the specialized layers latch onto that corpus's quirks; mapped data needs checking, not blind trust.
  • I5. Scoring with a hierarchical F-measure alongside exact match gives partial credit for predictions that reach the right ancestor, which shows how close a wrong prediction actually is.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): Speech-act typing in crate prag could predict a coarse act (e.g., information-seeking) and refine it to a subtype only when the cues support it, so every output stays honest about its level of certainty.
  • I2 (world): Typing an entity against the WordNet/Wikidata ontology could pick the most probable consistent chain of classes, so a confident leaf can never contradict its parent.
  • I3 (learn): Small gold sets, such as rare speech-act or frame annotations, could train only thin heads on top of encoders trained with abundant silver data.
  • I4 (learn): Silver data from UD dialect conversion or mapping should be weighted or capped by domain, and its effect verified on held-out native gold before it is mixed into training.
  • I5 (answer): Evaluation of ontology-based answers could credit a hypernym-level answer partially, separating near misses from real errors.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Instituto de Engenharia de Sistemas e Computadores Investigação e DesenvolvimentoLisbon · Portugal21 works · 3,723 citations🎓Instituto Superior TécnicoLisbon · Portugal28 works · 3,944 citations🎓Iscte – Instituto Universitário de LisboaLisbon · Portugal5 works · 34 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 3Research subjects and languages the work is about.English · 1,373 worksDialogue systems · 401 worksSpeech act theory · 160 works

Cite

Cite as Eugenio Ribeiro; Ricardo Ribeiro; and David Martins de Matos. 2020. Automatic Recognition of the General-Purpose Communicative Functions defined by the ISO 24617-2 Standard for Dialog Act Annotation. arXiv:2003.03556.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2003.03556 · arXiv 2003.03556

@misc{ribeiro2020automatic,
  title = {{Automatic Recognition of the General-Purpose Communicative Functions defined by the ISO 24617-2 Standard for Dialog Act Annotation}},
  author = {Eugenio Ribeiro; Ricardo Ribeiro; and David Martins {de Matos}},
  year = {2020},
  doi = {10.48550/arxiv.2003.03556},
  eprint = {2003.03556},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2003.03556}
}

References

49 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abadi, M., et al. (2015). TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/. ↗
  2. Allen, J. F., & Schubert, L. K. (1991). The TRAINS Project. Tech. rep. Technical Report 382 and TRAINS Technical Note 91-1, Computer Science Department, University of Rochester. ↗
  3. Anderson, A. H., Bader, M., Bard, E. G., Boyle, E., Doherty, G., Garrod, S., Isard, S., Kowtko, J., McAllister, J., Miller, J., et al. (1991). The HCRC Map Task Corpus. Language and Speech, 34 (4), 351–366. ↗
  4. Anikina, T., & Kruijff-Korbayová, I. (2019). Dialogue Act Classification in Team Communication for Robot Assisted Disaster Response. In SIGDIAL, pp. 399–410. find
  5. Benedı́, J.-M., Lleida, E., Varona, A., Castro, M.-J., Galiano, I., Justo, R., de Letona, I. L., & Miguel, A. (2006). Design and Acquisition of a Telephone Spontaneous Speech Dialogue Corpus in Spanish: DIHANA. In LREC, pp. 1636–1639. find
  6. Bothe, C., Weber, C., Magg, S., & Wermter, S. (2018). A Context-based Approach for Dialogue Act Recognition using Simple Recurrent Neural Networks. In LREC, pp. 1952–1957.
  7. Bunt, H., Alexandersson, J., Choe, J.-W., Fang, A. C., Hasida, K., Petukhova, V., Popescu-Belis, A., & Traum, D. (2012). ISO 24617-2: A Semantically-Based Standard for Dialogue Annotation. In LREC, pp. 430–437.
  8. Bunt, H., Petukhova, V., Malchanau, A., Fang, A., & Wijnhoven, K. (2019). The Dialog-Bank: Dialogues with Interoperable Annotations. Language Resources and Evaluation, 53 (2), 213–249. find
  9. Bunt, H., Petukhova, V., Malchanau, A., Wijnhoven, K., & Fang, A. (2016). The Dialog-Bank. In LREC, pp. 3151–3158. find
  10. Bunt, H., Petukhova, V., Traum, D., & Alexandersson, J. (2017). Dialogue Act Annotation with the ISO 24617-2 Standard. In Multimodal Interaction with W3C Standards, pp. 109–135. Springer. ↗
  11. Carroll, J. M., & Tanenhaus, M. K. (1978). Functional Clauses and Sentence Segmentation. Journal of Speech, Language, and Hearing Research, 21 (4), 793–808. ↗
  12. Chen, Z., Yang, R., Zhao, Z., Cai, D., & He, X. (2018). Dialogue Act Recognition via CRF-Attentive Structured Network. In SIGIR, pp. 225–234.
  13. Chollet, F., et al. (2015). Keras: The Python Deep Learning Library. https://keras.io/. ↗
  14. Devlin, J., Chang, M.-W., Kenton, L., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT, Vol. 1, pp. 4171–4186.
  15. Fang, A., Cao, J., Bunt, H., & Liu, X. (2012). The Annotation of the Switchboard Corpus with the New ISO Standard for Dialogue Act Analysis. In Workshop on Interoperable Semantic Annotation, pp. 13–18. find
  16. Gambäck, B., Olsson, F., & Täckström, O. (2011). Active Learning for Dialogue Act Classification. In INTERSPEECH, pp. 1329–1332. find
  17. Godfrey, J. J., Holliman, E. C., & McDaniel, J. (1992). SWITCHBOARD: Telephone Speech Corpus for Research and Development. In ICASSP, Vol. 1, pp. 517–520. find
  18. Ji, Y., Haffari, G., & Eisenstein, J. (2016). A Latent Variable Recurrent Neural Network for Discourse Relation Language Models. In NAACL-HLT, Vol. 1, pp. 332–342. ↗
  19. Jurafsky, D., Shriberg, E., & Biasca, D. (1997). Switchboard SWBD-DAMSL Shallow-Discourse-Function Annotation Coder Manual. Tech. rep. Draft 13, Institute of Cognitive Science, University of Colorado. find
  20. Kalchbrenner, N., & Blunsom, P. (2013). Recurrent Convolutional Neural Networks for Discourse Compositionality. In Workshop on Continuous Vector Space Models and their Compositionality, pp. 119–126.
  21. Khanpour, H., Guntakandla, N., & Nielsen, R. (2016). Dialogue Act Classification in Domain-Independent Conversations Using a Deep Recurrent Neural Network. In COLING, pp. 2012–2021.
  22. Kingma, D. P., & Ba, J. (2015). Adam: A Method for Stochastic Optimization. In ICLR. ↗
  23. Kiritchenko, S., Matwin, S., & Famili, F. (2005). Functional Annotation of Genes using Hierarchical Text Categorization. In BioLINK SIG. find
  24. Král, P., & Cerisara, C. (2010). Dialogue Act Recognition Approaches. Computing and Informatics, 29 (2), 227–250. find
  25. Kruijff-Korbayová, I., Colas, F., Gianni, M., Pirri, F., de Greeff, J., Hindriks, K., Neerincx, M., Ögren, P., Svoboda, T., & Worst, R. (2015). TRADR Project: Long-Term Human-Robot Teaming for Robot Assisted Disaster Response. KI - Künstliche Intelligenz, 29 (2), 193–201. ↗
  26. Kumar, H., Agarwal, A., Dasgupta, R., & Joshi, S. (2018). Dialogue Act Sequence Labeling Using Hierarchical Encoder With CRF. In AAAI, pp. 3440–3447. ↗
  27. Lee, J. Y., & Dernoncourt, F. (2016). Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks. In NAACL-HLT, Vol. 2, pp. 515–520.
  28. Li, R., Lin, C., Collinson, M., Li, X., & Chen, G. (2019). A Dual-Attention Hierarchical Recurrent Neural Network for Dialogue Act Classification. In CoNLL, pp. 383–392. ↗
  29. Liu, Y., Han, K., Tan, Z., & Lei, Y. (2017). Using Context Information for Dialog Act Classification in DNN Framework. In EMNLP, pp. 2160–2168.
  30. Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed Representations of Words and Phrases and their Compositionality. In NIPS, pp. 3111–3119. ↗
  31. Nair, V., & Hinton, G. E. (2010). Rectified Linear Units Improve Restricted Boltzmann Machines. In ICML, pp. 807–814. find
  32. Pennington, J., Socher, R., & Manning, C. D. (2014). GloVe: Global Vectors for Word Representation. In EMNLP, pp. 1532–1543.
  33. Petukhova, V., Gropp, M., Klakow, D., Schmidt, A., Eigner, G., Topf, M., Srb, S., Motlicek, P., Potard, B., Dines, J., Deroo, O., Egeler, R., Meinz, U., & Liersch, S. (2014). The DBOX Corpus Collection of Spoken Human-Human and Human-Machine Dialogues. In LREC, pp. 252–258. find
  34. Raheja, V., & Tetreault, J. (2019). Dialogue Act Classification with Context-Aware Self-Attention. In NAACL, pp. 3727–3733. ↗
  35. Raux, A., Bohus, D., Langner, B., Black, A. W., & Eskenazi, M. (2006). Doing Research on a Deployed Spoken Dialogue System: One Year of Let’s Go! Experience. In IN-TERSPEECH, pp. 65–68. find
  36. Ravi, S., & Kozareva, Z. (2018). Self-Governing Neural Networks for On-Device Short Text Classification. In EMNLP, pp. 804–810. find
  37. Ribeiro, E., Ribeiro, R., & Martins de Matos, D. (2015). The Influence of Context on Dialogue Act Recognition. Computing Research Repository, arXiv:1506.00839. ↗
  38. Ribeiro, E., Ribeiro, R., & Martins de Matos, D. (2018a). A Study on Dialog Act Recognition using Character-Level Tokenization. In AIMSA, pp. 93–103. ↗
  39. Ribeiro, E., Ribeiro, R., & Martins de Matos, D. (2018b). End-to-End Multi-Level Dialog Act Recognition. In IberSPEECH, pp. 301–305. find
  40. Ribeiro, E., Ribeiro, R., & Martins de Matos, D. (2019a). A Multilingual and Multidomain Study on Dialog Act Recognition using Character-Level Tokenization. Information, 10 (3), 94. ↗
  41. Ribeiro, E., Ribeiro, R., & Martins de Matos, D. (2019b). Deep Dialog Act Recognition using Multiple Token, Segment, and Context Information Representations. Journal of Artificial Intelligence Research, 66, 861–899. ↗
  42. Ribeiro, E., Ribeiro, R., & Martins de Matos, D. (2020). Mapping the Dialog Act Annotations of the LEGO Corpus into ISO 24617-2 Communicative Functions. In LREC, pp. 531–539.
  43. Schmitt, A., Ultes, S., & Minker, W. (2012). A Parameterized and Annotated Spoken Dialog Corpus of the CMU Let’s Go Bus Information System. In LREC, pp. 3369–3373. find
  44. Searle, J. R. (1969). Speech Acts: An Essay in the Philosophy of Language. Cambridge University Press. ↗
  45. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research, 15 (1), 1929–1958. find
  46. Tran, Q. H., Zukerman, I., & Haffari, G. (2017a). A Generative Attentional Neural Network Model for Dialogue Act Classification. In ACL, Vol. 2, pp. 524–529.
  47. Tran, Q. H., Zukerman, I., & Haffari, G. (2017b). A Hierarchical Neural Model for Learning Sequences of Dialogue Acts. In EACL, Vol. 1, pp. 428–437.
  48. Tran, Q. H., Zukerman, I., & Haffari, G. (2017c). Preserving Distributional Information in Dialogue Act Classification. In EMNLP, pp. 2151–2156.
  49. Wan, Y., Yan, W., Gao, J., Zhao, Z., Wu, J., & Yu, P. S. (2018). Improved Dynamic Memory Network for Dialogue Act Classification with Adversarial Training. In IEEE International Conference on Big Data, pp. 841–850.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ribeiro-2020-automatic-recognition-general-purpose-communicative/