Elamur Research⌕
☰
home / papers / 2011 / Incremental dialogue act understanding

Incremental dialogue act understanding

Paper · 2011 · Volha Petukhova and Harry Bunt

Summary

The paper asks whether a dialogue system can work out what a speaker is doing with an utterance (questioning, giving feedback, grabbing the turn, stalling) while the words are still arriving, instead of after the utterance ends. The authors treat unsegmented transcribed speech as a token stream. Each token gets a label that combines a segment-position prefix with a DIT++ communicative function, so segmentation and classification happen in a single step. There is one classifier per DIT++ dimension.

Local classifiers rely on lexical, prosodic, timing and dialogue-history features. Global classifiers then add the local predictions for preceding tokens, and in a later setup also for following tokens up to the segment edge. Training data are three AMI meetings, and the tested learners are BayesNet and Ripper. The global stage brings per-dimension F-scores to roughly the 70–98 range, and MapTask results are comparable.

This matters because the meaning of a segment is almost ready when its last token arrives. A system could therefore update its information state and respond immediately, and the recognized function could help later parsing and anaphora resolution.

Key points

  • Training data: three AMI meetings with 17,335 tokens forming 3,897 functional segments, described by 1,668 features (prosody, timing, lexical n-grams, tags of the 10 previous turns).
  • Labels follow DIT++ Release 4:
    • 10 dimensions, with 38 dimension-specific and 44 general-purpose functions;
    • each token gets a B/I/E/O/BE prefix plus its function.
  • With global classification per dimension:
    • Ripper reaches F1 of 86.1 (Task), 95.1 (Auto-Feedback) and 98.6 (Social Obligation Management);
    • Partner Communication Management stays weakest (about 70–72).
  • On MapTask (with the LIRICS variant of the tagset), global F-scores run from 66.7 to 91.2. Human expert agreement on that corpus was kappa 0.92–1.00.
  • A functional segment averages no more than 4.4 tokens.

Interesting ideas

  • I1. The system tags every token with both its position in a segment and its function, so finding the boundaries and naming the act happen together rather than in a pipeline.
  • I2. A second-stage classifier takes the first stage's predictions on neighbouring tokens as features, which lets context fix locally implausible labels.
  • I3. The authors deliberately delay committing to a label until predictions for the rest of the segment exist, so a lower-ranked option can win once later evidence arrives.
  • I4. Function is analysed separately in each dimension, so one segment can be, say, Task-level and Turn-Management-level at once.
  • I5. Acts like backchannels, stalling or thanking carry almost no semantic content, so their function alone is enough to act on.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): prag could label speech acts token by token over raw dialogue text, without first waiting for a sentence splitter to commit to boundaries.
  • I2 (learn): the same stacking pattern could feed witness-parser outputs on nearby tokens into a repair model, with each correction traceable to the specific neighbour predictions that triggered it.
  • I3 (tree): gparse or the perceptron parser could keep several candidate heads or labels open until the right context comes in, rather than locking in the top choice greedily.
  • I4 (frames): speech-act frames in prag could hold several simultaneous functions per span (content plus feedback or turn-taking), which avoids forcing one label per sentence.
  • I5 (world): such spans could update discourse state, for example acknowledgment or turn change, without building full entity or event structures, saving effort for content-bearing segments.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tilburg UniversityTilburg · Netherlands40 works · 6,231 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 2Research subjects and languages the work is about.Dialogue systems · 401 worksSpeech act theory · 160 works

Cite

Cite as Volha Petukhova and Harry Bunt. 2011. Incremental dialogue act understanding. Proceedings of the Ninth International Conference on Computational Semantics (IWCS 2011).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{petukhova2011incremental,
  title = {{Incremental dialogue act understanding}},
  author = {Volha Petukhova and Harry Bunt},
  booktitle = {Proceedings of the Ninth International Conference on Computational Semantics (IWCS 2011)},
  year = {2011},
  url = {https://aclanthology.org/W11-0125/}
}

References

28 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aist, G., J. Allen, E. Campana, C. Gomez Gallo, S. Stoness, M. Swift, and M. K. Tanenhaus (2007). Incremental understanding in human-computer dialogue and experimental evidence for advantages over nonincremental methods. In Proceedings of the 11th Workshop on the Semantics and Pragmatics of Dialogue, Trento, Italy, pp. 149–154. find
  2. Ang, J., Y. Liu, and E. Shriberg (2005). Automatic dialog act segmentation and classification in multiparty meetings. In Proceedings of the ICASSP, Volume vol. 1, Philadelphia, USA, pp. 10611064. find
  3. Bos, J. (2002). Underspecification and resolution in discourse semantics. PhD Thesis. Saarbrücken: Saarland University. find
  4. Bunt, H. (2007). Semantic underspecification: which techniques for what purpose? In Computing Meaning, Vol. 3, pp. 55–85. Dordrecht: Springer. find
  5. Bunt, H. (2009). The DIT++ taxonomy for functional dialogue markup. In Proceedings of the AAMAS 2009 Workshop ‘Towards a Standard Markup Language for Embodied Dialogue Acts’ (EDAML 2009), Budapest. find
  6. Bunt, H. (2010). Multifunctionality in dialogue and its interpretation. Computer, Speech and Language, Special issue on dialogue modeling. find
  7. Bunt, H., J. Alexandersson, J. Carletta, J.-W. Choe, A. Fang, K. Hasida, K. Lee, V. Petukhova, A. Popescu-Belis, L. Romary, C. Soria, and D. Traum (2010). Language resource management – Semantic annotation framework – Part 2: Dialogue acts. ISO DIS 24617-2. Geneva: ISO Central Secretariat. find
  8. Bunt, H., V. Petukhova, and A. Schiffrin (2007). Lirics deliverable d4.4. multilingual test suites for semantically annotated data. Available at http://lirics.loria.fr. ↗
  9. Cohen, W. (1995). Fast effective rule induction. In Proceedings of the 12th International Conference on Machine Learning (ICML’95), pp. 115–123. ↗
  10. DeVault, D. and M. Stone (2003). Domain inference in incremental interpretation. In Proceedings of the Workshop on Inference in Computational Semantics, INRIA Lorraine, Nancy, France. find
  11. Dietterich, T. (2002). Machine learning for sequential data: a review. In Proceedings of the Joint IAPR International Workshop on Structural, Syntactic, and Statistical Pattern Recognition, pp. 15–30. find
  12. Geertzen, J. (2009). Dialogue act recognition and prediction: exploration in computational dialogue modelling. The Netherlands: Tilburg University. find
  13. Geertzen, J., V. Petukhova, and H. Bunt (2007, September). A multidimensional approach to utterance segmentation and dialogue act classification. In Proceedings of the 8th SIGdial Workshop on Discourse and Dialogue, Antwerp, Belgium, pp. 140–149. Association for Computational Linguistics.
  14. Haddock, N. (1989). Computational models of incremental semantic interpretation. Language and Cognitive Processes Vol. 14 (3), SI337–SI380. ↗
  15. Hobbs, J. (1985). Ontological promiscuity. In Proceedings 23rd Annual Meeting of the ACL, Chicago, pp. 61–69. find
  16. Lendvai, P., v. d. A. Bosch, and E. Krahmer (2003). Machine learning for shallow interpretation of user utterances in spoken dialogue systems. In Proceedings of EACL-03 Workshop on Dialogue Systems: interaction, adaptation and styles of management, Budapest. find
  17. Lendvai, P. and J. Geertzen (2007). Token-based chunking of turn-internal dialogue act sequences. In Proceedings of the 8th SIGdial Workshop on Discourse and Dialogue, Antwerp, Belgium, pp. 174–181.
  18. Milward, D. and R. Cooper (2009). Incremental interpretation: applications, theory, and relationship to dynamic semantics. In Proceedings COLING 2009, Kyoto, Japan, pp. 748–754. find
  19. Nakano, M., N. Miyazaki, J. Hirasawa, K. Dohsaka, and T. Kawabata (1999). Understanding unsegmented user utterances in real-time spoken dialogue systems. In Proceedings of the 37th Annual Conference of the Association of Computational Linguistics, ACL, pp. 200–207. find
  20. Petukhova, V. and H. Bunt (2009). Who’s next? speaker-selection mechanisms in multiparty dialogue. In Proceedings of the Workshop on the Semantics and Pragmatics of Dialogue, Stockholm,, pp. 19–26. find
  21. Pinkal, M. (1999). On semantic underspecification. In Computing Meaning, Vol. 1, pp. 33–56. Dordrecht: Kluwer. ↗
  22. Poesio, M. and D. Traum (1998). Towards an Axiomatization of Dialogue Acts. In Proceedings of the Twente Workshop on the Formal Semantics and Pragmatics of Dialogue, Twente, pp. 309–347. find
  23. Sedivy, J. (2003). Pragmatic versus form-based accounts of referential contrast: Evidence for effects of informativity expectations. Journal of Psycolinguistic Research 32(1), 3–23. ↗
  24. Sedivy, J., M. Tanenhaus, C. Chambers, and G. Carlson (1999). Achieving incremental semantic interpretation through contextual representation. Cognition 71, 109–147. ↗
  25. Tanenhaus, M., M. Spivey-Knowlton, K. Eberhard, and J. Sedivy (1995). Intergration of visual and linguistic information in spoken language comprehension. Science 268, 1632–1634. find
  26. Tomita, M. (1986). Efficient parsing for natural language. Dordrecht: Kluwer. ↗
  27. Van den Bosch, A. (1997). Learning to pronounce written words: A study in inductive language learning. PhD thesis. The Netherlands: Maastricht University. ↗
  28. Zimmermann, M., Y. Lui, E. Shriberg, and A. Stolcke (2005). Toward joint segmentation and classification of dialog acts in multiparty meetings. In Proceedings of the Multimodal Interaction and Related Machine Learning Algorithms Workshop (MLMI05), pp. 187–193. Springer. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/petukhova-2011-incremental-dialogue-act-understanding/