Elamur Research⌕
☰
home / papers / 2009 / Towards Unsupervised Recognition of Dialogue Acts

Towards Unsupervised Recognition of Dialogue Acts

Paper · 2009 · Nicole Novielli and Carlo Strapparava

Summary

The paper tackles automatic dialogue act (DA) labeling: deciding whether each utterance asks, states, opines, agrees, greets, thanks, and so on. The authors use only lexical information, with no prosody, so the method also suits written dialogue from the Web. They test it on two very different corpora annotated with a DAMSL-derived set of nine labels: the English Switchboard corpus (open-topic phone calls) and a small Italian Wizard-of-Oz corpus about healthy eating. Utterances are lemmatized into lemma#POS form and enriched with rule-based linguistic markers, such as signals of question type or first-person verbs. A supervised SVM serves as a reference. The main contribution is an unsupervised method. It builds an LSA space from the unlabeled corpus, represents each DA by a small set of hand-chosen seed words, and assigns each utterance the act whose seed vector is most similar. The unsupervised version comes reasonably close to the SVM and beats a random baseline by a wide margin in both languages. This suggests DA tagging is possible without annotated training data.

Key points

  • Data: Switchboard has 1155 conversations, about 205,000 utterances and 1.4M words, of which 131,265 utterances were used. The Italian corpus has 60 dialogues, 1448 user utterances and 15,500 words.
  • Nine labels: INFO-REQUEST, STATEMENT, S-OPINION, AGREE-ACCEPT, REJECT, OPENING, CLOSING, KIND-ATT and GEN-ANS. The SWBD-DAMSL tags were mapped automatically onto this set.
  • Supervised results: SVM-light with default settings on an 80/20 split reached micro F1 of .71 (Italian) and .77 (English). The most-frequent-label baselines are 37% and 57%.
  • Unsupervised results: LSA with 400 dimensions and the same seed definitions for both languages, scored on the same test split. Micro F1 was .66 (Italian) and .69 (English), against an 11% random baseline.
  • Main errors:
    • Too many utterances are labeled STATEMENT.
    • S-OPINION and STATEMENT are confused, since they differ mostly in evaluative wording.
    • The backchannel labels (GEN-ANS, AGREE-ACC, REJECT) are confused because yes/no/ok words are ambiguous.

Interesting ideas

  • I1. Each act is defined by a small seed set chosen only from the act's communicative purpose, with no domain vocabulary and minimal overlap between acts, so the definition carries across domains and languages.
  • I2. Before classification, cheap rule-based markers are derived from tagger and morphology output: WH-QTN, ASK-IF, I-PERS, COND, SUPER, AGR-EX, NAME and OR-CLAUSE. These turn shallow cues into explicit, named evidence for particular acts.
  • I3. An unsupervised method is scored on the very same test split as a supervised one, and each is compared with a suitable baseline (majority label vs. random). This shows how much labeled data actually buys.
  • I4. The error analysis shows that long utterances pick up cues of many acts and default to STATEMENT. It also shows that bare yes/no answers cannot be resolved without the previous turn, which argues for using a context window.
  • I5. Opinions differ from plain statements mainly through slanted, affect-loaded words, so polarity and subjectivity information could separate them.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): speech-act rules in prag could keep a small, inspectable cue lexicon per act, kept domain-free, so every act label traces back to a named cue.
  • I2 (frames): similar markers read off the UD tree (interrogative wh-dependents, subject-auxiliary inversion, first-person subjects, conditional mood) could feed speech-act detection as explainable features.
  • I3 (learn): when evaluating self-induced or rule-only components, run them on the same split as the trained model, with the right trivial baseline as a negative control.
  • I4 (frames): classify short replies such as yes/no/ok against the preceding utterance's act (e.g., the reply to an INFO-REQUEST is GEN-ANS), rather than by the words alone.
  • I5 (world): a lexicon of evaluative words could mark a clause as the speaker's opinion rather than a reported fact, keeping it out of the text's world state.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Bari Aldo MoroBari · Italy6 works · 2,210 citations🔬FBK Center for Information and Communication TechnologyTrento · Italy3 works · 117 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 3Research subjects and languages the work is about.English · 2,263 worksDialogue systems · 624 worksSpeech act theory · 229 works

Cite

Cite as Nicole Novielli and Carlo Strapparava. 2009. Towards Unsupervised Recognition of Dialogue Acts. Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Student Research Workshop and Doctoral Consortium.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{novielli2009unsupervised,
  title = {{Towards Unsupervised Recognition of Dialogue Acts}},
  author = {Nicole Novielli and Carlo Strapparava},
  booktitle = {Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Student Research Workshop and Doctoral Consortium},
  year = {2009},
  doi = {10.3115/1620932.1620947},
  url = {https://doi.org/10.3115/1620932.1620947}
}

References

23 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. J. Austin. 1962. How to do Things with Words. Oxford University Press, New York. find
  2. M. Berry. 1992. Large-scale sparse singular value computations. International Journal of Supercomputer Applications, 6(1). ↗
  3. G. Clarizio, I. Mazzotta, N. Novielli, and F. deRosis. 2006. Social attitude towards a conversational character. In Proceedings of the 15th IEEE International Symposium on Robot and Human Interactive Communication, pages 2–7, Hatfield, UK, September. find
  4. M. Core and J. Allen. 1997. Coding dialogs with the DAMSL annotation scheme. In Working Notes of the AAAI Fall Symposium on Communicative Action in Humans and Machines, Cambridge, MA. find
  5. A. Gliozzo and C. Strapparava. 2005. Domains kernels for text categorization. In Proceedengs of (CoNLL-2005), University of Michigan, Ann Arbor, June. find
  6. J. Godfrey, E. Holliman, and J. McDaniel. 1992. SWITCHBOARD: Telephone speech corpus for research and development. In Proceedings of ICASSP-92, pages 517–520, San Francisco, CA. IEEE. find
  7. S. Grau, E. Sanchis, M. J. Castro, and D. Vilar. 2004. Dialogue act classification using a bayesian approach. In Proceedings of SPECOM-04, pages 495–499, Saint-Petersburg, Russia, September. find
  8. T. Joachims. 1998. Text categorization with Support Vector Machines: learning with many relevant features. In Proceedings of the European Conference on Machine Learning. ↗
  9. D. Jurafsky, E. Shriberg, and D. Biasca. 1997. Switchboard SWBD-DAMSL shallow-discourse-function annotation coders manual, draft 13. Technical Report 97-01, University of Colorado. find
  10. S. Keizer, R. op den Akker, and A. Nijholt. 2002. Dialogue act recognition with bayesian networks for dutch dialogues. In K. Jokinen and S. McRoy, editors, Proceedings 3rd SIGdial Workshop on Discourse and Dialogue, pages 88–94, Philadelphia, PA, July.
  11. T. K. Landauer, P. Foltz, and D. Laham. 1998. Introduction to latent semantic analysis. Discourse Processes, 25. find
  12. A. Martalo, N. Novielli, and F. deRosis. 2008. Attitude display in dialogue patterns. In AISB 2008 Convention on Communication, Interaction and Social Intelligence, Aberdeen, Scotland, April. find
  13. E. Pianta, C. Girardi, and R. Zanoli. 2008. The TextPro tool suite. In Proceedings of LREC, Marrakech (Morocco), May. find
  14. M. Poesio and A. Mikheev. 1998. The predictive power of game structure in dialogue act recognition: Experimental results using maximum entropy estimation. In Proceedings of ICSLP-98, Sydney, December. find
  15. R. Power. 1979. The organisation of purposeful dialogues. Linguistics, 17:107–152. ↗
  16. H. Sacks, E. Schegloff, and G. Jefferson. 1974. A simplest systematics for the organization of turn-taking for conversation. Language, 50(4):696–735. ↗
  17. E. Schegloff. 1968. Sequencing in conversational openings. American Anthropologist, 70:1075–1095. find
  18. J. Searle. 1969. Speech Acts: An Essay in the Philosophy of Language. Cambridge University Press, Cambridge, London. ↗
  19. A. Stolcke, N. Coccaro, R. Bates, P. Taylor, C. Van Ess-Dykema, K. Ries, E. Shriberg, D. Jurafsky, R. Martin, and M. Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics, 26(3):339–373. ↗
  20. D. Traum. 2000. 20 questions for dialogue act taxonomies. Journal of Semantics, 17(1):7–30. find
  21. V. Vapnik. 1995. The Nature of Statistical Learning Theory. Springer-Verlag. ↗
  22. A. Venkataraman, Y. Liu, E. Shriberg, and A. Stolcke. 2005. Does active learning help automatic dialog act tagging in meeting data? In Proceedings of EUROSPEECH-05, Lisbon, Portugal. find
  23. V. Warnke, R. Kompe, H. Niemann, and E. Nöth. 1997. Integrated dialog act segmentation and classification using prosodic features and language models. In Proceedings of 5th European Conference on Speech Communication and Technology, volume 1, pages 207– 210, Rhodes, Greece. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/novielli-2009-unsupervised-recognition-dialogue-acts/