Elamur Research⌕
☰
home / papers / 2005 / Dialogue Act Tagging for Instant Messaging Chat Sessions

Dialogue Act Tagging for Instant Messaging Chat Sessions

Summary

This paper looks at automatic dialogue act classification for Instant Messaging (IM) conversations. Most earlier work used transcribed speech, and typed chat differs from it in important ways. The author collected nine task-oriented IM dialogues between customers and live shopping assistants on the MSN Shopping service. Three annotators labelled them with a reduced set of 12 dialogue act tags, adapted from the DAMSL-based inventory of Stolcke et al. (2000).

Two problems specific to IM are addressed. First, a single message may contain several utterances, which here were segmented by hand. Second, participants often reply before the other person has finished a turn, so messages arrive out of order. To fix the ordering, the author uses timestamps and typing speed to work out which earlier message each reply depends on.

Classification uses a bag-of-words naive Bayes model, with the dialogue act prior replaced by n-gram probabilities over previous tags. The bigram variant performs best, with about 81.6% mean accuracy. This shows that simple lexical features combined with adjacency-pair context work well for chat. The result is relevant to automated or assisted customer-support agents.

Key points

  • The corpus contains 9 annotated customer–assistant IM dialogues from MSN Shopping. The 12 tags were derived from Stolcke et al.'s 42 DAMSL-based tags, and the most frequent is STATEMENT at 36%. Inter-annotator agreement was 89%, with a Kappa of 0.87.
  • About 20% of messages were out of sync. A typing-rate threshold of 5 characters/second (the 90th percentile over roughly 300 hand-labelled messages) raised dependency detection accuracy from an 80% baseline to 94.2%.
  • In 9-fold cross-validation, the majority-class baseline was 36.0%. Naive Bayes likelihood alone reached 80.1%, and adding n-gram priors gave 80.6% (unigram), 81.6% (bigram) and 80.9% (trigram).
  • Roughly 40% of messages contained more than one utterance. Automatic utterance segmentation was left for future work.
  • Proposed improvements include word-order, part-of-speech and phrase-structure features, and dialogue macrogame theory for handling nested question–answer exchanges.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Speech act theory

Cited by

14 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 13 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{ivanovic2005dialogue,
  title = {{Dialogue Act Tagging for Instant Messaging Chat Sessions}},
  author = {Ivanovic, Edward},
  booktitle = {Proceedings of the ACL Student Research Workshop},
  year = {2005},
  doi = {10.3115/1628960.1628976},
  url = {https://doi.org/10.3115/1628960.1628976}
}

References

12 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. John L. Austin. 1962. How to do Things with Words. Clarendon Press, Oxford. find
  2. Jean Carletta. 1996. Assessing agreement on classification tasks: the kappa statistic. Computational Linguistics, 22(2):249–254. ↗ our card
  3. Mark Core and James Allen. 1997. Coding dialogs with the DAMSL annotation scheme. Working Notes of the AAAI Fall Symposium on Communicative Action in Humans and Machines, pages 28–35. find
  4. Thomas M. Cover and Joy A. Thomas. 1991. Elements of Information Theory. Wiley, New York. find
  5. Daniel Jurafsky, Rebecca Bates, Noah Coccaro, Rachel Martin, Marie Meteer, Klaus Ries, Elizabeth Shriberg, Andreas Stolcke, Paul Taylor, and Carol Van Ess-Dykema. 1997. Automatic detection of discourse structure for speech recognition and understanding. Proceedings of the 1997 IEEE Workshop on Speech Recognition and Understanding, pages 88–95. ↗
  6. Kenji Kita, Yoshikazu Fukui, Masaaki Nagata, and Tsuyoshi Morimoto. 1996. Automatic acquisition of probabilistic dialogue models. Proceedings of the Fourth International Conference on Spoken Language, 1:196–199. find
  7. Lori Levin, Klaus Ries, Ann Thyme-Gobbel, and Alon Lavie. 1999. Tagging of speech acts and dialogue games in spanish call home. Towards Standards and Tools for Discourse Tagging (Proceedings of the ACL Workshop at ACL’99), pages 42–47. ↗ our card
  8. William Mann. 2002. Dialogue macrogame theory. Proceedings of the 3rd SIGdial Workshop on Discourse and Dialogue, pages 129–141. find
  9. John R. Searle. 1979. Expression and Meaning: Studies in the Theory of Speech Acts. Cambridge University Press, Cambridge, UK. find
  10. Sidney Siegel and N. John Castellan, Jr. 1988. Nonparametric statistics for the behavioral sciences. McGraw-Hill, second edition. find
  11. Andreas Stolcke, Noah Coccaro, Rebecca Bates, Paul Taylor, Carol Van Ess-Dykema, Klaus Ries, Elizabeth Shriberg, Daniel Jurafsky, Rachel Martin, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics, 26(3):339–373. ↗ our card
  12. Hideki Tanaka and Akio Yokoo. 1999. An efficient statistical speech act type tagging system for speech translation systems. In Proceedings of the 37th conference on Association for Computational Linguistics, pages 381–388. Association for Computational Linguistics. find

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ivanovic-2005-dialogue-act-tagging-instant-messaging/