Elamur Research⌕
☰
home / papers / 2013 / Dialogue Act Recognition in Synchronous and Asynchronous Conversations

Dialogue Act Recognition in Synchronous and Asynchronous Conversations

Summary

The paper asks whether one supervised dialogue act (DA) recognizer can work across very different conversation types. Earlier work usually focused on a single domain. The authors evaluate on four corpora: two asynchronous (email, forum) and two synchronous (meeting, phone). They use a deliberately domain-independent feature set made up of unigram frequencies, utterance length, speaker identity and relative position within a post.

Conversational structure is modeled differently by modality. Spoken data uses plain temporal order. Written threads use reply links or a Fragment Quotation Graph, following Joty et al. (2011).

Three learners are compared with 5-fold cross-validation against a majority-class baseline:

  • SVM-multiclass, which ignores sequence
  • SVM-hmm, a structured sequence model
  • CRF, also a structured sequence model

Both sequence models beat SVM-multiclass everywhere, and SVM-hmm is the strongest on every corpus. Results are better on synchronous conversations, which the authors attribute to their simpler sequential structure, and email proves hardest. The study matters because it gives an early cross-modality benchmark and shows that sequence-aware models are worthwhile regardless of domain.

Key points

  • Corpora used:
    • BC3: 40 W3C email threads, 12 DAs
    • CNET: 320 forum threads and 1332 posts, 11 DAs, labeled per post
    • ICSI-MRDA: 75 meetings, reduced to 11 general tags
    • Switchboard-DAMSL (SWBD): 220 tags, mapped to 16 via Jeong (2009)
  • SVM-hmm micro-accuracy compared with the majority baseline:
    • BC3: 77.75 vs. 69.56
    • CNET: 58.7 vs. 36.75
    • MRDA: 80.5 vs. 66.47
    • SWBD: 74.32 vs. 46.44
  • CRF consistently placed second, with micro-accuracy of 72.18 (BC3), 40.3 (CNET), 77.8 (MRDA) and 73.04 (SWBD). SVM-multiclass barely improved on the baseline.
  • Macro-accuracy remained low overall (SVM-hmm best: 17.1–32.4). Rare classes were mostly misclassified as statements, including polite closings, rhetorical questions, self-talk and signal-non-understanding.
  • Tools were SVM-light packages for the two SVM models and Mallet for CRF. Planned future work includes POS/syntactic features, prosodic features and semi-supervised learning.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Speech act theory

Cited by

23 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 17 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{tavafi2013dialogue,
  title = {{Dialogue Act Recognition in Synchronous and Asynchronous Conversations}},
  author = {Tavafi, Maryam and Mehdad, Yashar and Joty, Shafiq and Carenini, Giuseppe and Ng, Raymond},
  booktitle = {Proceedings of the SIGDIAL 2013 Conference},
  year = {2013},
  doi = {10.14288/1.0052186},
  url = {https://doi.org/10.14288/1.0052186}
}

References

20 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Congkai Sun and Louise-Philippe Morency. 2012. Dialogue Act Recognition using Reweighted Speaker Adaptation. 13th Annual SIGdial Meeting on Discourse and Dialogue. find
  2. Dan Jurafsky, Elizabeth Shriberg, and Debra Biasca. 1997. Switchboard SWBD-DAMSL labeling project coder’s manual, draft 13. Technical report, Univ. of Colorado Institute of Cognitive Science. find
  3. Elizabeth Shriberg, Raj Dhillon, Sonali Bhagat, Jeremy Ang, and Hannah Carvey. 2004. The ICSI Meeting Recorder Dialog Act (MRDA) Corpus. HLT-NAACL SIGDIAL Workshop. ↗ our card
  4. Gabriel Murray, Giuseppe Carenini, and Raymond T. Ng. 2010. Generating and validating abstracts of meeting conversations: a user study. INLG’10. find
  5. Ioannis Tsochantaridis, Thomas Hofmann, Thorsten Joachims, and Yasemin Altun. 2004. Support vector machine learning for interdependent and structured output spaces. Proceedings of the 21st International Conference on Machine Learning (ICML). find
  6. Jan Ulrich, Gabriel Murray, and Giuseppe Carenini. 2008. A publicly available annotated corpus for supervised email summarization. EMAIL’08 Workshop. AAAI. find
  7. John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. Intl. Conf. on Machine Learning. find
  8. Koby Crammer and Yoram Singer. 2001. On the algorithmic implementation of multiclass kernel-based vector machines. Journal of Machine Learning Research. find
  9. Lari Carlson. 1983. Dialogue Games: An Approach to Discourse Analysis. D. Reidel. find
  10. Lokesh Shrestha and Kathleen McKeown. 2004. Detection of question-answer pairs in email conversations. Proceedings of the 20th Biennial Int. Conf. on Computational Linguistics. find
  11. Minwoo Jeong, Chin-Yew Lin, and Gary G. Lee. 2009. The Semi-supervised speech act recognition in emails and forums. Proceedings of the 2009 Conf. Empirical Methods in Natural Language Processing. find
  12. Oliver Ferschke, Iryna Gurevych, and Yevgen Chebotar. 2012. Behind the Article: Recognizing Dialog Acts in Wikipedia Talk Pages. Proceedings of the 13th Conference of the European Chapter of the ACL. find our card
  13. Rajesh Ranganath, Dan Jurafsky, and Dan Mcfarland. 2009. Its not you, its me: Detecting flirting and its misperception in speed-dates. EMNLP-09. find
  14. S. S. Keerthi and S. Sundararajan. 2007. CRF versus SVM-Struct for sequence labeling. Technical report, Yahoo Research. find
  15. Shafiq R. Joty, Giuseppe Carenini, and Chin-Yew Lin. 2011. Unsupervised modeling of dialog acts in asynchronous conversations. IJCAI. find
  16. Su N. Kim, Lawrence Cavedon, and Timothy Baldwin. 2010a. Classifying dialogue acts in one-on-one live chats. EMNLP’10. find our card
  17. Su N. Kim, Li Wang, and Timothy Baldwin. 2010b. Tagging and linking web forum posts. Proceedings of the Fourteenth Conference on Computational Natural Language Learning, CoNLL ’10. find
  18. Sujith Ravi and Jihie Kim. 2007. Profiling student interactions in threaded discussions with speech act classifiers. AIED’07, LA, USA. find
  19. Vitor R. Carvalho and William W. Cohen. 2005. On the collective classification of email "speech acts". Proceedings of the 31st Annual Int. ACM SIGIR Conf. on Research and Development in Information Retrieval. find
  20. Yasemin Altun and Ioannis Tsochantaridis and Thomas Hofmann. 2003. Hidden Markov Support Vector Machines. Proceedings of the 20th International Conference on Machine Learning. find

↑ back to top

This page in Markdown →