Elamur Research⌕
☰
home / papers / 2016 / Speech Act Modeling of Written Asynchronous Conversations with Task-Specific Embeddings and Conditional Structured Models

Speech Act Modeling of Written Asynchronous Conversations with Task-Specific Embeddings and Conditional Structured Models

Summary

The paper addresses sentence-level speech act tagging in written asynchronous conversations such as forum threads and emails. In these threads, posts are not strictly sequential, and reply-to structure is often missing. Earlier systems tend to rely on sparse bag-of-words features and label each sentence independently. The authors propose a two-stage framework instead. First, an LSTM recurrent network, either unidirectional or bidirectional, with random or word2vec initialization, builds task-specific sentence embeddings that capture word order. Second, a pairwise Conditional Random Field over an arbitrary conversation graph uses these embeddings to label all sentences jointly. Several graph designs model links within and across comments, and inference uses loopy belief propagation. Models were trained on both synchronous and asynchronous corpora, and evaluation used a pair of forum datasets that includes the newly released QC3 corpus. LSTM encodings outperform BOW features, and bidirectional encoders beat unidirectional ones. Joint CRF models improve on local classifiers, but only when the graph structure fits the conversation. This matters for downstream uses such as thread summarization and question answering.

Key points

  • Labels include Statement, Question, Suggestion, Polite and Response, following a standard speech act tagset. Evaluation uses two forum datasets, QC3 and TA.
  • A new forum corpus, QC3, annotated with speech acts, was released together with source code at alt.qcri.org/tools/speech-act/.
  • A plain linear-chain CRF in temporal order (CRF(LC-LC)) gives little gain. CRF(LC-LC1), which chains sentences inside each comment and links them to the first comment, significantly beats the local B-LSTM_p classifier at 99% confidence.
  • Connecting to every sentence of the first comment (CRF(LC-FC1)) lowers performance. A fully connected CRF(FC-FC) performs on par with the best model.
  • LSTM representations need more training data. Future work aims to train the LSTM and CRF end-to-end and to exploit explicit thread structure, such as email quotation graphs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Speech act theory

Cited by

9 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 8 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p16-1165

@inproceedings{joty2016speech,
  title = {{Speech Act Modeling of Written Asynchronous Conversations with Task-Specific Embeddings and Conditional Structured Models}},
  author = {Joty, Shafiq and Hoque, Enamul},
  booktitle = {Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2016},
  doi = {10.18653/v1/p16-1165},
  url = {https://doi.org/10.18653/v1/p16-1165}
}

References

31 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. John Langshaw Austin. 1962. How to do things with words. Harvard University Press. find
  2. Srinivas Bangalore, Giuseppe Di Fabbrizio, and Amanda Stent. 2006. Learning the structure of task-driven human-human dialogs. In Proceedings of the 44nd Annual Meeting on Association for Computational Linguistics, ACL’06, pages 201–208. ACL. find
  3. Sumit Bhatia, Prakhar Biyani, and Prasenjit Mitra. 2014. Summarizing online forum discussions – can dialog acts of individual messages help? In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2127–2131, Doha, Qatar, October. ACL. find our card
  4. Giuseppe Carenini, Raymond T. Ng, and Xiaodong Zhou. 2008. Summarizing emails with conversational cohesion and subjectivity. In Proceedings of the 46nd Annual Meeting on Association for Computational Linguistics, ACL’08, pages 353–361, OH. ACL. find
  5. Vitor R. Carvalho and William W. Cohen. 2005. On the collective classification of email ”speech acts”. In SIGIR ’05: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, pages 345– 352, New York, NY, USA. ACM Press. find
  6. William W. Cohen, Vitor R. Carvalho, and Tom M. Mitchell. 2004. Learning to classify email into “speech acts”. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 309–316. find our card
  7. Rajdip Dhillon, Sonali Bhagat, Hannah Carvey, and Elizabeth Shriberg. 2004. Meeting Recorder Project: Dialog Act Labeling Guide. Technical report, ICSI Tech. Report. ↗
  8. Donghui Feng, Erin Shaw, Jihie Kim, and Eduard Hovy. 2006. Learning to detect conversation focus of threaded discussions. In Proceedings of the Main Conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, HLT-NAACL ’06, pages 208–215, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  9. Kevin Gimpel, Nathan Schneider, Brendan O’Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah A Smith. 2011. Part-of-speech tagging for twitter: Annotation, features, and experiments. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2, pages 42–47. ACL. ↗
  10. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. ↗
  11. Liangjie Hong and Brian D. Davison. 2009. A classification-based approach to question answering in discussion boards. In 32nd Annual International ACM SIGIR Conference on Research and Development on Information Retrieval, pages 171–178, Boston, USA. ACM Press. find
  12. Ozan Irsoy and Claire Cardie. 2014. Opinion mining with deep recurrent neural networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 720–728, Doha, Qatar. ACL. find
  13. Minwoo Jeong, Chin-Yew Lin, and Gary Geunbae Lee. 2009. Semi-supervised speech act recognition in emails and forums. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1250–1259, Singapore. ACL. find
  14. Shafiq Joty, Giuseppe Carenini, and Chin-Yew Lin. 2011. Unsupervised Modeling of Dialog Acts in Asynchronous Conversations. In Proceedings of the twenty second International Joint Conference on Artificial Intelligence, IJCAI’11, pages 1–130, Barcelona. find
  15. Dan Jurafsky, Liz Shriberg, and Debra Biasca. 1997. Switchboard SWBD-DAMSL Shallow-Discourse-Function Annotation Coders Manual, Draft 13. Technical report, University of Colorado at Boulder & +SRI International. find
  16. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. CoRR, abs/1412.6980. ↗
  17. John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Conference on Machine Learning, ICML ’01, pages 282–289, San Francisco, CA. find
  18. Jiwei Li, Thang Luong, Dan Jurafsky, and Eduard Hovy. 2015. When are tree structures necessary for deep learning of representations? In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2304– 2314, Lisbon, Portugal, September. ACL. find
  19. Pengfei Liu, Shafiq Joty, and Helen Meng. 2015. Fine-grained opinion mining with recurrent neural networks and word embeddings. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1433–1443, Lisbon, Portugal, September. ACL. find
  20. Kathleen McKeown, Lokesh Shrestha, and Owen Rambow. 2007. Using question-answer pairs in extractive summarization of email conversations. In CI-CLing, pages 542–550. find
  21. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. ↗
  22. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013b. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, pages 3111–3119. ↗
  23. Kevin P. Murphy, Yair Weiss, and Michael I. Jordan. 1999. Loopy belief propagation for approximate inference: An empirical study. In Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, UAI’99, pages 467–475, Stockholm, Sweden. Morgan Kaufmann Publishers Inc. find
  24. Kevin Murphy. 2012. Machine Learning A Probabilistic Perspective. The MIT Press. find
  25. Tatsuro Oya and Giuseppe Carenini. 2014. Extractive summarization and dialogue act modeling on email threads: An integrated probabilistic approach. In Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), page 133–140, Philadelphia, PA, U.S.A. ACL. find our card
  26. Sebastian Padó, 2006. User’s guide to sigf: Significance testing by approximate randomisation. find
  27. Judea Pearl. 1988. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA. find
  28. Ashequl Qadir and Ellen Riloff. 2011. Classifying sentences as speech acts in message board posts. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 748–758, Edinburgh, Scotland, UK. Association for Computational Linguistics. Dialogue (SIGDIAL), page 117–121, Metz, France. ACL. find our card
  29. Jan Ulrich, Gabriel Murray, and Giuseppe Carenini. 2008. A publicly available annotated corpus for supervised email summarization. In AAAI’08 EMAIL Workshop, Chicago, USA. AAAI. find
  30. S. V. N. Vishwanathan, Nicol N. Schraudolph, Mark W. Schmidt, and Kevin P. Murphy. 2006. Accelerated training of conditional random fields with stochastic gradient methods. In Proceedings of the 23rd International Conference on Machine Learning, ICML ’06, pages 969–976, Pittsburgh, USA. ACM. find
  31. Yair Weiss. 2001. Comparing the mean field method and belief propaga- tion for approximate inference in mrfs. In Advanced Mean Field Methods. MIT Press. A find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/joty-2016-speech-act-modeling-written-asynchronous/