Elamur Research⌕
☰
home / papers / 2011 / Classifying Sentences as Speech Acts in Message Board Posts

Classifying Sentences as Speech Acts in Message Board Posts

Summary

Message board posts mix factual, expository content with conversational moves aimed at readers, such as greetings, requests for help and promises to follow up. Qadir and Riloff tackle two tasks on this hybrid genre. The first is separating sentences that carry a speech act from purely informative ones. The second is assigning speech act sentences to four of Searle's (1976) categories: Commissives, Directives, Expressives and Representatives. Declarations were left out because they almost never occurred.

The authors wrote annotation guidelines adapted to forum language and labelled posts from a veterinary medicine message board. They then trained sentence classifiers on unigrams, a compact set of lexical and syntactic cues, lists of speech act terms drawn from outside sources, and semantic class features tailored to the domain.

Speech act filtering, Directives and Expressives were recognised well, although simple heuristics such as question marks also work for these. Commissives benefited most from the richer features, while Representatives remained the hardest class. The features rely only on POS tags and, apart from the semantic ones, are domain-independent. This makes the approach practical for noisy web text and useful for information extraction, question tracking and summarisation.

Key points

  • Data: message board posts from the veterinary medicine domain, annotated at the sentence level with four Searle classes; Representatives were rare, covering only 2.91% of sentences, and had the lowest inter-annotator agreement.
  • A reduced "minimal lexsyn" set of 10 features worked better than the full lexical/syntactic set. It comprises 4 Tense+Person features, Early Punctuation, a sentence-initial Modal/Verb feature and 4 Sentence Position features.
  • Semantic features for TEST and DRUG categories came from a Basilisk lexicon and a semantic tagger. Together they raised Commissive F-score from 41% to 48%, likely because tests and drugs stand in for the doctor's future actions.
  • Adding speech act clue words gave small recall gains for Expressives (+3%) and Representatives (+2%) but slightly hurt Commissives.
  • The method requires no parsing, only part-of-speech tags. Proposed future work includes segmenting sentences into clauses and extracting the exact speech act expressions.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Speech act theory · Part-of-speech tagging

Cited by

11 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 9 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{qadir2011classifying,
  title = {{Classifying Sentences as Speech Acts in Message Board Posts}},
  author = {Qadir, Ashequl and Riloff, Ellen},
  booktitle = {Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing},
  year = {2011},
  url = {https://aclanthology.org/D11-1069/}
}

References

21 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. James Allen. 1995. Natural language understanding (2nd ed.). Benjamin-Cummings Publishing Co., Inc., Redwood City, CA, USA. find
  2. Jean Carletta. 1996. Assessing agreement on classification tasks: the kappa statistic. Comput. Linguist., 22:249–254, June. ↗ our card
  3. Vitor R. Carvalho and William W. Cohen. 2005. On the collective classification of email ”speech acts”. In SI-GIR ’05: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, pages 345–352, New York, NY, USA. ACM Press. find
  4. Vitor R. Carvalho and William W. Cohen. 2006. Improving ”email speech acts” analysis via n-gram selection. In Proceedings of the HLT-NAACL 2006 Workshop on Analyzing Conversations in Text and Speech, ACTS ’09, pages 35–41, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  5. William W. Cohen, Vitor R. Carvalho, and Tom M. Mitchell. 2004. Learning to classify email into “speech acts”. In EMNLP, pages 309–316. ACL. find our card
  6. Jade Goldstein and Roberta Evans Sabin. 2006. Using speech acts to categorize email and identify email genres. In Proceedings of the 39th Annual Hawaii International Conference on System Sciences - Volume 03, pages 50.2–, Washington, DC, USA. IEEE Computer Society. find
  7. Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. 2009. The weka data mining software: an update. SIGKDD Explor. Newsl., 11:10–18, November. ↗
  8. Ruihong Huang and Ellen Riloff. 2010. Inducing domain-specific semantic class taggers from (almost) nothing. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, ACL ’10, pages 275–285, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  9. Minwoo Jeong, Chin-Yew Lin, and Gary Geunbae Lee. 2009. Semi-supervised speech act recognition in emails and forums. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 3 - Volume 3, EMNLP ’09, pages 1250–1259, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  10. Andrew Lampert, Robert Dale, and Cecile Paris. 2006. Classifying speech acts using verbal response modes. In Proceedings of the 2006 Australasian Language Technology Workshop (ALTW2006), pages 34–41. Sydney Australia : ALTA. find
  11. Tara McIntosh. 2010. Unsupervised discovery of negative categories in lexicon bootstrapping. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, EMNLP ’10, pages 356–365, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  12. John Mildinhall and Jan Noyes. 2008. Toward a stochastic speech act model of email behavior. In CEAS. find
  13. Jacqueline Nastri, Jorge Pena, and Jeffrey T. Hancock. 2006. The construction of away messages: A speech act analysis. J. Computer-Mediated Communication, pages 1025–1045. ↗
  14. Sujith Ravi and Jihie Kim. 2007. Profiling student interactions in threaded discussions with speech act classifiers. In Proceeding of the 2007 conference on Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work, pages 357–364, Amsterdam, The Netherlands, The Netherlands. IOS Press. find
  15. Ellen Riloff, Janyce Wiebe, and Theresa Wilson. 2003. Learning subjective nouns using extraction pattern bootstrapping. In Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003 - Volume 4, CONLL ’03, pages 25–32, Stroudsburg, PA, USA. Association for Computational Linguistics. find our card
  16. John R. Searle. 1976. A classification of illocutionary acts. Language in Society, 5(1):pp. 1–23. ↗
  17. Michael Thelen and Ellen Riloff. 2002. A bootstrapping method for learning semantic lexicons using extraction pattern contexts. In Proceedings of the ACL-02 conference on Empirical methods in natural language processing - Volume 10, EMNLP ’02, pages 214–221, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  18. Kristina Toutanova, Dan Klein, Christopher D. Manning, and Yoram Singer. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - Volume 1, NAACL ’03, pages 173–180, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  19. Douglas P. Twitchell and Jay F. Nunamaker Jr. 2004. Speech act profiling: a probabilistic method for analyzing persistent conversations and their participants. In System Sciences, 2004. Proceedings of the 37th Annual Hawaii International Conference on, pages 1–10, January. find
  20. Douglas P. Twitchell, Mark Adkins, Jay F. Nunamaker Jr., and Judee K. Burgoon. 2004. Using speech act theory to model conversations for automated classification and retrieval. In Proceedings of the International Working Conference Language Action Perspective Communication Modelling (LAP 2004), pages 121–130. find
  21. A. Wierzbicka. 1987. English speech act verbs: a semantic dictionary. Academic Press, Sydney, Orlando. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/qadir-2011-classifying-sentences-as-speech-acts/