Elamur Research⌕
☰
home / papers / 2011 / Cats Rule and Dogs Drool!: Classifying Stance in Online Debate

Cats Rule and Dogs Drool!: Classifying Stance in Online Debate

Paper · 2011 · Pranav Anand, Marilyn Walker, Rob Abbott et al.

Summary

The paper tackles stance classification in online debates: deciding which side a post supports. The authors also address a related task, detecting whether a post is a rebuttal to another post. Their data comes from ConvinceMe.net, where users place each argument on a side, so posts are effectively self-labelled. The corpus has 4873 posts from 1113 two-sided debates covering 14 topics, from light ones like Cats vs. Dogs to ideological ones like abortion and the death penalty.

The authors find that ideological topics contain more rebuttals and more posts per author. Rebuttals are harder to side, both for Mechanical Turk annotators and for classifiers. They train Naive Bayes models on unigrams, LIWC categories and generalized dependency features, with and without features from the parent post. Rebuttals are detected with 63% accuracy. Per-topic stance accuracy ranges from 54% to 69%, against unigram baselines of 49–60%. Their simple context features do not help consistently, which suggests that modelling the dialogic link between posts needs better tools.

Key points

  • Corpus. 1113 debates, 4873 posts and 14 topics from ConvinceMe.net, about 2.7M words in total. The topic-labelled subset used in the experiments has 507,827 words.
  • Rebuttal share. The proportion of rebuttal posts ranges from 34% to 80% depending on the topic, and it is higher for ideological debates.
  • Dialogic markers. Rebuttals use "you" more often than non-rebuttals (mean 9.6 vs. 8.5, p < .001). They also show more anaphora, ellipsis and dialogic cue words.
  • Features and evaluation. The feature sets are unigrams, LIWC, GdepO (dependencies generalized with MPQA terms) and GdepP (dependencies generalized with POS tags). All models use Naive Bayes with 10-fold cross-validation. The human topline from MTurk ranges from 66% to 94% per topic.
  • Effect of context. Averaged over topics, context features made no significant difference (53.0% with context vs. 54.62% without). They helped a lot on some topics, for example 69.23% on 2nd Amendment using unigrams plus context.

Interesting ideas

  • I1. Free labels from site design. Platforms where authors pick a side as they post provide stance labels at no cost. This could be a cheap source of supervision for low-resource languages, wherever similar debate sites exist.
  • I2. Grouping topics by style. The authors split topics into playful/technical vs. ideological and compare their structure, such as rebuttal share and posts per author. Reporting results per topic group shows where a method works, rather than hiding variation behind one average.
  • I3. A human topline. Crowd annotators sided posts without seeing the context. This gives a realistic upper bound and shows how much room for improvement remains on each topic.
  • I4. Simple corpus statistics as evidence of dialogue. Counting pronouns like "you" in rebuttals vs. non-rebuttals is a cheap way to check whether a corpus is conversational before choosing a modelling approach.
  • I5. Candid analysis of a negative result. The authors conclude that their naive context features miss the relation between a post and its parent, and they call for sequential models. This is a useful warning: simply adding parent-post features is not enough to capture dialogue.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of California, Santa CruzSanta Cruz · United States24 works · 3,112 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

Cite

Cite as Pranav Anand, Marilyn Walker, Rob Abbott, Jean E. Fox Tree, Robeson Bowmani and Michael Minor. 2011. Cats Rule and Dogs Drool!: Classifying Stance in Online Debate. Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis (WASSA 2.011).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{anand2011cats,
  title = {{Cats Rule and Dogs Drool!: Classifying Stance in Online Debate}},
  author = {Pranav Anand, Marilyn Walker, Rob Abbott, Jean E. Fox Tree, Robeson Bowmani and Michael Minor},
  booktitle = {Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis (WASSA 2.011)},
  year = {2011},
  url = {https://aclanthology.org/W11-1701/}
}

References

25 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rob Abbott, Marilyn Walker, Jean E. Fox Tree, Pranav Anand, Robeson Bowmani, and Joseph King. 2011. How can you say such things?!?: Recognizing Disagreement in Informal Political Argument. In Proceedings of the ACL Workshop on Language and Social Media. find
  2. R. Agrawal, S. Rajagopalan, R. Srikant, and Y. Xu. 2003. Mining newsgroups using networks arising from social behavior. In Proceedings of the 12th international conference on World Wide Web, pages 529–535. ACM. find
  3. D. Biber. 1991. Variation across speech and writing. Cambridge Univ Pr. find
  4. David Crystal. 2001. Language and the Internet. Cambridge University Press. find
  5. M.C. De Marneffe, B. MacCartney, and C.D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In Proceedings of LREC, volume 6, pages 449–454. Citeseer.
  6. J.E. Fox Tree and J.C. Schrock. 1999. Discourse Markers in Spontaneous Speech: Oh What a Difference an Oh Makes. Journal of Memory and Language, 40(2):280–295. find
  7. J.E. Fox Tree and J.C. Schrock. 2002. Basic meanings of you know and I mean. Journal of Pragmatics, 34(6):727–747. find
  8. J. E. Fox Tree. 2010. Discourse markers across speakers and settings. Language and Linguistics Compass, 3(1):113. find
  9. S. Greene and P. Resnik. 2009. More than words: Syntactic packaging and implicit sentiment. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 503–511. Association for Computational Linguisti…
  10. M. Groen, J. Noyes, and F. Verstraten. 2010. The Effect of Substituting Discourse Markers on Their Role in Dialogue. Discourse Processes: A Multidisciplinary Journal, 47(5):33. find
  11. M. Joshi and C. Penstein-Rosé. 2009. Generalizing dependency features for opinion mining. In Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, pages 313–316. Association for Computational Linguistics. find
  12. D. Klein and C.D. Manning. 2003. Accurate unlexicalized parsing. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics-Volume 1, pages 423–430. Association for Computational Linguistics.
  13. W.H. Lin, T. Wilson, J. Wiebe, and A. Hauptmann. 2006. Which side are you on?: identifying perspectives at the document and sentence levels. In Proceedings of the Tenth Conference on Computational Natural Language Learning, pages 109–116. Association for Computational Linguistics. find
  14. R. Malouf and T. Mullen. 2008. Taking sides: User classification for informal online political discourse. Internet Research, 18(2):177–190. find
  15. Daniel Marcu. 2000. Perlocutions: The Achilles’ heel of Speech Act Theory. Journal of Pragmatics, 32(12):1719–1741. find
  16. G. Marwell and D. Schmitt. 1967. Dimensions of compliance-gaining behavior: An empirical analysis. sociomety, 30:350–364. find
  17. G. Mishne and N. Glance. 2006. Leave a reply: An analysis of weblog comments. In Third annual workshop on the Weblogging ecosystem. Citeseer. find
  18. A. Murakami and R. Raymond. 2010. Support or Oppose? Classifying Positions in Online Debates from Reply Activities and Opinion Expressions. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pages 869–875. Association for Computational Linguistics. find
  19. B. Pang and L. Lee. 2008. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1-2):1–135. find
  20. J. W. Pennebaker, L. E. Francis, and R. J. Booth, 2001. LIWC: Linguistic Inquiry and Word Count. find
  21. S. Somasundaran and J. Wiebe. 2009. Recognizing stances in online debates. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1-Volume 1, pages 226–234. Association for Computatio… find
  22. S. Somasundaran and J. Wiebe. 2010. Recognizing stances in ideological on-line debates. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, pages 116–124. Association for Computational Linguistics.
  23. Karen Sparck-Jones. 1999. Automatic summarizing; factors and directions. In Inderjeet Mani and Mark Maybury, editors, Advances in Automatic Text Summarization. MIT Press. find
  24. Marilyn A. Walker. 1996. Inferring acceptance and rejection in dialogue by default rules of inference. Language and Speech, 39-2:265–304. find
  25. Y.C. Wang and C.P. Rosé. 2010. Making conversational structure explicit: identification of initiation-response pairs within online discussions. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 673–676. A… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/anand-2011-cats-rule-dogs-drool-classifying/