Elamur Research⌕
☰
home / papers / 2014 / Why are You Taking this Stance? Identifying and Classifying Reasons in Ideological Debates

Why are You Taking this Stance? Identifying and Classifying Reasons in Ideological Debates

Summary

Hasan and Ng study stance classification in online ideological debates and move beyond it. Detecting whether an author is for or against an issue is not enough; they also want to know which reasons the author gives. They define reason classification (RC) at two levels. Post-level RC finds which predefined reasons a post uses. Sentence-level RC also locates the sentences that express each reason. They build a reason-annotated corpus of createdebate.com posts in four domains: Abortion, Gay Rights, Obama and Marijuana. Each domain has its own inventory of reasons for each side. Their baseline is a sentence-level MaxEnt classifier with lexical, dependency and quotation-aware features. They then add automatically predicted stance information, first in simple pipeline models and then in progressively richer joint models of stances and reasons. The more sophisticated models outperform simpler ones on both reason and stance classification. Still, the task remains hard, with the best post-level F-scores only in the low 50s. The released corpus is described as the first public resource for sentence- and post-level RC.

Key points

  • Data: Four domains (ABO, GAY, OBA, MAR) with 1741, 1376, 985 and 626 stance-labeled posts. Authors supplied their own stance labels.
    • Reason-labeled subsets contain 463, 561, 447 and 432 posts.
    • Between 20.4% and 43.7% of sentences carry a reason tag.
  • Reason inventories: Each stance in each domain has its own list of reasons, with 5–9 classes per stance including an "Others" class.
    • Reasons below 2% frequency were merged into OTHER.
    • Sentences with no reason are labeled NONE.
    • The roughly 3% of sentences with multiple reasons were reduced to the most prominent reason.
  • Agreement: Cohen's Kappa is 0.61–0.67 at sentence level and 0.78–0.83 at post level, computed without sentences both annotators marked NONE.
    • Most disagreement came from differing span lengths for the same reason.
  • Model: A MaxEnt baseline with five feature types, including n-gram, dependency-based and quote-related features (IsQuote, FollowsQuote).
    • Stance-supported variants range from pipelines to joint models.
    • Threads are unrolled into root-to-leaf post sequences.
  • Error analysis: The main sources of error are missing background knowledge, complex discourse (concessions that invert meaning), and sarcasm or rhetorical questions.
    • The authors suggest knowledge bases such as YAGO2, Freebase and BabelNet as possible help.

Interesting ideas

  • I1. Solve post-level labels from sentence-level predictions. Classify each sentence, then take the union of predicted labels as the post's label set. This lets the learner tie each label to a specific span, and the same design could suit any multi-label document task where evidence is localized.
  • I2. Use a noisy related task as auxiliary signal, and model it carefully. Predicted stance can mislead reason prediction when it is wrong. Comparing pipeline against joint modeling is a reusable way to test whether an auxiliary prediction actually helps.
  • I3. Compute agreement fairly on sparse labels. Leaving out items both annotators marked as the empty class keeps Kappa from being inflated. Reporting agreement at two granularities also separates span-boundary disagreement from label disagreement.
  • I4. Merge rare classes into OTHER with a fixed threshold. A cut-off such as 2% keeps the label space manageable without discarding the data. This is a simple, documented design choice for any new taxonomy.
  • I5. Turn reply threads into root-to-leaf sequences. This gives conversational context to each post. Quote-aware features, such as whether a sentence follows a quoted one, cheaply capture counter-argument structure in forum data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

88 papers of our library cite it (of the 140884 whose bibliographies we read); 15 of them have a page here, 73 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/d14-1083

@inproceedings{hasan2014why,
  title = {{Why are You Taking this Stance? Identifying and Classifying Reasons in Ideological Debates}},
  author = {Hasan, Kazi Saidul and Ng, Vincent},
  booktitle = {Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2014},
  doi = {10.3115/v1/d14-1083},
  url = {https://doi.org/10.3115/v1/d14-1083}
}

References

38 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rakesh Agrawal, Sridhar Rajagopalan, Ramakrishnan Srikant, and Yirong Xu. 2003. Mining newsgroups using networks arising from social behavior. In Proceedings of the 12th International Conference on World Wide Web, pages 529–535. find
  2. Pranav Anand, Marilyn Walker, Rob Abbott, Jean E. Fox Tree, Robeson Bowmani, and Michael Minor. 2011. Cats rule and dogs drool!: Classifying stance in online debate. In Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis, pages 1–9. find
  3. Kevin D. Ashley and Vern R. Walker. 2013. From information retrieval (IR) to argument retrieval (AR) for legal cases: Report on a baseline study. In Proceedings of the 26th International Conference on Legal Knowledge and Information System, pages 29– 38. find
  4. Mohit Bansal, Claire Cardie, and Lillian Lee. 2008. The power of negative thinking: Exploiting label disagreement in the min-cut classification framework. In COLING 2008: Companion volume: Posters, pages 15–18. find
  5. Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pages 1247–1250. find
  6. Filip Boltužić and Jan Šnajder. 2014. Back up your stance: Recognizing arguments in online discussions. In Proceedings of the First Workshop on Argumentation Mining, pages 49–58. find
  7. Stefanie Brüninghaus and Kevin D. Ashley. 2005. Reasoning with textual cases. In Proceedings of the 6th International Conference on Case-Based Reasoning, pages 137–151. find
  8. Clinton Burfoot, Steven Bird, and Timothy Baldwin. 2011. Collective classification of congressional floor-debate transcripts. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 1506–1515. find
  9. Elena Cabrio and Serena Villata. 2012. Natural language arguments: A combined approach. In Proceedings of the 20th European Conference on Artificial Intelligence, pages 205–210. find
  10. Jean Carletta. 1996. Assessing agreement on classification tasks: The Kappa statistic. Computational Linguistics, 22(2):249–254. find our card
  11. Dipanjan Das, Nathan Schneider, Desai Chen, and Noah A. Smith. 2010. SEMAFOR 1.0: A probabilistic frame-semantic parser. Technical report, Carnegie Mellon University Technical Report CMU-LTI-10-001. find
  12. Vanessa Wei Feng and Graeme Hirst. 2011. Classifying arguments by scheme. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 987–996. find
  13. Kazi Saidul Hasan and Vincent Ng. 2013a. Extra-linguistic constraints on stance recognition in ideological debates. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 816–821. find
  14. Kazi Saidul Hasan and Vincent Ng. 2013b. Frame semantics for stance classification. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning, pages 124–132. find
  15. Kazi Saidul Hasan and Vincent Ng. 2013c. Stance classification of ideological debates: Data, models, features, and constraints. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, pages 1348–1356. find
  16. Soo-Min Kim and Eduard Hovy. 2006. Automatic identification of pro and con reasons in online reviews. In Proceedings of the COLING/ACL Main Conference Poster Sessions, pages 483–490. find
  17. Andrew McCallum, Dayne Freitag, and Fernando Pereira. 2000. Maximum entropy Markov models for information extraction and segmentation. In Proceedings of the 17th International Conference on Machine Learning, pages 591–598. find
  18. Ryan McDonald, Kerry Hannan, Tyler Neylon, Mike Wells, and Jeff Reynar. 2007. Structured models for fine-to-coarse sentiment analysis. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 432–439. find
  19. Roberto Navigli and Simone Paolo Ponzetto. 2012. BabelNet: The automatic construction, evaluation and application of a wide-coverage multilingual semantic network. Artificial Intelligence, 193:217– 250. find
  20. Raquel Mochales Palau and Marie-Francine Moens. 2011. Argumentation mining. Artificial Intelligence and Law, 19(1):1–22. find
  21. Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42nd Meeting of the Association for Computational Linguistics, pages 271–278. find our card
  22. Isaac Persing and Vincent Ng. 2009. Semi-supervised cause identification from aviation safety reports. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 843–851. find
  23. Dan Roth and Wen-tau Yih. 2004. A linear programming formulation for global inference in natural language tasks. In Proceedings of the Eighth Conference on Computational Natural Language Learning, pages 1–8. find our card
  24. Swapna Somasundaran and Janyce Wiebe. 2009. Recognizing stances in online debates. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 226–234. find
  25. Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. YAGO: A core of semantic knowledge. In Proceedings of the 16th International World Wide Web Conference, pages 697–706. find
  26. Simone Teufel and Marc Moens. 2000. What’s yours and what’s mine: Determining intellectual attribution in scientific text. In Proceedings of the 2000 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, pages 9–17. find
  27. Simone Teufel. 2001. Task-based evaluation of summary quality: Describing relationships between scientific papers. In Proceedings of the NAACL Workshop on Automatic Summarization, pages 12–21. find
  28. Matt Thomas, Bo Pang, and Lillian Lee. 2006. Get out the vote: Determining support or opposition from congressional floor-debate transcripts. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 327–335. find our card
  29. Rakshit Trivedi and Jacob Eisenstein. 2013. Discourse connectors for latent subjectivity in sentiment analysis. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 808–813. find
  30. Maria Paz Garcia Villalba and Patrick Saint-Dizier. 2012. Some facets of argument mining for opinion analysis. In Proceedings of the Fourth International Conference on Computational Models of Argument, pages 23–34. find
  31. Andrew J. Viterbi. 1967. Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Transactions on Information Theory, 13(2):260–269. find
  32. Marilyn Walker, Pranav Anand, Rob Abbott, and Ricky Grant. 2012. Stance classification using dialogic properties of persuasion. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 592–596. find our card
  33. Janyce Wiebe and Ellen Riloff. 2005. Creating subjective and objective sentence classifiers from unannotated texts. In Proceedings of the 6th International Conference on Computational Linguistics and Intelligent Text Processing, pages 486–497. find
  34. Adam Wyner, Raquel Mochales-Palau, Marie-Francine Moens, and David Milward. 2010. Approaches to text mining arguments from legal cases. In Enrico Francesconi, Simonetta Montemagni, Wim Peters, and Daniela Tiscornia, editors, Semantic Processing of Legal Texts: Where the Language of Law Meets the Law… find
  35. Adam Wyner, Jodi Schneider, Katie Atkinson, and Trevor J. M. Bench-Capon. 2012. Semi-automated argumentative analysis of online product reviews. In Proceedings of the Fourth International Conference on Computational Models of Argument, pages 43– 50. find
  36. Ainur Yessenalina, Yejin Choi, and Claire Cardie. 2010. Automatically generating annotator rationales to improve sentiment classification. In Proceedings of the ACL 2010 Conference Short Papers, pages 336–341. find
  37. Omar Zaidan, Jason Eisner, and Christine Piatko. 2007. Using ”annotator rationales” to improve machine learning for text categorization. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Confere… find our card
  38. Jun Zhao, Kang Liu, and Gen Wang. 2008. Adding redundant features for crfs-based sentence sentiment classification. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 117–126. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hasan-2014-why-are-you-taking-this/