Elamur Research⌕
☰
home / papers / 2012 / Stance Classification using Dialogic Properties of Persuasion

Stance Classification using Dialogic Properties of Persuasion

Summary

The paper addresses stance classification in online debate forums: deciding which side (for example Pro or Con) a post supports. Posts on these forums are emotional, sarcastic and personal, which makes them hard to classify from text alone. The authors use two-sided debates from Convinceme.net, where authors label their own side. A JRIP rule learner is trained on content features: n-grams, LIWC-based sentiment features, dependency features, cue words and parent-post context.

They then build a graph over posts. Rebuttal links are treated as disagreement, and posts by the same author are treated as agreement. A MaxCut partition splits each connected component into two camps. The classifier's margins then decide which camp gets which stance label.

On most topics this dialogic approach clearly beats the text-only classifier, often by more than 20 accuracy points. The gain grows with how interactive a debate is. The work shows that the network of who argues with whom carries strong stance signal beyond the words themselves.

Key points

  • Corpus: 2902 two-sided Convinceme.net debates with 36,307 posts and about 3.08M words. The topic-labelled subset used in experiments has 575,818 words. The text says 14 topics, but Table 2 reports results for 18. The corpus is publicly released.
  • Features and classifier: Three feature groups are used (basic, sentiment, argument), including LIWC categories and POS-generalized dependencies. Features go through χ² selection, and JRIP is the base learner. NB and SVM did not give significantly different results.
  • Graph weights:
    • Rebuttal edges get +100 divided by the number of rebuttals.
    • Same-author edges get −60 divided by the author's post count.
    • All post pairs between two authors get 50 if one author rebutted the other, and 100 if the rebuttals went both ways.
  • Results (MaxCut vs. JRIP accuracy): Abortion 82% vs. 55%, Evolution 82% vs. 56%, Gay Marriage 84% vs. 60%, Iphone v. Blackberry 88% vs. 71%.
  • Failures and correlations:
    • MaxCut accuracy drops on some topics: Death Penalty 35%, Gun Control 37%, Mac v. PC 19%, Superman v. Batman 50%. The paper explains that in such cases the posts are grouped correctly but the two groups get swapped labels. Immigration (33% vs. 53%) is also worse, though the paper does not name it.
    • Precision and recall correlate with rebuttal share (R = .16 and .22) and with posts per author (R = .25 and .43).
    • The earlier best result on online debates was a 64% average (Somasundaran and Wiebe, 2010).

Interesting ideas

  • I1. Separate grouping from labelling: First cluster items into two camps using only relational structure, then use a text classifier just to name the camps. The same pattern could suit any binary task where relations are reliable but per-item labels are noisy.
  • I2. Orient camps by summed margins: Name each partition by comparing the classifier's summed confidence for each label across its members. This turns many weak per-post predictions into one robust group-level decision, with a coin flip only on exact ties.
  • I3. Normalize edge weights: Dividing weights by the number of rebuttals or an author's post count keeps very active users or threads from dominating the partition. This is a cheap safeguard for any graph built from user activity.
  • I4. Analyze errors by orientation: Check whether failures come from wrong grouping or from a wrong label assignment. This shows where to focus improvements, which here is better orientation rather than better clustering.
  • I5. Profile the dataset structure: Report per-topic structural statistics, such as rebuttal share and posts per author, and correlate them with performance. This explains when a relational method will help and could guide method choice on new forums or datasets.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

80 papers of our library cite it (of the 140884 whose bibliographies we read); 14 of them have a page here, 66 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{walker2012stance,
  title = {{Stance Classification using Dialogic Properties of Persuasion}},
  author = {Walker, Marilyn and Anand, Pranav and Abbott, Rob and Grant, Ricky},
  booktitle = {Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2012},
  url = {https://aclanthology.org/N12-1072/}
}

References

19 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. R. Agrawal, S. Rajagopalan, R. Srikant, and Y. Xu. 2003. Mining newsgroups using networks arising from social behavior. In Proceedings of the 12th international conference on World Wide Web, pages 529–535. ACM. find
  2. A. Balahur, Z. Kozareva, and A. Montoyo. 2009. Determining the polarity and source of opinions expressed in political debates. Computational Linguistics and Intelligent Text Processing, pages 468–480. find
  3. M. Bansal, C. Cardie, and L. Lee. 2008. The power of negative thinking: Exploiting label disagreement in the min-cut classification framework. Proceedings of COLING: Companion volume: Posters, pages 13–16. find
  4. O. Biran and O. Rambow. 2011. Identifying justifications in written dialogs. In 2011 Fifth IEEE International Conference on Semantic Computing (ICSC), pages 162–168. IEEE. find
  5. C. Burfoot, S. Bird, and T. Baldwin. 2011. Collective classification of congressional floor-debate transcripts. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 1506–1515. Association for Computational Linguistics… find
  6. S. Chaiken. 1980. Heuristic versus systematic information processing and the use of source versus message cues in persuasion. Journal of personality and social psychology, 39(5):752. find
  7. Robert B. Cialdini. 2000. Influence: Science and Practice (4th Edition). Allyn & Bacon. find
  8. M.F. Davies. 1998. Dogmatism and belief formation: Output interference in the processing of supporting and contradictory cognitions. Journal of personality and social psychology, 75(2):456. find
  9. R. Malouf and T. Mullen. 2008. Taking sides: User classification for informal online political discourse. Internet Research, 18(2):177–190. find
  10. A. Murakami and R. Raymond. 2010. Support or Oppose? Classifying Positions in Online Debates from Reply Activities and Opinion Expressions. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pages 869–875. Association for Computational Linguistics. find
  11. J. W. Pennebaker, L. E. Francis, and R. J. Booth, 2001. LIWC: Linguistic Inquiry and Word Count. find
  12. Richard E. Petty and John T. Cacioppo. 1988. The effects of involvement on responses to argument quantity and quality: Central and peripheral routes to persuasion. Journal of Personality and Social Psychology, 46(1):69–81. find
  13. R.E. Petty, J.T. Cacioppo, and R. Goldman. 1981. Personal involvement as a determinant of argument-based persuasion. Journal of Personality and Social Psychology, 41(5):847. find
  14. S. Somasundaran and J. Wiebe. 2009. Recognizing stances in online debates. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1-Volume 1, pages 226–234. Association for Computatio… find
  15. S. Somasundaran and J. Wiebe. 2010. Recognizing stances in ideological on-line debates. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, pages 116–124. Association for Computational Linguistics. find our card
  16. M. Thomas, B. Pang, and L. Lee. 2006. Get out the vote: Determining support or opposition from Congressional floor-debate transcripts. In Proceedings of the 2006 conference on empirical methods in natural language processing, pages 327–335. Association for Computational Linguistics. find our card
  17. M. Walker, P. Anand, J. Fox Tree, R. Abbott, and J. King. 2012. A corpus for research on deliberation and debate. In Proceedings of the Eighth International Conference on Language Resources and Evaluation, LREC 2012, Istanbul, Turkey, May 23-25, 2012. find our card
  18. Y.C. Wang and C.P. Rosé. 2010. Making conversational structure explicit: identification of initiation-response pairs within online discussions. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 673–676. A… find
  19. A. Yessenalina, Y. Yue, and C. Cardie. 2010. Multi-level structured models for document-level sentiment classification. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1046–1056. Association for Computational Linguistics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/walker-2012-stance-classification-dialogic-properties-persuasion/