Elamur Research⌕
☰
home / papers / 2017 / Analyzing the Semantic Types of Claims and Premises in an Online Persuasive Forum

Analyzing the Semantic Types of Claims and Premises in an Online Persuasive Forum

Paper · 2017 · Christopher Hidey, Elena Musi, Alyssa Hwang et al.

Summary

The paper tackles a gap in argument mining. Most persuasion research has studied discourse-level signals such as connectives, hedges and argument structure, but the conceptual content of arguments has received little attention, largely because annotated data is missing. The authors build such data from the Change My View subreddit, where the original poster awards a ∆ when a reply changes their mind. This gives a natural persuasive/non-persuasive label.

The annotation has two stages:

  • Stage 1: trained student annotators mark claims and premises at the proposition level.
  • Stage 2: Amazon Mechanical Turk workers assign semantic types. Premises get Aristotle's modes (ethos, logos, pathos, or combinations). Claims are labeled as interpretation, rational or emotional evaluation, agreement, or disagreement.

Agreement is moderate for components and premise types. Claim types prove harder to label reliably. The analysis finds that persuasive and non-persuasive replies differ significantly in the types of claims and premises they contain. This makes the released corpus a useful starting point for modeling the conceptual side of persuasion.

Key points

  • The corpus contains 78 Change My View threads (39 ending with a ∆, 39 without). It draws on the Tan et al. (2016) dataset, keeping only dialogs between the original poster and a single responder.
  • Claims and premises were marked by 8 students with Linguistics or NLP backgrounds. Semantic types were crowdsourced on Amazon Mechanical Turk.
  • Krippendorff's α exceeds 0.63 for claims, premises and premise types, but is only 0.46 for claim types. The main confusion is between interpretations and rational evaluations.
  • Positive threads more often contain consecutive arguments of the same mode, such as logos followed by logos (p < 0.01). Rational evaluations are less frequent in winning replies.
  • The annotated data is publicly available on GitHub (chridey/change-my-view-modes).

Interesting ideas

  • I1. Platform feedback as free labels. The forum's own ∆ award serves as a persuasion label, so no extra annotation is needed for the outcome. Other platforms with built-in feedback, such as accepted answers or upvotes, could supply weak labels the same way.
  • I2. Split annotation by difficulty. Experts handle the harder segmentation task, while crowd workers assign types to segments that are already fixed. This cuts costs and keeps the crowdsourced task simple and well-defined.
  • I3. Proposition-level units. Annotating whole propositions rather than clauses avoids splitting premises that span several clauses. This choice is worth considering whenever reasoning units cross clause boundaries.
  • I4. Dialog-specific labels. Agreement and disagreement are added as claim types to capture responses to the previous speaker. Taxonomies for conversational data can benefit from labels for these anaphoric moves.
  • I5. Studying label sequences. Beyond counting label frequencies, the authors examine transitions between types and where types appear within a post. This reveals patterns such as conceptual coherence that frequency counts alone would miss.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Columbia UniversityNew York · United States149 works · 48,852 citations

📚 Shelves 5 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Philosophy · 211 works

Cite

Cite as Christopher Hidey, Elena Musi, Alyssa Hwang, Smaranda Muresan and Kathy McKeown. 2017. Analyzing the Semantic Types of Claims and Premises in an Online Persuasive Forum. Proceedings of the 4th Workshop on Argument Mining.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w17-5102

@inproceedings{hidey2017analyzing,
  title = {{Analyzing the Semantic Types of Claims and Premises in an Online Persuasive Forum}},
  author = {Christopher Hidey, Elena Musi, Alyssa Hwang, Smaranda Muresan and Kathy McKeown},
  booktitle = {Proceedings of the 4th Workshop on Argument Mining},
  year = {2017},
  doi = {10.18653/v1/w17-5102},
  url = {https://doi.org/10.18653/v1/w17-5102}
}

References

30 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ivan Habernal and Iryna Gurevych. 2017. Argumentation mining in user-generated web discourse. Computational Linguistics . our card
  2. Jean-Claude Anscombre and Oswald Ducrot. 1983. L’argumentation dans la langue. Editions Mardaga. find
  3. Colin Higgins and Robyn Walker. 2012. Ethos, logos, pathos: Strategies of persuasion in social/environmental reports. In Accounting Forum. Elsevier, volume 36, pages 194–208. find
  4. Katie Atkinson and Trevor JM Bench-Capon. 2016. Argument schemes for reasoning about the actions of others. In COMMA. pages 71–82. find
  5. Maria Becker, Alexis Palmer, and Anette Frank. 2016. Argumentative texts and clause types. find
  6. Klaus Krippendorff. 1970. Estimating the reliability, systematic error and random error of interval data. Educational and Psychological Measurement 30(1):61–70. find
  7. Or Biran, Sara Rosenthal, Jacob Andreas, Kathleen McKeown, and Owen Rambow. 2012. Detecting influencers in written online conversations. In Proceedings of the Second Workshop on Language in Social Media. Association for Computational Linguistics, pages 37–45. find
  8. John Lawrence and Chris Reed. 2016. Argument mining using argumentation scheme structures. In COMMA. pages 379–390. find
  9. Bing Liu. 2012. Sentiment analysis and opinion mining. Synthesis lectures on human language technologies 5(1):1–167. find
  10. Katarzyna Budzynska, Mathilde Janier, Chris Reed, Patrick Saint-Dizier, Manfred Stede, and Olena Yaskorska. 2014. A model for processing illocutionary structures and argumentation in debates. In LREC. pages 917–924. our card
  11. Stephanie Lukin, Pranav Anand, Marilyn Walker, and Steve Whittaker. 2017. Argument strength is in the eye of the beholder: Audience effects in persuasion . find
  12. Katarzyna Budzynska and Chris Reed. 2011. Speech acts of argumentation: Inference anchors and peripheral cues in dialogue. In Computational Models of Natural Argument. find
  13. Jan Nuyts. 2012. Notions of (inter) subjectivity. English Text Construction 5(1):53–76. find
  14. Joonsuk Park, Cheryl Blake, and Claire Cardie. 2015. Toward machine-assisted participation in erulemaking: An argumentation model of evaluability. In Proceedings of the 15th International Conference on Artificial Intelligence and Law. ACM, pages 206– 210. find
  15. Robert B Cialdini. 2005. Influence: The psychology of persuasion collins. Revised edition (October 7, 2005) . find
  16. Abhimanyu Das, Sreenivas Gollapudi, Emre Kıcıman, and Onur Varol. 2016. Information dissemination in heterogeneous-intent networks. In Proceedings of the 8th ACM Conference on Web Science. ACM, pages 259–268. find
  17. Andreas Peldszus and Manfred Stede. 2016. An annotated corpus of argumentative microtexts. In D. Mohammed and M. Lewinski, editors, Argumentation and Reasoned Action - Proc. of the 1st European Conference on Argumentation, Lisbon, 2015, College Publications, London. find
  18. Rory Duthie, Katarzyna Budzynska, and Chris Reed. 2016. Mining ethos in political debate. In Proceedings of 6th International Conference on Computational Models of Argument (COMMA 2016). IOS Press, Frontiers in Artificial Intelligence and Applications. find
  19. Chaim Perelman and Lucie Olbrechts-Tyteca. 1973. The new rhetoric: A treatise on argumentation. University of Notre Dame Pess. find
  20. James Ravenscroft, Anika Oellrich, Shyamasree Saha, and Maria Liakata. 2016. Multi-label annotation in scientific articles - the multi-label cancer risk assessment corpus. In Proceedings of the Tenth International Conference on Language Resources and Evaluation LREC 2016, Portorož, Slovenia, May 23… find
  21. James B Freeman. 2000. What types of statements are there? Argumentation 14(2):135–157. find
  22. Debanjan Ghosh, Aquila Khanam, Yubo Han, and Smaranda Muresan. 2016. Coarse-grained argumentation features for scoring persuasive essays. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. pages 549– 554. find
  23. Sara Rosenthal and Kathleen Mckeown. 2017. Detecting influencers in multiple online genres. ACM Transactions on Internet Technology (TOIT) 17(2):12. find
  24. Ivan Habernal and Iryna Gurevych. 2016. Which argument is more convincing? analyzing and predicting convincingness of web arguments using bidirectional lstm. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL). our card
  25. Christian Stab and Iryna Gurevych. 2014a. Annotating argument components and relations in persuasive essays. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers. Dublin City University and Association for Computational Linguistics, Dublin,… our card
  26. Christian Stab and Iryna Gurevych. 2014b. Identifying argumentative discourse structures in persuasive essays. In EMNLP. pages 46–56. our card
  27. Chenhao Tan, Vlad Niculae, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2016. Winning arguments: Interaction dynamics and persuasion strategies in good-faith online discussions. In Proceedings of the 25th International Conference on World Wide Web. International World Wide Web Conferences Stee… find
  28. Frans H van Eemeren and Frans Hendrik Eemeren. 2009. Examining argumentation in context: Fifteen studies on strategic maneuvering, volume 1. John Benjamins Publishing. find
  29. Marilyn A Walker, Jean E Fox Tree, Pranav Anand, Rob Abbott, and Joseph King. 2012. A corpus for research on deliberation and debate. In LREC. pages 812–817. our card
  30. Zhongyu Wei, Yang Liu, and Yi Li. 2016. Is this post persuasive? ranking argumentative comments in the online forum. In The 54th Annual Meeting of the Association for Computational Linguistics. page 195. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hidey-2017-analyzing-semantic-types-claims-premises/