Elamur Research
☰
home / papers / 2014 / Identifying Argumentative Discourse Structures in Persuasive Essays

Identifying Argumentative Discourse Structures in Persuasive Essays

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/D14-1006.pdf

Summary

The paper tackles the automatic recovery of argument structure in student persuasive essays, motivated by writing-support tools that could give learners feedback on their reasoning rather than only on spelling or style. Earlier work mostly labelled argument components in isolation, or relied on hand-built grammars and discourse markers suited to formulaic legal texts. Using the essay corpus of Stab and Gurevych (2014), the authors split the problem into two stages. First, a multiclass classifier labels text units as argument components (major claim, claim, premise) or non-argumentative. Second, pairs of components are classified as support or non-support. They compare several classifiers and introduce new structural, lexical, syntactic, indicator and contextual feature sets. Macro F1 reaches 0.726 for components and 0.722 for relations, roughly 88% and 90% of human agreement. The work matters because it handles implicit and non-adjacent relations, which marker-based or RST-style methods miss. This makes it an early step toward argument analysis in educational settings.

Key points

  • Results: macro F1 of 0.726 for component classification and 0.722 for support relations. These correspond to 88.1% and 90.5% of human performance.
  • Structural features were the strongest for component identification. Telling claims apart from premises was the hardest part.
  • Discourse markers are weak cues in student essays. Indicator features alone give an F1 of only 0.265 for claims, compared with 0.673 reported by Mochales-Palau and Moens (2009) on legal documents using a CFG.
  • Supplying gold component types raises relation identification to a macro F1 above 0.85. This suggests the two tasks reinforce each other and could be modelled jointly.
  • Motivating statistics:
    • Only 37% of premises in the corpus sit next to their claim.
    • Only 26% of evidence relations in the RST Discourse Treebank carry explicit markers (Marcu and Echihabi, 2002).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Discourse analysis

Cited by

154 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 149 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/d14-1006

@inproceedings{stab2014identifying,
  title = {{Identifying Argumentative Discourse Structures in Persuasive Essays}},
  author = {Stab, Christian and Gurevych, Iryna},
  booktitle = {Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2014},
  doi = {10.3115/v1/d14-1006},
  url = {https://doi.org/10.3115/v1/d14-1006}
}

References

30 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Or Biran and Owen Rambow. 2011. Identifying justifications in written dialogs by classifying text as argumentative. International Journal of Semantic Computing, 05(04):363–381. find
  2. M. Anne Britt and Aaron A. Larson. 2003. Constructing representations of arguments. Journal of Memory and Language, 48(4):794 – 810. find
  3. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski. 2001. Building a discourse-tagged corpus in the framework of rhetorical structure theory. In Proceedings of the Second SIGdial Workshop on Discourse and Dialogue - Volume 16, SIGDIAL ’01, pages 1–10, Aalborg, Denmark. find our card
  4. Johannes Daxenberger, Oliver Ferschke, Iryna Gurevych, and Torsten Zesch. 2014. DKPro TC: A Java-based framework for supervised learning experiments on textual data. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. System Demonstrations, pages 61–66, Baltim… find
  5. Vanessa Wei Feng and Graeme Hirst. 2011. Classifying arguments by scheme. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1, HLT ’11, pages 987–996, Portland, OR, USA. find
  6. Eirini Florou, Stasinos Konstantopoulos, Antonis Koukourikos, and Pythagoras Karampiperis. 2013. Argument extraction for supporting public policy formulation. In Proceedings of the 7th Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities, pages 49–54, Sofia, Bulgari… find
  7. George Forman and Martin Scholz. 2010. Apples-to-apples in cross-validation studies: Pitfalls in classifier performance measurement. SIGKDD Explor. Newsl., 12(1):49–57. find
  8. Iryna Gurevych, Max Mühlhäuser, Christof Mueller, Juergen Steimle, Markus Weimer, and Torsten Zesch. 2007. Darmstadt Knowledge Processing Repository based on UIMA. In Proceedings of the First Workshop on Unstructured Information Management Architecture at Biannual Conference of the Society for Com… find
  9. Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. 2009. The weka data mining software: An update. SIGKDD Explor. Newsl., 11(1):10–18. find
  10. Dan Klein and Christopher D. Manning. 2003. Accurate unlexicalized parsing. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - Volume 1, ACL ’03, pages 423– 430, Sapporo, Japan. find our card
  11. Klaus Krippendorff. 2004. Measuring the Reliability of Qualitative Text Analysis Data. Quality & Quantity, 38(6):787–800. find
  12. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing implicit discourse relations in the Penn Discourse Treebank. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1, EMNLP ’09, pages 343–351, Stroudsburg, PA, USA. find our card
  13. Annie Louis, Aravind Joshi, Rashmi Prasad, and Ani Nenkova. 2010. Using entity features to classify implicit discourse relations. In Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue, SIGDIAL ’10, pages 59–62, Stroudsburg, PA, USA. find
  14. William C. Mann and Sandra A. Thompson. 1987. Rhetorical structure theory: A theory of text organization. Technical Report ISI/RS-87-190, Information Sciences Institute. find
  15. Daniel Marcu and Abdessamad Echihabi. 2002. An unsupervised approach to recognizing discourse relations. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, pages 368–375. find our card
  16. Quinn McNemar. 1947. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153–157. find
  17. Raquel Mochales-Palau and Marie-Francine Moens. 2009. Argumentation mining: The detection, classification and structure of arguments in text. In Proceedings of the 12th International Conference on Artificial Intelligence and Law, ICAIL ’09, pages 98– 107, New York, NY, USA. ACM. find
  18. Marie-Francine Moens, Erik Boiy, Raquel Mochales Palau, and Chris Reed. 2007. Automatic detection of arguments in legal texts. In Proceedings of the 11th International Conference on Artificial Intelligence and Law, ICAIL ’07, pages 225–230, Stanford, California. find
  19. Andreas Peldszus and Manfred Stede. 2013. From Argument Diagrams to Argumentation Mining in Texts: A Survey. International Journal of Cognitive Informatics and Natural Intelligence (IJCINI), 7(1):1–31. find
  20. Emily Pitler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, ACL ’09, pages 6… find our card
  21. Rashmi Prasad, Eleni Miltsakaki, Nikhil Dinesh, Alan Lee, Aravind Joshi, Livio Robaldo, and Bonnie L. Webber. 2007. The Penn Discourse Treebank 2.0 annotation manual. Technical report, Institute for Research in Cognitive Science, University of Pennsylvania. find
  22. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn Discourse Treebank 2.0. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco. find our card
  23. Chris Reed, Raquel Mochales-Palau, Glenn Rowe, and Marie-Francine Moens. 2008. Language resources for studying argument. In Proceedings of the Sixth International Conference on Language Resources and Evaluation, LREC ’08, pages 2613–2618, Marrakech, Morocco. find
  24. Niall Rooney, Hui Wang, and Fiona Browne. 2012. Applying kernel methods to argumentation mining. In Proceedings of the Twenty-Fifth International Florida Artificial Intelligence Research Society Conference, FLAIRS ’12, pages 272–275, Marco Island, FL, USA. find
  25. Alan Sergeant. 2013. Automatic argumentation extraction. In Proceedings of the 10th European Semantic Web Conference, ESWC ’13, pages 656–660, Montpellier, France. find
  26. Christian Stab and Iryna Gurevych. 2014. Annotating argument components and relations in persuasive essays. In Proceedings of the 25th International Conference on Computational Linguistics (COLING 2014), pages 1501–1510, Dublin, Ireland, August. find
  27. Simone Teufel. 1999. Argumentative Zoning: Information Extraction from Scientific Text. Ph.D. thesis, University of Edinburgh. find
  28. Kristina Toutanova, Dan Klein, Christopher D. Manning, and Yoram Singer. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, NAAC… find
  29. Douglas N Walton. 1996. Argumentation schemes for presumptive reasoning. Routledge. find
  30. Adam Wyner, Raquel Mochales Palau, Marie-Francine Moens, and David Milward. 2010. Approaches to text mining arguments from legal cases. In Semantic Processing of Legal Texts, volume 6036 of Lecture Notes in Computer Science, pages 60–79. find