Elamur ResearchβŒ•
☰
home / papers / 2017 / Examples and Specifications that Prove a Point: Identifying Elaborative and Argumentative Discourse Relations

Examples and Specifications that Prove a Point: Identifying Elaborative and Argumentative Discourse Relations

Summary

Examples and specifications are among the most frequent discourse relations, but annotated corpora often disagree on how to label them. Using the WSJ-Aligned corpus, which maps Penn Discourse Treebank (PDTB) and RST-DT annotations of the same Wall Street Journal texts onto each other, Scholman and Demberg show that PDTB's INSTANTIATION and SPECIFICATION relations are often labelled as argumentative (causal) in RST-DT. The reverse pattern is much rarer.

To test how ordinary readers actually understand these relations, they ran a crowdsourcing study. Untrained participants inserted connectives, chosen from a small and fairly unambiguous predefined list, between the two segments. A replication without surrounding context gave highly similar results.

Readers frequently interpret these relations as argumentative. Examples and specifications can therefore illustrate a situation and support a claim at the same time. The authors argue that annotation schemes should capture such multiple simultaneous readings. They also see connective-insertion crowdsourcing as a cheap and replicable way to obtain distributions of relation senses, and potentially to build a corpus.

Key points

  • Agreement between frameworks: PDTB and RST-DT agree on about 60% of explicit relations but only about 35% of implicit ones. Agreement is just 32% for INSTANTIATION and 14% for SPECIFICATION.
  • Frequency: INSTANTIATION and SPECIFICATION together make up 24% of implicit relations in the PDTB.
  • Asymmetry between frameworks:
    • Among 306 PDTB INSTANTIATION relations, 25% receive causal labels (EXPLANATION-ARGUMENTATIVE, EVIDENCE) in RST-DT.
    • Among 426 PDTB SPECIFICATION relations, the figure is 21%.
    • In the other direction, only 7% of RST EXAMPLE and 12% of RST GENERAL-SPECIFIC relations get causal labels in the PDTB.
  • Use of the second connective: Participants could insert a second connective but rarely did. The authors suggest making a second choice mandatory, with a "no other connective fits" option.
  • Domain knowledge: Items with highly scattered answers likely reflect participants' limited knowledge of economics, which the authors note could also affect trained annotators.

Our summary of the paper, in our own words β€” not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

15 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 9 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.5087/dad.2017.203

@misc{scholman2017examples,
  title = {{Examples and Specifications that Prove a Point: Identifying Elaborative and Argumentative Discourse Relations}},
  author = {Scholman, Merel C.J. and Demberg, Vera},
  year = {2017},
  doi = {10.5087/dad.2017.203},
  url = {https://doi.org/10.5087/dad.2017.203}
}

References

54 references, 9 of them with a card of ours. β†— leads to the work, Β«findΒ» searches for one we have no id for yet.

  1. Artstein, R., & Poesio, M. (2005). Bias decreases in proportion to the number of annotators. Proceedings of the Conference on Formal Grammar and Mathematics of Language (FG-MoL), (pp. 141–150). find
  2. Asr, F. T., & Demberg, V. (2013). On the information conveyed by discourse markers. In Proceedings of the Fourth Annual Workshop on Cognitive Modeling and Computational Linguistics (pp. 84–93). find our card
  3. Biran, O., & Rambow, O. (2011). Identifying justifications in written dialogs by classifying text as argumentative. International Journal of Semantic Computing, 5, 363–381. find
  4. Blakemore, D. (1997). Restatement and exemplification: A relevance theoretic reassessment of elaboration. Pragmatics & Cognition, 5, 1–19. find
  5. Carlson, L., & Marcu, D. (2001). Discourse tagging reference manual. ISI Technical Report ISI-TR-545, 54, 1–56. find
  6. Carlson, L., Marcu, D., & Okurowski, M. E. (2003). Building a discourse-tagged corpus in the framework of Rhetorical Structure Theory. In Current and new directions in discourse and dialogue (pp. 85–112). Springer. find our card
  7. Carston, R. (1993). Conjunction, explanation and relevance. Lingua, 90, 27–48. find
  8. Cohen, R. (1987). Analyzing the structure of argumentative discourse. Computational Linguistics, 13, 11–24. find
  9. Crible, L., & Degand, L. (in press). Reliability vs. granularity in discourse annotation: What is the trade-off? Corpus Linguistics and Linguistic Theory, . find
  10. Cuenca, M.-J. (2003). Two ways to reformulate: a contrastive analysis of reformulation markers. Journal of Pragmatics, 35, 1069–1093. find
  11. Cuenca, M.-J., & Marı́n, M.-J. (2009). Co-occurrence of discourse markers in Catalan and Spanish oral narrative. Journal of Pragmatics, 41, 899–914. find
  12. Degand, L. (1998). On classifying connectives and coherence relations. In Proceedings of the 1998 ACL Workshop on Discourse Relations and Discourse Markers (pp. 29–35). find
  13. Demberg, V., Asr, F., & Scholman, M. (2017). How consistent are our discourse annotations? Insights from mapping RST-DT and PDTB annotations. ArXiv e-prints, April. arXiv:1704.08893. β€” [arXiv:1704.08893](https://arxiv.org/abs/1704.08893) find
  14. González, M. (2005). Pragmatic markers and discourse coherence relations in English and Catalan oral narrative. Discourse Studies, 7, 53–86. find
  15. Halliday, M. A. (1994). Functional grammar. London: Edward Arnold, . find
  16. Hobbs, J. R. (1979). Coherence and coreference. Cognitive Science, 3, 67–90. find
  17. Hovy, E. H. (1995). The multifunctionality of discourse markers. In Proceedings of the Workshop on Discourse Markers (pp. 1–11). Citeseer. find
  18. Hovy, E. H., & Maier, E. (1995). Parsimonious or profligate: How many and which discourse structure relations. Unpublished manuscript, . find
  19. Hyland, K. (2007). Applying a gloss: Exemplifying and reformulating in academic discourse. Applied Linguistics, 28, 266–285. find
  20. Jasinskaja, K. (2013). Corrective elaboration. Lingua, 132, 51–66. find
  21. Jasinskaja, K., & Karagjosova, E. (2011). Elaboration and explanation. Constraints in Discourse, 4. find
  22. Kawahara, D., Machida, Y., Shibata, T., Kurohashi, S., Kobayashi, H., & Sassano, M. (2014). Rapid development of a corpus with discourse annotations using two-stage crowdsourcing. In Proceedings of the International Conference on Computational Linguistics (pp. 269–278). find
  23. Knott, A., & Dale, R. (1994). Using linguistic phenomena to motivate a set of coherence relations. Discourse Processes, 18, 35–62. find
  24. Konig, E., & Siemund, P. (2000). Causal and concessive clauses: Formal and semantic relations. Topics in English Linguistics, 33, 341–360. find
  25. Krippendorff, K. (1980). Content analysis: An introduction to its methodology. Sage. find
  26. Li, J. J., & Nenkova, A. (2016). The Instantiation discourse relation: A corpus analysis of its properties and improved detection. In Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (pp. 1181–1186). find
  27. Mann, W. C., & Thompson, S. A. (1988). Rhetorical Structure Theory: Toward a functional theory of text organization. Text-Interdisciplinary Journal for the Study of Discourse, 8, 243–281. find our card
  28. Martins, D., Kigiel, D., & Jhean-Larose, S. (2006). Influence of expertise, coherence, and causal connectives on comprehension and recall of an expository text. Current Psychology Letters. Behaviour, Brain & Cognition, 3. find
  29. Maschler, Y., & Schiffrin, D. (2015). Discourse markers: Language, meaning, and context. In D. S. D. Tannen, H.E. Hamilton (Ed.), The Handbook of Discourse Analysis, Second edition, Chichester, UK: John Wiley & Sons (pp. 189–221). find
  30. McNamara, D. S., Kintsch, E., Songer, N. B., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14, 1–43. find
  31. Moore, J. D., & Pollack, M. E. (1992). A problem for RST: The need for multi-level discourse analysis. Computational Linguistics, 18, 537–544. find
  32. Peldszus, A., & Stede, M. (2013). From argument diagrams to argumentation mining in texts: A survey. International Journal of Cognitive Informatics and Natural Intelligence (IJCINI), 7, 1–31. find
  33. Prasad, R., Dinesh, N., Lee, A., Miltsakaki, E., Robaldo, L., Joshi, A. K., & Webber, B. (2008). The Penn Discourse TreeBank 2.0. In Proceedings of the International Conference on Language Resources and Evaluation (LREC). Citeseer. find our card
  34. Prasad, R., Miltsakaki, E., Dinesh, N., Lee, A., Joshi, A., Robaldo, L., & Webber, B. (2007). The Penn Discourse Treebank 2.0 annotation manual. find
  35. Pusse, F., Sayeed, A., & Demberg, V. (2016). Lingoturk: Managing crowdsourced tasks for psycholinguistics. In Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). find
  36. Redeker, G. (1990). Ideational and pragmatic markers of discourse structure. Journal of Pragmatics, 14, 367–381. find
  37. Riezler, S. (2014). On the problem of theoretical terms in empirical computational linguistics. Computational Linguistics, 40, 235–245. find
  38. Robaldo, L., & Miltsakaki, E. (2014). Corpus-driven semantics of concession: Where do expectations come from? Dialogue & Discourse, 5, 1–36. find
  39. Rohde, H., Dickinson, A., Clark, C. N., Louis, A., & Webber, B. (2015). Recovering discourse relations: Varying influence of discourse adverbials. In Proceedings of the Workshop on Linking Models of Lexical, Sentential and Discourse-level Semantics (LSDSem) (pp. 1–22). find our card
  40. Rohde, H., Dickinson, A., Schneider, N., Clark, C. N., Louis, A., & Webber, B. (2016). Filling in the blanks in understanding discourse adverbials: Consistency, conflict, and context-dependence in a crowdsourced elicitation task. In Proceedings of the 10th Linguistic Annotation Workshop (LAW X) (pp.… find
  41. Sanders, T. J., Demberg, V., Hoek, J., Scholman, M. C., Torabi Asr, F., Zufferey, S., & Evers-Vermuel, J. (Submitted). Unifying dimensions in discourse relations: How various annotation frameworks are related. Corpus Linguistics and Linguistic Theory, . find
  42. Sanders, T. J., Demberg, V., Hoek, J., Scholman, M. C., Zufferey, S., & Evers-Vermuel, J. (2016). How can we relate various annotation schemes? unifying dimensions in discourse relations. In TextLink Second Action Conference (pp. 110–112). find
  43. Sanders, T. J., Spooren, W. P., & Noordman, L. G. (1992). Toward a taxonomy of coherence relations. Discourse Processes, 15, 1–35. find
  44. Scholman, M. C., & Demberg, V. (2017). Crowdsourcing discourse interpretations: On the influence of context and the reliability of a connective insertion task. In Proceedings of the 11th Linguistic Annotation Workshop (LAW) (pp. 24–33). find
  45. Scholman, M. C., Evers-Vermeul, J., & Sanders, T. J. (2016). Categories of coherence relations in discourse annotation: Towards a reliable categorization of coherence relations. Dialogue & Discourse, 7, 1–28. find our card
  46. Snow, R., O’Connor, B., Jurafsky, D., & Ng, A. Y. (2008). Cheap and fastβ€”but is it good?: Evaluating non-expert annotations for natural language tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 254–263). Association for Computational Linguistic… find
  47. Stab, C., & Gurevych, I. (2014). Identifying argumentative discourse structures in persuasive essays. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 46–56). find our card
  48. Taboada, M., & Das, D. (2013). Annotation upon annotation: Adding signalling information to a corpus of discourse relations. Dialogue & Discourse, 4, 249–281. find our card
  49. Vergez-Couret, M., & Adam, C. (2012). Signaling Elaboration: Combining french gerund clauses with lexical cohesion cues. Discours. Revue de linguistique, psycholinguistique et informatique. A journal of linguistics, psycholinguistics and computational linguistics, . find
  50. Versley, Y. (2011). Multilabel tagging of discourse relations in ambiguous temporal connectives. In International Conference on Recent Advances in Natural Language Processing (RANLP) (pp. 154–161). find
  51. Webber, B. (2013). What excludes an alternative in coherence relations. In Proceedings of the International Conference on Computational Semantics (IWCS). find our card
  52. Webber, B., Knott, A., & Joshi, A. (2001). Multiple discourse connectives in a lexicalized grammar for discourse. In Computing Meaning (pp. 229–245). Springer. find
  53. Webber, B., Prasad, R., Lee, A., & Joshi, A. (2016). A discourse-annotated corpus of conjoined VPs. In Proceedings of the 10th Linguistic Annotation Workshop (LAW) (pp. 22–31). find
  54. Zufferey, S., & Degand, L. (2013). Annotating the meaning of discourse connectives in multilingual corpora. Corpus Linguistics and Linguistic Theory, 1, 1–24. find

↑ back to top