Elamur Research⌕
☰
home / papers / 2013 / Annotation upon Annotation: Adding Signalling Information to a Corpus of Discourse Relations

Annotation upon Annotation: Adding Signalling Information to a Corpus of Discourse Relations

Summary

Studies of how discourse relations are recognized have mostly looked at one cue type, discourse markers. This has led to the common view that most relations are implicit. Taboada and Das challenge this view by placing a new layer of annotation over Carlson et al.'s (2002) RST Discourse Treebank, a resource that already labels rhetorical relations following Rhetorical Structure Theory. They built a classification of signalling devices that goes beyond connectives and includes syntactic and other cues. They then marked which signals accompany each relation in a pilot sample of about a tenth of the corpus. The authors also discuss reliability, mismatches between the existing segmentation and their own views, and the difficulty of linking signals to relations. Their main finding is that most relations in the sample carry some overt signal, and many have more than one. Truly implicit relations therefore appear to be much rarer than earlier work suggested. The resource is intended to support psycholinguistic experiments that manipulate cues other than markers, and to support discourse parsing across all text levels.

Key points

  • The base corpus is the RST Discourse Treebank. The Penn Discourse Treebank is described as the only other resource with signalling information, and it covers mainly discourse markers, including AltLex phrases.

  • The work is a pilot covering roughly 10% of the corpus. Extending it to the full corpus will require a way to layer annotations on top of the treebank's LISP-style notation.

  • Clause-internal relations are largely signalled by syntax:

    • reported speech patterns: 223 cases, 220 of them Attribution
    • relative clauses: 112
    • reduced relative clauses: 55
    • infinitival or noun clauses: 27
    • participial clauses: 19

    Most of these map to Elaboration subtypes.

  • Agreement was high for these syntactic cues. However, the authors argue that such relations are syntactic rather than genuine discourse relations, and they question the treebank's decision to treat all clause types as EDUs.

  • The authors noticed apparent errors in the original RST labels and nuclearity but left them unchanged, because the corpus is a widely used standard.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis · Treebanks

Cited by

28 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 22 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.5087/dad.2013.211

@inproceedings{taboada2013annotation,
  title = {{Annotation upon Annotation: Adding Signalling Information to a Corpus of Discourse Relations}},
  author = {Taboada, Maite and Das, Debopam},
  booktitle = {Dialogue \& Discourse Volume 4},
  year = {2013},
  doi = {10.5087/dad.2013.211},
  url = {https://doi.org/10.5087/dad.2013.211}
}

References

83 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Afantenos, Stergos D., Denis, Pascal, Muller, Philippe, & Danlos, Laurence. (2010). Learning recursive segments for discourse parsing. International Conference on Language Resources and Evaluation (LREC). Malta. ↗ our card
  2. Asher, Nicholas, & Lascarides, Alex. (2003). Logics of Conversation. Cambridge: Cambridge University Press. find
  3. Baldridge, Jason, Asher, Nicholas, & Hunter, Julie. (2007). Annotation for and robust parsing of discourse structure on unrestricted texts. Zeitschrift für Sprachwissenschaft(26), 213-239. ↗
  4. Bateman, John, Kamps, Thomas, Kleinz, Jörg, & Reichenberger, Klaus. (2001). Towards constructive text, diagram, and layout generation for information presentation. Computational Linguistics, 27(3), 409-449. ↗
  5. Blakemore, Diane. (1987). Semantic Constraints on Relevance. Oxford: Blackwell. find
  6. Blakemore, Diane. (1992). Understanding Utterances: An Introduction to Pragmatics. Oxford: Blackwell. find
  7. Blakemore, Diane. (2002). Relevance and Linguistic Meaning: The Semantics and Pragmatics of Discourse Markers. Cambridge: Cambridge University Press. find
  8. Britton, Bruce K., Glynn, Shawn M., Meyer, Bonnie J. F., & Penland, M. J. (1982). Effects of text structure on use of cognitive capacity during reading. Journal of Educational Psychology, 74(1), 51-61. ↗
  9. Carlson, Lynn, & Marcu, Daniel. (2001). Discourse Tagging Manual. Manual. http://www.isi.edu/~marcu/discourse/tagging-ref-manual.pdf. ↗
  10. Carlson, Lynn, Marcu, Daniel, & Okurowski, Mary Ellen. (2002). RST Discourse Treebank, LDC2002T07 [Corpus]. Philadelphia, PA: Linguistic Data Consortium. find
  11. Corston-Oliver, Simon. (1998a). Beyond string matching and cue phrases: Improving efficiency and coverage in discourse analysis. Proceedings of AAAI 1998 Spring Symposium Series, Intelligent Text Summarization (pp. 9-15). Madison, Wisconsin. find
  12. Corston-Oliver, Simon. (1998b). Identifying the linguistic correlates of rhetorical relations. Proceedings of the Workshop on Discourse Relations and Discourse Markers, COLING/ACL '98 (pp. 8-14). find
  13. Cristea, Dan, Ide, Nancy, & Romary, Laurent. (1998). Veins Theory: A model of global discourse cohesion and coherence. Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and the 17th International Conference on Computational Linguistics (ACL-98/COLING-98) (pp. 281-285). Montréal, Canada. find
  14. Cunningham, Hamish, Maynard, Diana, Bontcheva, Kalina, Tablan, Valentin, Aswani, Niraj, Roberts, Ian, . . . Peters, Wim. (2011). Text Processing with GATE (6th ed.). Sheffield, UK: Gate. find
  15. da Cunha, Iria, San Juan, Eric, Torres-Moreno, Juan Manuel, Cabré, María Teresa, & Sierra, Gerardo. (2012). A symbolic approach for automatic detection of nuclearity and rhetorical relations among intra-sentence discourse segments in Spanish. Proceedings of CICLing (pp. 462-474). New Delhi, India. find
  16. Dale, Robert. (1991). The role of punctuation in discourse structure. Proceedings of AAAI Fall Symposium on Discourse Structure in Natural Language Understanding and Generation (pp. 13-14). Asilomar, CA. find
  17. Degand, Liesbeth, & Sanders, Ted. (2002). The impact of relational markers on expository text comprehension in L1 and L2. Reading and Writing, 15(7-8), 739-758. ↗
  18. Fellbaum, Christiane (Ed.). (1998). WordNet: An Electronic Lexical Database. Cambridge, MA: MIT Press. find
  19. Fraser, Bruce. (1990). An approach to discourse markers. Journal of Pragmatics, 14(3), 383-398. ↗
  20. Fraser, Bruce. (1999). What are discourse markers? Journal of Pragmatics, 31(7), 931-952. ↗
  21. Fraser, Bruce. (2006). Towards a theory of discourse markers. In K. Fischer (Ed.), Approaches to Discourse Particles (pp. 189-204). Amsterdam: Elsevier. ↗
  22. Fraser, Bruce. (2009). An account of discourse markers. International Review of Pragmatics, 1, 293-320. ↗
  23. Grice, H. Paul. (1975). Logic and conversation. In P. Cole & J. L. Morgan (Eds.), Speech Acts. Syntax and Semantics, Volume 3 (pp. 41-58). New York: Academic Press. ↗
  24. Haberlandt, Karl. (1982). Reader expectations in text comprehension. In J.-F. Le Ny & W. Kintsch (Eds.), Language and Comprehension (pp. 239-249). Amsterdam: North-Holland. ↗
  25. Halliday, Michael A. K., & Hasan, Ruqaiya. (1976). Cohesion in English. London: Longman. find
  26. Hanneforth, Thomas, Heintze, Silvan, & Stede, Manfred. (2003). Rhetorical parsing with underspecification and forests. Proceedings of HLT-NAACL 2003. Edmonton, Canada. find
  27. Hernault, Hugo, Bollegala, Danushka, & Ishizuka, Mitsuru. (2011). Semi-supervised discourse relation classification with structural learning. Proceedings of the 12th international conference on Computational linguistics and intelligent text processing (CICLing '11). Tokyo, Japan. find
  28. Hernault, Hugo, Prendinger, Helmut, duVerle, David A., & Ishizuka, Mitsuru. (2010). HILDA: A discourse parser using Support Vector Machine classification. Dialogue and Discourse, 1(3). find our card
  29. Hirschberg, Julia, & Litman, Diane J. (1993). Empirical studies on the disambiguation of cue phrases. Computational Linguistics, 19(3), 501-530. find
  30. Horn, Laurence. (1984). Toward a new taxonomy for pragmatic inference: Q-based and R-based implicature. In D. Schiffrin (Ed.), Meaning, form and use in context: Linguistic Implications (pp. 11-42). Washington, DC: Georgetown University Press. find
  31. Hovy, Eduard, & Arens, Yigal. (1991). Automatic generation of formatted text. Proceedings of 9th AAAI Conference. Anaheim, California. find
  32. Huong, Le Thanh. (2007). An approach in automatically generating discourse structure of text. Journal of Computer Science and Cybernetics, 23(3), 212-230. find
  33. Karamanis, Nikiforos. (2007). Supplementing entity coherence with local rhetorical relations for information ordering. Journal of Logic, Language and Information, 16, 445-464. ↗
  34. Keenan, Janice M., Baillet, Susan D., & Brown, Polly. (1984). The effects of causal cohesion on comprehension and memory. Journal of Verbal Learning and Verbal Behaviour, 23(2), 115-126. ↗
  35. Knott, Alistair. (1996). A Data-Driven Methodology for Motivating a Set of Coherence Relations. Ph.D. dissertation, University of Edinburgh, Edinburgh, UK. Retrieved from http://citeseer.nj.nec.com/knott96data.html ↗
  36. Knott, Alistair, & Dale, Robert. (1994). Using linguistic phenomena to motivate a set of coherence relations. Discourse Processes, 18(1), 35-62. ↗
  37. Knott, Alistair, Oberlander, Jon, O'Donnell, Michael, & Mellish, Chris. (2001). Beyond elaboration: The interaction of relations and focus in coherent text. In T. Sanders, J. Schilperoord & W. Spooren (Eds.), Text Representation: Linguistic and Psycholinguistic Aspects (pp. 181-196). Amsterdam and Philadelphia: John Benjamins. find
  38. Knott, Alistair, & Sanders, Ted. (1998). The classification of coherence relations and their linguistic markers: An exploration of two languages. Journal of Pragmatics, 30, 135-175. ↗
  39. Lapata, Mirella, & Lascarides, Alex. (2004). Inferring sentence-internal temporal relations. Proceedings of the North American Chapter of the Assocation of Computational Linguistics (NAACL) (pp. 153-160). Boston, MA. find our card
  40. Lin, Ziheng, Kan, Min-Yen, & Ng, Hwee Tou. (2009). Recognizing implicit discourse relations in the Penn Discourse Treebank. Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (pp. 343-351). Singapore. find our card
  41. Louis, Annie, Prasad, Rashmi, Joshi, Aravind K, & Nenkova, Ani. (2010). Using entity features to classify implicit discourse relations. Proceedings of the SIGDIAL conference (pp. 59-62). Tokyo, Japan. find our card
  42. Mann, William C., & Taboada, Maite. (2010). RST Web Site, from http://www.sfu.ca/rst ↗
  43. Mann, William C., & Thompson, Sandra A. (1988). Rhetorical Structure Theory: Toward a functional theory of text organization. Text, 8(3), 243-281. ↗ our card
  44. Marcu, Daniel. (1999). A decision-based approach to rhetorical parsing. Proceedings of 37th Annual Meeting of the Association for Computational Linguistics (ACL'99) (pp. 365-372). College Park, Maryland. find
  45. Marcu, Daniel. (2000a). The rhetorical parsing of unrestricted texts: A surface based approach. Computational Linguistics, 26(3), 395-448. ↗ our card
  46. Marcu, Daniel. (2000b). The Theory and Practice of Discourse Parsing and Summarization. Cambridge, MA: MIT Press. ↗
  47. Marcu, Daniel, & Echihabi, Abdessamad. (2002). An unsupervised approach to recognising discourse relations. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL'02) (pp. 368-375). Philadelphia, Pennsylvania. find
  48. Meyer, Bonnie J. F., Brandt, David M., & Bluth, George J. (1980). Use of top-level structure in text: Key for reading comprehension in ninth-grade students. Reading Research Quarterly, 16(1), 72-103. find
  49. Millis, Keith K., & Just, Marcel A. (1994). The influence of connectives on sentence comprehension. Journal of Memory and Language, 33, 128-147. ↗
  50. Mithun, Shamima, & Kosseim, Leila. (2011). Comparing approaches to tag discourse relations. Proceedings of the 12th international conference on Computational linguistics and intelligent text processing (CICLing '11) (pp. 328-339). Tokyo, Japan. find
  51. Myers, Jerome L., Shinjo, Makiko, & Duffy, Susan A. (1987). Degree of causal relatedness and memory. Journal of Memory and Language, 26(4), 453-465. ↗
  52. O'Donnell, Michael. (1997). RST-Tool: An RST analysis tool. Proceedings of the 6th European Workshop on Natural Language Generation. Duisburg, Germany. find
  53. Pardo, Thiago Alexandre Salgueiro, & Nunes, Maria das Graças Volpe. (2008). On the development and evaluation of a Brazilian Portuguese discourse parser. Journal of Theoretical and Applied Computing, 15(2), 43-64. ↗
  54. Pitler, Emily, Louis, Annie, & Nenkova, Ani. (2009). Automatic sense prediction for implicit discourse relations in text. Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics (pp. 683-691). Suntec, Singapore. find our card
  55. Poesio, Massimo, Stevenson, Rosemary, Di Eugenio, Barbara, & Hitzeman, Janet. (2004). Centering: A parametric theory and its instantiations. Computational Linguistics, 30(3), 309-363. ↗
  56. Polanyi, Livia, Culy, Christopher, van der Berg, Martin, Thione, Gian Lorenzo, & Ahn, David. (2004). A rule based approach to discourse parsing. Proceedings of the 5th SIGdial Workshop in Discourse and Dialogue (pp. 108-117). Cambridge, MA. find our card
  57. Polanyi, Livia, & Scha, R. J. H. (1983). The syntax of discourse. Text, 3(3), 261-270. ↗
  58. Prasad, Rashmi, Joshi, Aravind K, & Webber, Bonnie. (2010). Realization of discourse relations by other means: Alternative lexicalizations. Proceedings of COLING 2010 (pp. 1023-1031). Beijing. find our card
  59. Prasad, Rashmi, Lee, Alan, Dinesh, Nikhil, Miltsakaki, Eleni, Campion, Geraud, Joshi, Aravind K., & Webber, Bonnie. (2008). Penn Discourse Treebank Version 2.0, LDC2008T05 [Corpus]. Philadelphia, PA: Linguistic Data Consortium. find
  60. Prasad, Rashmi, Miltsakaki, Eleni, Dinesh, Nikhil, Lee, Alan, Joshi, Aravind K, Robaldo, Livio, & Webber, Bonnie. (2007). The Penn Discourse Treebank 2.0 Annotation Manual. Philadelphia: University of Pennsylvania. find our card
  61. Sanders, Ted, Land, Jentine, & Mulder, Gerben. (2007). Linguistic markers of coherence improve text comprehension in functional contexts. Information Design Journal, 15(3), 219-235. find
  62. Sanders, Ted, & Noordman, Leo. (2000). The role of coherence relations and their linguistic markers in text processing. Discourse Processes, 29(1), 37-60. ↗
  63. Sanders, Ted, Spooren, Wilbert, & Noordman, Leo. (1992). Toward a taxonomy of coherence relations. Discourse Processes, 15(1), 1-35. ↗
  64. Sanders, Ted, Spooren, Wilbert, & Noordman, Leo. (1993). Coherence relations in a cognitive theory of discourse representation. Cognitive Linguistics, 4(2), 93-133. ↗
  65. Schiffrin, Deborah. (1987). Discourse Markers. Cambridge: Cambridge University Press. find
  66. Schiffrin, Deborah. (2001). Discourse markers: Language, meaning and context. In D. Schiffrin, D. Tannen & H. E. Hamilton (Eds.), The Handbook of Discourse Analysis (pp. 54-75). Malden, MA: Blackwell. find
  67. Schilder, Frank. (2002). Robust discourse parsing via discourse markers, topicality and position. Natural Language Engineering, 8(2/3), 235-255. ↗
  68. Scott, Donia, & de Souza, Clarisse Sieckenius. (1990). Getting the message across in RST-based text generation. In R. Dale, C. Mellish & M. Zock (Eds.), Current Research in Natural Language Generation (pp. 47-73). London: Academic Press. find
  69. Siegel, Sidney, & Castellan, N.J. (1988). Nonparametric Statistics for the Behavioral Sciences. New York: McGraw-Hill. find
  70. Spooren, Wilbert. (1997). The processing of underspecified coherence relations. Discourse Processes, 24, 149-168. ↗
  71. Spooren, Wilbert, & Degand, Liesbeth. (2010). Coding coherence relations: Reliability and validity. Corpus Linguistics and Linguistic Theory, 6(2), 241-266. ↗
  72. Sporleder, Caroline, & Lascarides, Alex. (2005). Exploiting linguistic cues to classify rhetorical relations. Proceedings of Recent Advances in Natural Language Processing (RANLP-05). Borovets, Bulgaria. find
  73. Sporleder, Caroline, & Lascarides, Alex. (2008). Using automatically labelled examples to classify rhetorical relations: An assesment. Natural Language Engineering, 14(3), 369-416. find
  74. Stede, Manfred. (2007). Korpusgestützte Texanalyse: Grundzüge der Ebenen-orientierten Textlinguistik. Tübingen: Gunter Narr. find
  75. Stede, Manfred. (2012). Discourse Processing: Morgan & Claypool. find
  76. Taboada, Maite. (2004). Building Coherence and Cohesion: Task-Oriented Dialogue in English and Spanish. Amsterdam and Philadelphia: John Benjamins. find
  77. Taboada, Maite. (2006). Discourse markers as signals (or not) of rhetorical relations. Journal of Pragmatics, 38(4), 567-592. ↗
  78. Taboada, Maite. (2008). SFU Review Corpus [Corpus]. Vancouver: Simon Fraser University, http://www.sfu.ca/~mtaboada/research/SFU_Review_Corpus.html. ↗
  79. Taboada, Maite. (2009). Implicit and explicit coherence relations. In J. Renkema (Ed.), Discourse, of Course (pp. 127-140). Amsterdam and Philadelphia: John Benjamins. ↗
  80. Taboada, Maite, & Mann, William C. (2006). Rhetorical Structure Theory: Looking back and moving ahead. Discourse Studies, 8(3), 423-459. ↗
  81. Teijssen, Daphne, van Halteren, Hans, Verberne, Suzan, & Boves, Lou. (2008). Features for automatic discourse analysis of paragraphs. Proceedings of the 18th Meeting of Computational Linguistics in the Netherlands (CLIN 2007) (pp. 53-68). Leuven, Belgium. find
  82. Tofiloski, Milan, Brooke, Julian, & Taboada, Maite. (2009). A syntactic and lexical-based discourse segmenter. Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics (pp. 77-80). Singapore. find
  83. Trabasso, Tom, & Sperry, Linda L. (1985). Causal relatedness and importance of story events. Journal of Memory and Language, 24(5), 595-611. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/taboada-2013-annotation-upon-annotation-adding-signalling/