Elamur Research⌕
☰
home / papers / 2004 / Annotating Discourse Connectives and Their Arguments

Annotating Discourse Connectives and Their Arguments

Summary

The Penn Treebank and Propbank annotate structure only within sentences. Corpora built on abstract rhetorical relations, such as RST-style annotation, have proven hard to label reliably and do not record why a relation was chosen. This paper introduces the Penn Discourse TreeBank (PDTB), which takes a bottom-up, lexically anchored approach. Discourse connectives are treated as two-place predicates, analogous to verbs at sentence level, and annotators mark the spans of their two arguments.

The approach is motivated by DLTAG, but the guidelines are deliberately framework-neutral. Annotation is layered on the Penn Treebank and Propbank. It covers subordinating and coordinating conjunctions, adverbial connectives, and implicit connectives between adjacent sentences. For implicit cases, annotators also supply an explicit connective that captures the inferred relation.

Preliminary agreement studies show that argument spans can be annotated reliably. Most disagreements are partial overlaps in span boundaries, not genuine conflicts. This suggests the PDTB is a dependable resource for building and evaluating discourse-processing algorithms.

Key points

  • Planned scope: about 30,000 annotations, with roughly 20,000 for the 250 explicit connectives found in the corpus and 10,000 implicit ones. Release was expected by November 2005.
  • Data reported so far: 10 explicit connectives totalling 2,717 annotations, plus 386 implicit-connective tokens. The explicit connectives are the adverbials 'therefore', 'as a result', 'instead', 'otherwise' and 'nevertheless', together with the subordinators 'so that', 'when', 'even though', 'although' and 'because'.
  • Annotation process: four annotators per token, working in a modified version of the WordFreak tool and handling one connective at a time.
  • Agreement on implicit connectives:
    • Arguments: 85.1% exact-match agreement between two annotators (657 of 772 ARG1/ARG2 tokens), rising to 92.6% under partial match.
    • Inferred relation type: 72% agreement across five coarse groups (addition, cause-effect, temporal, contrast, restatement).
  • Agreement on explicit connectives and sources of disagreement: partial-match agreement reached about 94.5% overall. The main challenge is determining argument extent. Disagreements are classified as partial overlap (e.g., parentheticals, dependent clauses), no overlap, missing annotations, or unresolved guideline issues.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis · Treebanks

Cited by

39 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 31 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{miltsakaki2004annotating,
  title = {{Annotating Discourse Connectives and Their Arguments}},
  author = {Miltsakaki, Eleni and Joshi, Aravind and Prasad, Rashmi and Webber, Bonnie},
  booktitle = {Proceedings of the Workshop Frontiers in Corpus Annotation at HLT-NAACL 2004},
  year = {2004},
  url = {https://aclanthology.org/W04-2703/}
}

References

18 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Nicholas Asher and Alex Lascarides. 1998. The semantics and pragmatics of presupposition. Journal of Semantics, 15(3):239–300. ↗
  2. Jean Carletta. 1996. Assessing agreement on classification tasks. Computational Linguistics, 22:249–254. find
  3. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski, 2003. Current Directions in Discourse and Dialogue, chapter Building a Discourse-Tagged Corpus in the Framework of Rhetorical Structure Theory. Kluwer Academic Publishers. find our card
  4. Kate Forbes. 2003. Discourse Semantics of S-Modifying Adverbials. Ph.D. thesis, Department of Linguistics, University of Pennsylvania. find
  5. Claire Gardent. 1997. Discourse tree adjoining grammars. Claus 89, University of the Saarlandes, Saarbrucken. find
  6. Paul Kingsbury and Martha Palmer. 2002. From Treebank to Propbank. In Third International Conference on Language Resources and Evaluation, LREC-02, Las Palmas, Canary Islands, Spain. find our card
  7. Alistair Knott. 1996. A Data-Driven Methodology for Motivating a Set of Coherence Relations. Ph.D. thesis, University of Edinburgh. find
  8. William Mann and Sandra Thompson. 1988. Rhetorical structure theory. toward a functional theory of text organization. Text, 8(3):243–281. ↗ our card
  9. Mitch Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of english: the Penn Treebank. Computational Linguistics, 19:313–330. ↗ our card
  10. Eleni Miltsakaki, Cassandre Creswell, Kate Forbes, Aravind Joshi, and Bonnie Webber. 2003. Anaphoric arguments of discourse connectives: Semantic properties of antecedents versus non-antecedents. In Proceedings of the Computational Treatment of Anaphora Workshop, EACL 2003, Budapest. find
  11. Livia Polanyi and Martin van den Berg. 1996. Discourse structure and discourse interpretation. In Proceedings of the Tenth Amsterdam Colloquium, University of Amsterdam, pages 113–131. ↗
  12. Frank Schilder. 1997. Discourse tree grammar or how to get attached to a discourse? In Proceedings of the the second International Workshop on Computational Semantics (IWCS-II), Tilburg, The Netherlands, pages 261–273. find
  13. Sidney Siegel and N. J. Castellan. 1988. Nonparama-teric Statistics for the Behavioral Sciences. McGraw-Hill, 2nd edition. find
  14. Bonnie Webber and Aravind Joshi. 1998. Anchoring a lexicalized tree adjoining grammar for discourse. In ACL/COLING Workshop on Discourse Relations and Discourse Markers, Montreal, pages 8–92. Montreal, Canada. ↗
  15. Bonnie Webber, Alistair Knott, Matthew Stone, and Aravind Joshi. 1999a. Discourse relations: A structural and presuppositional account using lexicalized TAG. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics, Maryland, pages 41–48. College Park MD. find
  16. Bonnie Webber, Alistair Knott, Matthew Stone, and Aravind Joshi. 1999b. What are little texts made of? A structural and presuppositional account using lexicalized TAG. In Proceedings of the International Workshop on Levels of Representation in Discourse (LORID ’99), Edinburgh, pages 145–149. find
  17. Bonnie Webber, Alistair Knott, and Aravind Joshi. 2000. Multiple discourse connectives in a lexicalized grammar for discourse. In Proceedings of the Third International Workshop on Computational Semantics, Tilburg, The Netherlands. ↗
  18. Bonnie Webber, Matthew Stone, Aravind Joshi, and Alistair Knott. 2003. Anaphora and discourse structure. Computational Linguistics, 29:545–587. ↗ our card

↑ back to top