Elamur Research
☰
home / papers / 2003 / Sentence Level Discourse Parsing using Syntactic and Lexical Information

Sentence Level Discourse Parsing using Syntactic and Lexical Information

What it is about

The paper builds sentence-level RST discourse trees in two steps. First a probabilistic segmenter finds elementary discourse units inside a sentence, using syntactic and lexical features. Then a probabilistic parser builds the tree, using "dominance sets" taken from the Penn Treebank parse. The data are 385 WSJ articles with RST annotation, used with a training part of 5,809 sentence triples and a test part of 946. On the test set, boundary recall rises from 77.1% (the earlier decision-based segmenter) to 82.7%, and the discourse parser's F-score rises from 37.2% to 49.0% with 18 labels (an error reduction of 18.8%). Human agreement is 77.0%. With perfect syntax and perfect segmentation together, the F-score reaches 75.5%. The authors conclude that segmentation, not the parser, is the bottleneck.

Ideas

For Mova

The chapter is languages/en/36-discourse-relations.md, and languages/en/24-coordination-subordination.md for the clause boundaries. No discourse module exists in the code yet, so check with grep first. The natural place is a new module in prag that reads the UD tree from en/src/parse.rs. The measure is boundary P/R/F1 on a held-out set. The RST corpus is licensed through its own distributor, so use it for evaluation only.

Library summary

The Research library's own summary of the paper, in our words — not the paper's text.

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/N03-1030.pdf

Summary

The paper addresses how to automatically build discourse trees for single sentences in the Rhetorical Structure Theory framework. The authors split the task into two stages. First, a sentence is segmented into elementary discourse units (edus). Then those units are combined into a binary tree, labelled with rhetorical relations and nuclearity. Both stages use probabilistic models trained on lexicalized syntactic parse trees. This is possible because the RST-DT corpus of Wall Street Journal articles is aligned with Penn Treebank syntax. The segmenter estimates a boundary probability for each word from lexicalized syntactic rules around its head node. The parser scores candidate trees using syntactic and lexical dominance relations between spans. With automatic parses from Charniak's parser and automatic segmentation, the system clearly beats Marcu's (2000) decision-based parser. Given gold syntax and gold edus, it comes within about 1.5 F-score points of agreement between human annotators. The work shows that sentence-internal discourse structure can largely be derived from syntax. It also shows that segmentation, not syntactic parsing, is the main bottleneck.

Key points

  • Data: RST-DT has 385 WSJ articles, split into 347 for training and 38 for testing. About 95% of sentences have a well-formed discourse sub-tree, which gives 5809 training and 946 test sentences. There are 110 fine-grained rhetorical relations, also grouped into 18 coarser classes.
  • Fully automatic pipeline: Charniak's parser (about 90% accurate) and the discourse segmenter (83% accurate) are used. The system reaches an F-score of 49.0% with 18 labels and 45.6% with 110 labels. Marcu (2000) scores 37.2% and 35.5%, so the error reductions are 18.8% and 15.7%. A right-branching ELABORATION-NS baseline scores 23.4% and 20.7%.
  • Oracle experiments: These use the 18-label setting.
    • Gold Penn Treebank trees raise the F-score to 56.4%.
    • Gold edus raise it to 63.8%, so perfect segmentation helps about twice as much as perfect syntax.
    • With both, the scores are 75.5% (18 labels) and 70.3% (110 labels). Human agreement is 77.0% and 71.9%.
  • Unlabeled structure: With gold inputs, the parser's unlabeled tree structure reaches an F-score of 96.2%, close to human level.
  • Takeaway: Syntax and discourse are strongly correlated within a sentence, even though RST-DT annotators were not told to follow syntax. Better discourse segmentation is the most promising way to improve results.

Topics: Discourse analysis

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1073445.1073475

@inproceedings{soricut2003sentence,
  title = {{Sentence Level Discourse Parsing using Syntactic and Lexical Information}},
  author = {Soricut, Radu and Marcu, Daniel},
  booktitle = {Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2003},
  doi = {10.3115/1073445.1073475},
  url = {https://doi.org/10.3115/1073445.1073475}
}

References

16 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. E. Black, S. Abney, D. Flickinger, C. Gdaniec, R. Grishman, P. Harrison, D. Hindle, R. Ingria, F. Jelinek, J. Klavans, M. Liberman, M. Marcus, S. Roukos, B. Santorini, and T. Strzalkowski. 1991. A procedure for quantitatively comparing the syntactic coverage of English grammars. In Proceedings of Speech and Natural Language Workshop, pages 306–311, Pacific Groove, CA. DARPA. find
  2. L. Carlson, D. Marcu, and M. E. Okurowski. 2003. Building a discourse-tagged corpus in the framework of Rhetorical Structure Theory. In Jan van Kuppevelt and Ronnie Smith, editors, Current Directions in Discourse and Dialogue. Kluwer Academic Publishers. To appear. find our card
  3. Eugene Charniak. 2000. A maximum-entropy-inspired parser. In Proceedings of the NAACL 2000, pages 132– 139, Seattle, Washington, April 29 – May 3. find
  4. Michael Collins. 2000. Discriminative reranking for natural language parsing. In Proceedings of ICML 2000, Stanford University, Palo Alto, CA, June 29–July 2. find our card
  5. C. J. Fillmore, C. F. Baker, and S. Hiroaki. 2002. The framenet database and software tools. In Proceedings of the LREC 2002, pages 1157–1160, LREC. find
  6. K. Forbes, E. Miltsakaki, R. Prasad, A. Sarkar, A. Joshi, and B. Webber. 2001. D-LTAG System: Discourse parsing with a lexicalized tree-adjoining grammar. In ESSLLI’2001 Workshop on Information Structure, Discourse Structure and Discourse Semantics. find
  7. Daniel Gildea and Daniel Jurafsky. 2002. Automatic labeling of semantic role. Computational Linguistics, 28(3):245–288. find
  8. Paul Kingsbury and Martha Palmer. 2002. From Treebank to Propbank. In Proceedings of the LREC 2002, Las Palmas, Canary Islands, Spain, May 28-June 3. find our card
  9. David M. Magerman. 1995. Statistical decision-tree models for parsing. In Proceedings of the ACL 1995, pages 276–283, Cambridge, Massachusetts, June 26-30. find
  10. William C. Mann and Sandra A. Thompson. 1988. Rhetorical Structure Theory: Toward a functional theory of text organization. Text, 8(3):243–281. ↗ our card
  11. Daniel Marcu. 2000. The Theory and Practice of Discourse Parsing and Summarization. The MIT Press, Cambridge, Massachusetts. find
  12. M. Marcus, B. Santorini, and M. Marcinkiewicz. 1993. Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics, 19(2):313–330. find our card
  13. Senko K. Maynard. 1998. Principles of Japanese Discourse: A Handbook. Cambridge University Press. find
  14. David D. Palmer and Marti A. Hearst. 1997. Adaptive multilingual sentence boundary disambiguation. Computational Linguistics, 23(2):241–269, June. find
  15. Adwait Ratnaparkhi. 1998. Maximum Entropy Models for Natural Language Ambiguity Resolution. Ph.D. thesis, University of Pennsylvania. find
  16. RST-DT. 2002. RST Discourse Treebank. Linguistic Data Consortium. http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2002T07. ↗