Elamur Research⌕
☰
home / papers / 2021 / RST Parsing from Scratch

RST Parsing from Scratch

Paper · 2021 · Thanh-Tung Nguyen, Xuan-Phi Nguyen, Shafiq Joty et al.

Summary

The paper tackles document-level discourse parsing in the Rhetorical Structure Theory (RST) framework. Earlier systems first split the text into elementary discourse units (EDUs) with a separate segmenter and often depended on handcrafted features. The authors recast the whole task as a top-down sequence of splitting decisions placed at token boundaries rather than at EDU boundaries. Under this view, segmentation is simply a split that marks a span as terminal, so one neural model produces both the EDUs and the tree. A seq2seq pointer network with a biaffine scorer selects each split point. Beam search explores high-scoring tree structures, and relation labels are assigned only after the structure is fixed. On the English RST Discourse Treebank (RST-DT), the parser sets a new state of the art in both end-to-end and gold-segmentation settings, using no handcrafted features, not even POS tags. It is also several times faster than earlier parsers, and the authors argue it should be easier to port to new languages and domains.

Key points

  • End-to-end parsing on RST-DT reaches an F1 of 46.6 on the Full (structure + label) metric, beating many parsers that are given gold EDUs.
  • With gold segmentation, Full F1 is 50.2, which is 2.1 points above the best previous system.
  • The probability of a tree is factored as structure first, then labels given structure. Multi-nuclear constituents are binarized as right-branching subtrees, and splits are decoded in depth-first order, which worked better than breadth-first.
  • In ablations, a sentence-boundary constraint during decoding proved important for long documents, and GloVe pretraining helped.
  • Speed on the RST-DT test set:
    • The GloVe gold-segmentation model parses it in 19 s, about 11.1x faster than Feng and Hirst (2014), even with that system's features precomputed.
    • The XLNet variant is roughly 2.4x faster than Yu et al. (2018).
    • The end-to-end version is still faster than baselines that receive EDUs.

Interesting ideas

  • I1. Segmentation is folded into parsing: a span that "splits" at its own right edge becomes a leaf, so one decoder decides both units and structure. This avoids errors cascading from a separate segmenter.
  • I2. Each internal node of a binarized tree maps to exactly one split decision, so predicting the decision sequence is provably the same as predicting the tree.
  • I3. Decoding structure first and labels second lets beam search operate over tree shapes alone, giving a broader search than greedy top-down parsers.
  • I4. A hard constraint that respects sentence boundaries steers a long-document decoder and noticeably improves accuracy.
  • I5. The error analysis treats confusions between close relations (Summary vs Elaboration, Explanation vs Elaboration) as arguably valid readings rather than plain mistakes.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): Clause or EDU-like units for discourse frames could come out of the same top-down splitting pass that builds the discourse structure, instead of a fixed pre-segmentation step.
  • I2 (back): This exact, reversible tree-to-sequence encoding is the kind of property the text → graph → text compressor needs to guarantee lossless round trips.
  • I3 (tree): gparse could keep a small beam of unlabeled structures and attach UD labels afterwards, making the structural choice inspectable separately from the labeling choice.
  • I4 (world): When ordering events or linking entities across a whole fable, the search can first be limited to splits that respect sentence boundaries, which narrows the space before finer cross-sentence reasoning.
  • I5 (learn): Evaluation of discourse or speech-act labels could use a list of "near-equivalent" label pairs, so silver-data disagreement among witness parsers is not counted as hard error.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢SalesforceSan Francisco · United States37 works · 15,019 citations🎓Nanyang Technological UniversitySingapore · Singapore73 works · 24,390 citations🔬Institute for Infocomm ResearchSingapore · Singapore33 works · 14,373 citations🏛️Agency for Science, Technology and ResearchSingapore · Singapore19 works · 13,974 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Discourse analysis · 394 works

Cite

Cite as Thanh-Tung Nguyen, Xuan-Phi Nguyen, Shafiq Joty and Xiaoli Li. 2021. RST Parsing from Scratch. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.naacl-main.128

@inproceedings{nguyen2021rst,
  title = {{RST Parsing from Scratch}},
  author = {Thanh-Tung Nguyen, Xuan-Phi Nguyen, Shafiq Joty and Xiaoli Li},
  booktitle = {Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2021},
  doi = {10.18653/v1/2021.naacl-main.128},
  url = {https://doi.org/10.18653/v1/2021.naacl-main.128}
}

References

42 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Michael Lyu. 2020. Discern: Discourse-aware entailment reasoning network for conversational machine reading. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2439–2449, Online. Association for Computational Linguistics. find
  2. Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv:2004.05150. ↗
  3. Parminder Bhatia, Yangfeng Ji, and Jacob Eisenstein. 2015. Better document-level sentiment analysis from RST discourse parsing. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2212–2218, Lisbon, Portugal. Association for Computational Linguistics. ↗
  4. Chloé Braud, Maximin Coavoux, and Anders Søgaard. 2017. Cross-lingual RST discourse parsing. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 292–304, Valencia, Spain. Association for Computational Linguistics. ↗
  5. Chloé Braud, Barbara Plank, and Anders Søgaard. 2016. Multi-view and multi-task training of RST discourse parsers. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1903–1913, Osaka, Japan.
  6. The COLING 2016 Organizing Committee. find
  7. Lynn Carlson and Daniel Marcu. 2001. Discourse tagging reference manual. Technical Report ISI-TR-545, University of Southern California Information Sciences Institute. find
  8. James Cross and Liang Huang. 2016. Span-based constituency parsing with a structure-label system and provably optimal dynamic oracles. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1–11, Austin, Texas. Association for Computational Linguistics.
  9. Timothy Dozat and Christopher D. Manning. 2017. Deep biaffine attention for neural dependency parsing. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings.
  10. Vanessa Wei Feng and Graeme Hirst. 2014. A linear-time bottom-up discourse parser with constraints and post-editing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 511– 521, Baltimore, Maryland. Association for Computational Linguistics.
  11. Seeger Fisher and Brian Roark. 2007. The utility of parse-derived features for automatic discourse segmentation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 488–495, Prague, Czech Republic. Association for Computational Linguistics. find
  12. Yifan Gao, Chien-Sheng Wu, Jingjing Li, Shafiq Joty, Steven C.H. Hoi, Caiming Xiong, Irwin King, and Shima Gerani, Yashar Mehdad, Giuseppe Carenini, Raymond T. Ng, and Bita Nejat. 2014. Abstractive summarization of product reviews using discourse structure. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1602–1613, Doha, Qatar. Association for Computational Linguistics.
  13. Hugo Hernault, Helmut Prendinger, David A. duVerle, and Mitsuru Ishizuka. 2010. Hilda: A discourse parser using support vector machine classification. Dialogue and Discourse, 1(3):1–33.
  14. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. ↗
  15. Yangfeng Ji and Jacob Eisenstein. 2014. Representation learning for text-level discourse parsing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13–24, Baltimore, Maryland. Association for Computational Linguistics.
  16. Yangfeng Ji and Noah A. Smith. 2017. Neural discourse structure for text categorization. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 996–1005, Vancouver, Canada. Association for Computational Linguistics.
  17. Shafiq Joty, Giuseppe Carenini, and Raymond Ng. 2012. A novel discriminative framework for sentence-level discourse analysis. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL’12, pages 904–915, Jeju Island, Korea. ACL. find
  18. Shafiq Joty, Giuseppe Carenini, and Raymond T. Ng. 2015. CODRA: A novel discriminative framework for rhetorical analysis. Computational Linguistics, 41(3):385–435. ↗
  19. Shafiq Joty, Giuseppe Carenini, Raymond T. Ng, and Yashar Mehdad. 2013. Combining Intra- and Multi-sentential Rhetorical Parsing for Document-level Discourse Analysis. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, ACL’13, pages 486–496, Sofia, Bulgaria. ACL.
  20. Shafiq Joty, Francisco Guzmán, Lluís Màrquez, and Preslav Nakov. 2017. Discourse structure in machine translation evaluation. Computational Linguistics, 43(4):683–722. ↗
  21. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. ↗
  22. Naoki Kobayashi, Tsutomu Hirao, Hidetaka Kamigaito, Manabu Okumura, and Masaaki Nagata. 2020. Top-down rst parsing utilizing granularity levels in documents. In Proceedings of the 2020 Conference on Artificial Intelligence for the American (AAAI), pages 8099–8106. find
  23. Jing Li, Aixin Sun, and Shafiq Joty. 2018. Segbot: A generic neural text segmentation model with pointer network. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pages 4166–4172. International Joint Conferences on Artificial Intelligence Organization. find
  24. Qi Li, Tianshi Li, and Baobao Chang. 2016. Discourse parsing with attention-based hierarchical neural networks. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 362–371, Austin, Texas. Association for Computational Linguistics.
  25. Xiang Lin, Shafiq Joty, Prathyusha Jwalapuram, and M Saiful Bari. 2019. A unified linear-time framework for sentence-level discourse parsing. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4190– 4200, Florence, Italy. Association for Computational Linguistics.
  26. Carlson Lynn, Daniel Marcu, and Mary Ellen Okurowski. 2002. Rst discourse treebank (rst–dt) ldc2002t07. Linguistic Data Consortium. find
  27. Xuezhe Ma, Zecong Hu, Jingzhou Liu, Nanyun Peng, Graham Neubig, and Eduard Hovy. 2018. Stack-pointer networks for dependency parsing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1403–1414, Melbourne, Australia. Association for Computational Linguistics. ↗
  28. Andrew L. Maas, Awni Y. Hannun, and Andrew Y. Ng. 2013. Rectifier nonlinearities improve neural network acoustic models. In in ICML Workshop on Deep Learning for Audio, Speech and Language Processing. find
  29. William C Mann and Sandra A Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text, 8(3):243–281. ↗
  30. Daniel Marcu. 2000. The rhetorical parsing of unrestricted texts: a surface-based approach. Computational Linguistics, 26(3):395–448. ↗
  31. Mathieu Morey, Philippe Muller, and Nicholas Asher. 2017. How much progress have we made on RST discourse parsing? a replication study of recent results on the RST-DT. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1319–1324, Copenhagen, Denmark. Association for Computational Linguistics. ↗
  32. Mathieu Morey, Philippe Muller, and Nicholas Asher. 2018. A dependency perspective on RST discourse parsing and evaluation. Computational Linguistics, 44(2):197–235. ↗
  33. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. ↗
  34. Radu Soricut and Daniel Marcu. 2003. Sentence level discourse parsing using syntactic and lexical information. In Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pages 228–235.
  35. Mitchell Stern, Jacob Andreas, and Dan Klein. 2017. A minimal span-based neural constituency parser. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, pages 818–827.
  36. Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015. Pointer networks. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28, pages 2692–2700. Curran Associates, Inc. find
  37. Yizhong Wang, Sujian Li, and Houfeng Wang. 2017. A two-stage parsing method for text-level discourse analysis. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 184–188, Vancouver, Canada. Association for Computational Linguistics. ↗
  38. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in Neural Information Processing Systems, volume 32, pages 5753–5763. Curran Associates, Inc. ↗
  39. Nan Yu, Meishan Zhang, and Guohong Fu. 2018. Transition-based neural RST parsing with implicit syntax features. In Proceedings of the 27th International Conference on Computational Linguistics, pages 559–570, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find
  40. Amir Zeldes, Debopam Das, Erick Galani Maziero, Juliano Antonio, and Mikel Iruskieta. 2019. find
  41. The DIS-RPT 2019 shared task on elementary discourse unit segmentation and connective detection. In Proceedings of the Workshop on Discourse Relation Parsing and Treebanking 2019, pages 97–104, Minneapolis, MN. Association for Computational Linguistics. find
  42. Longyin Zhang, Yuqing Xing, Fang Kong, Peifeng Li, and Guodong Zhou. 2020. A top-down neural architecture towards text-level parsing of discourse rhetorical structure. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6386–6395, Online. Association for Computational Linguistics.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nguyen-2021-rst-parsing-scratch/