Elamur Research⌕
☰
home / papers / 2020 / Coreference for Discourse Parsing: A Neural Approach

Coreference for Discourse Parsing: A Neural Approach

Summary

The authors examine whether coreference information helps neural parsing of documents into Rhetorical Structure Theory (RST) trees. Their motivation comes from discourse theory, notably Veins theory, which holds that tree structure limits where antecedents of anaphors may appear, so coreference links could in turn narrow the parser's search space. They build a greedy shift-reduce parser with two MLP classifiers, one for action and nuclearity and one for relation, on top of SpanBERT-Base. Three levels of coupling are compared. The first is a baseline with no coreference input. The second injects clusters predicted by a pretrained SpanBERT coreference resolver into the input embeddings through attention and gating. The third is a multitask model in which a shared SpanBERT encoder is trained jointly on discourse parsing (RST-DT) and coreference (OntoNotes 5.0). Even the baseline exceeds earlier systems and comes close to human agreement. However, neither coreference-aware variant yields a statistically significant gain, and both show greater variance. The study is a useful negative result for discourse–coreference integration, though it covers only one dataset.

Key points

  • Evaluation uses RST-DT with RST-Parseval micro precision. Reported scores are the mean of five single-model runs.
  • SpanBERT-NoCoref reaches 87.8 / 75.8 / 63.4 (structure / nuclearity / relation). SpanBERT-CorefFeats gets 88.1 / 76.1 / 63.6, and SpanBERT-MultiTask gets 87.9 / 75.9 / 63.3. Human agreement is 88.3 / 77.3 / 65.4.
  • The baseline beats the previous best system, the D2P2S2E ensemble (87.0 / 74.6 / 60.0), especially on relation labels. The relation classifier covers 18 coarse-grained RST relations.
  • The authors offer several possible reasons for the null result:
    • SpanBERT may already encode the relevant coreference signal.
    • Coreference may be only a weak cue for discourse structure.
    • Predicted coreference clusters may be too noisy.
    • The ways of fusing the two tasks may be inadequate.
  • A case study (wsj_0631) shows the baseline producing a subtree that strict Veins theory would forbid when gold coreference is available.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis · Coreference resolution

Cited by

14 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 7 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.codi-1.17

@inproceedings{guz2020coreference,
  title = {{Coreference for Discourse Parsing: A Neural Approach}},
  author = {Guz, Grigorii and Carenini, Giuseppe},
  booktitle = {Proceedings of the First Workshop on Computational Approaches to Discourse},
  year = {2020},
  doi = {10.18653/v1/2020.codi-1.17},
  url = {https://doi.org/10.18653/v1/2020.codi-1.17}
}

References

34 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ralph Weischedel et al. 2013. Ontonotes release 5.0. Linguistic Data Consortium. find
  2. Nicholas Asher and Alex Lascarides. 2003. Logics of Conversation. Cambridge University Press. find
  3. Parminder Bhatia, Yangfeng Ji, and Jacob Eisenstein. 2015. Better document-level sentiment analysis from rst discourse parsing. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2212–2218. find our card
  4. Lynn Carlson, Mary Ellen Okurowski, and Daniel Marcu. 2002. RST discourse treebank. Linguistic Data Consortium, University of Pennsylvania. find
  5. Tuhin Chakrabarty, Christopher Hidey, Smaranda Muresan, Kathleen McKeown, and Alyssa Hwang. 2019. Ampersand: Argument mining for persuasive online discussions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natu… find
  6. Dan Cristea, Nancy Ide, and Laurent Romary. 1998. Veins theory: A model of global discourse cohesion and coherence. In 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 1, pages 281–285, Montreal, Quebec, Canad… find
  7. Zeyu Dai and Ruihong Huang. 2018. Improving implicit discourse relation classification by modeling inter-dependencies of discourse units in a paragraph. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find our card
  8. Zeyu Dai and Ruihong Huang. 2019. A regularization approach for incorporating event knowledge and coreference relations into neural discourse parsing. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Langu… find our card
  9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. find
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  11. Shima Gerani, Yashar Mehdad, Giuseppe Carenini, Raymond T Ng, and Bita Nejat. 2014. Abstractive summarization of product reviews using discourse structure. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1602–1613. find
  12. Dan Hendrycks and Kevin Gimpel. 2016. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415. find
  13. Hugo Hernault, Helmut Prendinger, Mitsuru Ishizuka, et al. 2010. Hilda: A discourse parser using support vector machine classification. Dialogue & Discourse, 1(3). find our card
  14. Alexander Hogenboom, Flavius Frasincar, Franciska De Jong, and Uzay Kaymak. 2015. Using rhetorical structure in sentiment analysis. Commun. ACM, 58(7):69–77. find
  15. Patrick Huber. 2019. Discourse-sentiment alignment tool (dsat). find
  16. Yangfeng Ji and Jacob Eisenstein. 2014. Representation learning for text-level discourse parsing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 13–24. find our card
  17. Yangfeng Ji and Noah A Smith. 2017. Neural discourse structure for text categorization. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 996–1005. find
  18. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020. Spanbert: Improving pre-training by representing and predicting spans. Transactions of the Association for Computational Linguistics, 8:64–77. find our card
  19. Shafiq Joty, Giuseppe Carenini, and Raymond T Ng. 2015. Codra: A novel discriminative framework for rhetorical analysis. Computational Linguistics, 41(3). find our card
  20. Naoki Kobayashi, Tsutomu Hirao, Hidetaka Kamigaito, Manabu Okumura, and Masaaki Nagata. 2020. Top-down rst parsing utilizing granularity levels in documents. Proceedings of the AAAI Conference on Artificial Intelligence, 34:8099–8106. find
  21. Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-order coreference resolution with coarse-to-fine inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 687–… find our card
  22. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations. find
  23. (Volume 2: Short Papers), pages 184–188, Vancouver, Canada. Association for Computational Linguistics. find
  24. William C Mann and Sandra A Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text-Interdisciplinary Journal for the Study of Discourse, 8(3):243–281. find our card
  25. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtow-icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin L… find our card
  26. Daniel Marcu. 2000. The Theory and Practice of Discourse Parsing and Summarization. find
  27. Mathieu Morey, Philippe Muller, and Nicholas Asher. 2017. How much progress have we made on RST discourse parsing? a replication study of recent results on the RST-DT. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1319–1324, Copenhagen, Denmark. Ass… find our card
  28. Mathieu Morey, Philippe Muller, and Nicholas Asher. 2018. A dependency perspective on rst discourse parsing and evaluation. Computational Linguistics, 44(2):197–235. find our card
  29. Bita Nejat, Giuseppe Carenini, and Raymond Ng. 2017. Exploring joint neural model for sentence level discourse parsing and sentiment analysis. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 289–298. find our card
  30. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Te-jani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and… find our card
  31. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… find
  32. Victor Sanh, Thomas Wolf, and Sebastian Ruder. 2018. A hierarchical multi-task approach for learning embeddings from semantic tasks. find
  33. Yizhong Wang, Sujian Li, and Houfeng Wang. 2017. A two-stage parsing method for text-level discourse analysis. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics Nan Yu, Meishan Zhang, and Guohong Fu. 2018. Transition-based neural rst parsing with implicit syn… find our card
  34. Amir Zeldes. 2017. A distributional view of discourse encapsulation: Multifactorial prediction of coreference density in RST. In Proceedings of the 6th Workshop on Recent Advances in RST and Related Formalisms, pages 20–28, Santiago de Compostela, Spain. Association for Computational Linguistics. find

↑ back to top