Elamur Research⌕
☰
home / papers / 2014 / Recursive Deep Models for Discourse Parsing

Recursive Deep Models for Discourse Parsing

Summary

The paper addresses document-level discourse parsing in the RST framework. In this task, a system must decide how clause-sized units join into a hierarchical tree and which rhetorical relation links each pair. The authors argue that hand-crafted lexical and syntactic features miss the semantic and intentional signals behind coherence. They propose a recursive neural approach instead.

Each sentence is first encoded bottom-up over its Stanford Parser tree, following the recursive network design of Socher et al. (2013). Adjacent discourse units are then combined step by step. A binary classifier estimates whether two neighbouring units should merge, and a multi-class classifier assigns the relation label. A vector is then computed for the new subtree. All components are trained jointly on gold-standard trees from RST-DT, with gold EDU segmentation assumed.

On the standard Span, Nuclearity and Relation metrics, the model performs on par with state-of-the-art parsers. It is consistently strong across all three metrics but does not lead on any single one. The work shows that learned distributed representations can replace much manual feature engineering in discourse parsing.

Key points

  • Data: RST Discourse Treebank (RST-DT), 385 Wall Street Journal documents split into 347 for training and 38 for testing. The 18 coarse-grained relation classes are used, which give 41 labels once nuclearity is included. Non-binary relations are converted into right-branching binary cascades.
  • Results: The full model with extra text-level features scores 84.0 Span, 70.8 Nuclearity and 58.6 Relation. Without those features it scores 82.4, 69.2 and 56.8. Human agreement is 88.7, 77.7 and 65.7.
  • Baselines: Feng and Hirst has the best Span score (85.7). Ji and Eisenstein has the best Nuclearity (71.1) and Relation (61.6) scores. HILDA trails at 75.3, 60.0 and 46.8.
  • Nuclearity handling: No separate classifier predicts nuclearity. It is derived from the relation label output by the multi-class classifier.
  • Ablation and tuning: A "unified" variant shares one composition function across all relation types and performs worse (82.0, 70.0, 57.1). This shows the value of relation-specific composition. The only hyperparameter, the regularization weight Q, was tuned with five-fold cross-validation on the training set.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

65 papers of our library cite it (of the 140673 whose bibliographies we read); 25 of them have a page here, 40 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/d14-1220

@inproceedings{li2014recursive,
  title = {{Recursive Deep Models for Discourse Parsing}},
  author = {Li, Jiwei and Li, Rumeng and Hovy, Eduard},
  booktitle = {Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2014},
  doi = {10.3115/v1/d14-1220},
  url = {https://doi.org/10.3115/v1/d14-1220}
}

References

39 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jason Baldridge and Alex Lascarides. 2005. Probabilistic head-driven parsing for discourse structure. In Proceedings of the Ninth Conference on Computational Natural Language Learning, pages 96–103. Association for Computational Linguistics. find
  2. Samuel R Bowman. 2013. Can recursive neural tensor networks learn logical reasoning? arXiv preprint arXiv:1312.6192. — [arXiv:1312.6192](https://arxiv.org/abs/1312.6192) find
  3. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski. 2003. Building a discourse-tagged corpus in the framework of rhetorical structure theory. Springer. find our card
  4. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. The Journal of Machine Learning Research, 12:2493–2537. find our card
  5. Ronan Collobert. 2011. Deep learning for efficient discriminative parsing. In International Conference on Artificial Intelligence and Statistics, number EPFL-CONF-192374. find
  6. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. The Journal of Machine Learning Research, 12:2121–2159. find
  7. David A Duverle and Helmut Prendinger. 2009. A novel discourse parser based on support vector machine classification. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2-Volume 2… find our card
  8. Vanessa Wei Feng and Graeme Hirst. 2012. Text-level discourse parsing with rich linguistic features. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pages 60–68. Association for Computational Linguistics. find our card
  9. Vanessa Wei Feng and Graeme Hirst. 2014. A linear time bottom-up discourse parser with constraints and post-editing. In ACL. find our card
  10. Seeger Fisher and Brian Roark. 2007. The utility of parse-derived features for automatic discourse segmentation. In ANNUAL MEETING-ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, volume 45, page 488. find
  11. Christoph Goller and Andreas Kuchler. 1996. Learning task-dependent distributed representations by backpropagation through structure. In Neural Networks, 1996., IEEE International Conference on, volume 1, pages 347–352. IEEE. find
  12. Hugo Hernault, Danushka Bollegala, and Mitsuru Ishizuka. 2010a. A semi-supervised approach to improve classification of infrequent discourse relations using feature vector extension. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 399–409. Association… find our card
  13. Hugo Hernault, Helmut Prendinger, Mitsuru Ishizuka, et al. 2010b. Hilda: a discourse parser using support vector machine classification. Dialogue & Discourse, 1(3). find our card
  14. Wang Houfeng, Longkai Zhang, and Ni Sun. 2013. Improving chinese word segmentation on microblog using rich punctuations. find
  15. Eduard H Hovy and Elisabeth Maier. 1997. Parsimonious or profligate: How many and which discourse structure relations. Discourse Processes. find
  16. Yangfeng Ji and Jacob Eisenstein. 2014. Representation learning for text-level discourse parsing. find our card
  17. Shafiq Joty, Giuseppe Carenini, and Raymond T Ng. 2012. A novel discriminative framework for sentence-level discourse analysis. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 904–915. Association for… find
  18. Shafiq Joty, Giuseppe Carenini, Raymond Ng, and Yashar Mehdad. 2013. Combining intra-and multi-sentential rhetorical parsing for document-level discourse analysis. In Proceedings of the 51st annual meeting of the association for computational linguistics (ACL), pages 486–496. find
  19. Dan Jurafsky and James H Martin. 2000. Speech & Language Processing. Pearson Education India. find
  20. Huong LeThanh, Geetha Abeysinghe, and Christian Huyck. 2004. Generating discourse structures for written texts. In Proceedings of the 20th international conference on Computational Linguistics, page 329. Association for Computational Linguistics. find
  21. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing implicit discourse relations in the penn discourse treebank. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1-Volume 1, pages 343–351. Association for Computational Linguistics. find our card
  22. Annie Louis, Aravind Joshi, and Ani Nenkova. 2010. Discourse indicators for content selection in summarization. In Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 147–156. Association for Computational Linguistics. find
  23. Minh-Thang Luong, Michael C Frank, and Mark Johnson. 2014. Parsing entire discourses as very long strings: Capturing topic continuity in grounded language learning. find
  24. William C Mann and Sandra A Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text, 8(3):243–281. find our card
  25. Daniel Marcu. 2000a. The rhetorical parsing of unrestricted texts: A surface-based approach. Computational Linguistics, 26(3):395–448. find our card
  26. Daniel Marcu. 2000b. The theory and practice of discourse parsing and summarization. MIT Press. find
  27. Wenzhe Pei, Tao Ge, and Chang Baobao. 2014. Max-margin tensor neural network for chinese word segmentation. In Proceedings of ACL. find
  28. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind K Joshi, and Bonnie L Webber. 2008. The penn discourse treebank 2.0. In LREC. Citeseer. find our card
  29. David Reitter. 2003. Simple signals for complex rhetorics: On rhetorical analysis with rich-feature support vector models. In LDV Forum, volume 18, pages 38–52. find
  30. Richard Socher, Christopher D Manning, and Andrew Y Ng. 2010. Learning continuous phrase representations and syntactic parsing with recursive neural networks. In Proceedings of the NIPS-2010 Deep Learning and Unsupervised Feature Learning Workshop, pages 1–9. find
  31. Richard Socher, Eric H Huang, Jeffrey Pennington, Andrew Y Ng, and Christopher D Manning. 2011a. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In NIPS, volume 24, pages 801–809. find
  32. Richard Socher, Cliff C Lin, Chris Manning, and Andrew Y Ng. 2011b. Parsing natural scenes and natural language with recursive neural networks. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pages 129–136. find
  33. Richard Socher, Brody Huval, Christopher D Manning, and Andrew Y Ng. 2012. Semantic compositionality through recursive matrix-vector spaces. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1201–1211. A… find
  34. Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages… find
  35. Radu Soricut and Daniel Marcu. 2003. Sentence level discourse parsing using syntactic and lexical information. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1, pages 149–156. Association for Co… find our card
  36. Caroline Sporleder and Mirella Lapata. 2005. Discourse chunking and its application to sentence compression. In Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, pages 257–264. Association for Computational Linguistics. find
  37. Rajen Subba and Barbara Di Eugenio. 2009. An effective discourse parser that uses rich linguistic information. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 566–574. Association for Comp… find our card
  38. Suzan Verberne, Lou Boves, Nelleke Oostdijk, and Peter-Arno Coppen. 2007. Evaluating discourse-based answer extraction for why-question answering. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pages 735–736. ACM. find
  39. Longkai Zhang Houfeng Wang and Xu Sun Mairgup Mansur. 2013. Exploring representations from unlabeled data with co-training for chinese word segmentation. find

↑ back to top