Elamur ResearchβŒ•
☰
home / papers / 2017 / Learning Contextually Informed Representations for Linear-Time Discourse Parsing

Learning Contextually Informed Representations for Linear-Time Discourse Parsing

Summary

The paper addresses a trade-off in RST discourse parsing. Cubic-time CKY-style parsers reach top accuracy but are slow on long documents. Linear-time parsers are fast but usually depend on local, hand-engineered features. The authors present CIDER, a greedy, bottom-up, two-stage parser. It first builds a discourse tree inside each sentence over EDUs, then joins sentences into a document-level tree. At each stage, two linear-chain CRFs make the decisions: one chooses which adjacent constituents to merge, and the other assigns the rhetorical relation. Unlike standard CRFs, pairwise scores are learned from observations rather than taken from a transition matrix. Constituent features come from LSTMs that read whole sentences and documents, using an "LSTM minus" span method, so they capture wider context and long-distance dependencies. At the document level, recursive neural networks add tree-shaped representations to the flat span representations. On the RST-DT test set, CIDER matches state-of-the-art cubic-time neural parsers and is the most balanced linear-time system. It is also fast and needs only POS tagging as preprocessing. This shows that global context can be obtained without high-order parsing.

Key points

  • Evaluated on the RST Discourse Treebank (RST-DT). CIDER with additional features scores 85.0 (span), 71.1 (nuclearity), and 59.0 (relation). Human agreement is 88.7 / 77.7 / 65.8.
  • Among linear-time parsers, CIDER has the best nuclearity score. Its gains are statistically significant (p < 0.05) over Heilman and Sagae (2015) across all metrics and over Feng and Hirst (2014) for relations.
  • CIDER beats the cubic-time Li et al. (2014) on every metric. Against Li et al. (2016), it trails slightly on span and nuclearity (85.8 / 71.7) but leads on relations (58.9).
  • Parsing takes about 3.80 s per document, compared with 9.88 s for Feng and Hirst (2014) and 26.00 s for Li et al. (2014). Preprocessing is limited to POS tagging, with no syntactic parser.
  • Combining flat LSTM span representations with recursive tree-based ones gives the best results, with tree representations especially helpful for relation labeling.

Our summary of the paper, in our own words β€” not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

16 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 10 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d17-1133

@inproceedings{liu2017learning,
  title = {{Learning Contextually Informed Representations for Linear-Time Discourse Parsing}},
  author = {Liu, Yang and Lapata, Mirella},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1133},
  url = {https://doi.org/10.18653/v1/d17-1133}
}

References

34 references, 18 of them with a card of ours. β†— leads to the work, Β«findΒ» searches for one we have no id for yet.

  1. Jason Baldridge and Alex Lascarides. 2005. Probabilistic head-driven parsing for discourse structure. In Proceedings of the CoNLL conference. pages 96– 103. find
  2. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski. 2003. Building a discourse-tagged corpus in the framework of rhetorical structure theory. In Current and new directions in discourse and dialogue, Springer, pages 85–112. find our card
  3. Joyce Y. Chai and Rong Jin. 2004. Discourse structure for context question answering. In HLT-NAACL 2004: Workshop on Pragmatics of Question Answering. pages 23–30. find
  4. Vanessa Wei Feng and Graeme Hirst. 2012. Text-level discourse parsing with rich linguistic features. In Proceedings of the ACL conference. pages 60–68. find our card
  5. Vanessa Wei Feng and Graeme Hirst. 2014. A linear-time bottom-up discourse parser with constraints and post-editing. In Proceedings of the ACL conference. pages 511–521. find our card
  6. Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the 13th International Conference on Artificial Intelligence and Statistics. volume 9, pages 249–256. find
  7. Alex Graves, Jaitly Navdeep, and A-R Mohamed. 2013. Hybrid speech recognition with deep bidirectional lstm. In IEEE Workshop on Automatic Speech Recognition and Understanding. pages 293–278. find
  8. Michael Heilman and Kenji Sagae. 2015. Fast rhetorical structure theory discourse parsing. arXiv preprint arXiv:1505.02425 . find our card
  9. Hugo Hernault, Helmut Prendinger, David A DuVer-le, Mitsuru Ishizuka, and Tim Paek. 2010. Hilda: a discourse parser using support vector machine classification. Dialogue and Discourse 1(3):1–33. find our card
  10. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9(8):1735–1780. find
  11. Yangfeng Ji and Jacob Eisenstein. 2014. Representation learning for text-level discourse parsing. In Proceedings of the ACL conference. pages 13–24. find our card
  12. Shafiq Joty, Giuseppe Carenini, Raymond Ng, and Yashar Mehdad. 2013. Combining intra- and multi-sentential rhetorical parsing for document-level discourse analysis. In Proceedings of the ACL conference. pages 486–496. find
  13. Eliyahu Kiperwasser and Yoav Goldberg. 2016. Simple and accurate dependency parsing using bidirectional lstm feature representations. Transactions of the Association for Computational Linguistics 4:313–327. find our card
  14. James Cross and Liang Huang. 2016. Span-based constituency parsing with a structure-label system and provably optimal dynamic oracles. In Proceedings of the EMNLP conference. pages 1–11. find
  15. Huong LeThanh, Geetha Abeysinghe, and Christian Huyck. 2004. Generating discourse structures for written texts. In Proceedings of the 20th international conference on Computational Linguistics. page 329. find
  16. Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A. Smith. 2015. Transition-based dependency parsing with stack long short-term memory. In Proceedings of the ACL conference. pages 334–343. find our card
  17. Jiwei Li, Rumeng Li, and Eduard Hovy. 2014. Recursive deep models for discourse parsing. In Proceedings of the EMNLP conference. pages 2061–2069. find our card
  18. Bradley Efron and Robert J. Tibshirani. 1993. An Introduction ot the Bootstrap. Chapman & Hall, New York, NY. find
  19. Qi Li, Tianshi Li, and Baobao Chang. 2016. Discourse parsing with attention-based hierarchical neural networks. In Proceedings of the EMNLP conference. pages 362–371. find our card
  20. William C Mann and Sandra A Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text-Interdisciplinary Journal for the Study of Discourse 8(3):243–281. find our card
  21. Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of the ACL conference: System Demonstrations. pages 55–60. find our card
  22. Wenhui Wang and Baobao Chang. 2016. Graph-based dependency parsing with bidirectional lstm. In Proceedings of the ACL conference. pages 2306–2315. find
  23. Yasuhisa Yoshida, Jun Suzuki, Tsutomu Hirao, and Masaaki Nagata. 2014. Dependency-based discourse parser for single-document summarization. In Proceedings of the EMNLP conference. pages 1834–1839. find our card
  24. Daniel Marcu. 2000. The rhetorical parsing of unrestricted texts: A surface-based approach. Computational linguistics 26(3):395–448. find our card
  25. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 . find
  26. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind K Joshi, and Bonnie L Webber. 2008. The Penn discourse treebank 2.0. In Proceedings of LREC. find our card
  27. Kenji Sagae. 2009. Analysis of discourse structure with syntactic dependencies and data-driven shift-reduce parsing. In Proceedings of the 11th International Conference on Parsing Technologies. pages 81–84. find
  28. Richard Socher, Brody Huval, Christopher D. Manning, and Andrew Y. Ng. 2012. Semantic compositionality through recursive matrix-vector spaces. In Proceedings of the EMNLP conference. pages 1201– 1211. find
  29. Swapna Somasundaran. 2010. Discourse-level Relation for Opinion Analysis. Ph.D. thesis, University of Pittsburgh. find
  30. Radu Soricut and Daniel Marcu. 2003. Sentence level discourse parsing using syntactic and lexical information. In Proceedings of the NAACL conference. pages 149–156. find our card
  31. Rajen Subba and Barbara Di Eugenio. 2009. An effective discourse parser that uses rich linguistic information. In Proceedings of the ACL conference. pages 566–574. find our card
  32. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of the NIPS conference. pages 3104–3112. find
  33. Oriol Vinyals, Łukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2015a. Grammar as a foreign language. In Proceedings of the NIPS conference. pages 2773–2781. find
  34. Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015b. Show and tell: A neural image caption generator. In Proceedings of the CVPR conference. pages 3156–3164. find our card

↑ back to top