Elamur Research⌕
☰
home / papers / 2016 / Multi-view and multi-task training of RST discourse parsers

Multi-view and multi-task training of RST discourse parsers

Summary

RST discourse parsing, which builds document-level trees over elementary discourse units (EDUs), is limited by how little annotated training data exists. Braud, Plank and Søgaard address this by regularizing a parser with extra supervision. Their model is a hierarchical bi-LSTM that treats parsing as sequence labeling. Each EDU gets a label describing its local bracket and relation structure, and heuristics then repair and binarize the output into valid trees. Training is multi-task and draws on two kinds of auxiliary signal. The first is alternative views of the RST-DT trees themselves: a dependency encoding, nuclearity-only and relation-only labels, and the fine-grained relations. The second is related tasks built from other corpora: the PDTB, TimeBank, FactBank, OntoNotes and the Santa Barbara corpus. The best combination cuts errors by 12–15% relative to a single-task baseline. It slightly exceeds prior state of the art on unlabeled structure (Span) and is competitive on nuclearity and relation labels. This shows that output-side views and loosely related annotations can partly compensate for small discourse treebanks.

Key points

  • Data: the main task uses the RST-DT. Its 18 coarse relations combine with nuclearity to give 41 labels, and the 78 fine-grained relations serve as an auxiliary task.
  • Auxiliary views include RST-Dep, which converts trees to dependencies with the nucleus of the root node as head, plus Nuc and Lab. External tasks cover modality, factuality, aspect, polarity, tense, coreference, speaking turns and PDTB inter-sentential relations.
  • Best setup is Nuc+Lab + RSTDep + Mod (FactBank modality) + PDTB. It scores 83.62% Span F1, 69.77% Nuclearity F1 and 55.11% Relation F1, closing 60.7% of the gap to human agreement on unlabeled parsing.
  • Tense and coreference tasks gave no gains, and the PDTB task used alone slightly hurt. Factuality, aspect, modality and polarity were beneficial.
  • Code and preprocessing scripts are released at bitbucket.org/chloebt/discourse.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

20 papers of our library cite it (of the 140673 whose bibliographies we read); 11 of them have a page here, 9 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{braud2016multi,
  title = {{Multi-view and multi-task training of RST discourse parsers}},
  author = {Braud, Chloé and Plank, Barbara and Søgaard, Anders},
  booktitle = {Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers},
  year = {2016},
  url = {https://aclanthology.org/C16-1179/}
}

References

50 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rami Al-Rfou, Bryan Perozzi, and Steven Skiena. 2013. Polyglot: Distributed word representations for multilingual nlp. In Proceedings of Conll. find our card
  2. Nicholas Asher and Alex Lascarides. 2003. Logics of Conversation. Cambridge University Press. find
  3. Parminder Bhatia, Yangfeng Ji, and Jacob Eisenstein. 2015. Better document-level sentiment analysis from rst discourse parsing. In Proceedings of EMNLP. find our card
  4. Jill Burstein, Daniel Marcu, and Kevin Knight. 2003. Finding the write stuff: automatic identification of discourse structure in student essays. IEEE Intelligent Systems: Special Issue on Advances in Natural Language Processing, 18. find
  5. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski. 2001. Building a discourse-tagged corpus in the framework of rhetorical structure theory. In Proceedings of the Second SIGdial Workshop on Discourse and Dialogue. find our card
  6. Rich Caruana. 1993. Multitask learning: a knowledge-based source of inductive bias. In ICML. find
  7. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. The Journal of Machine Learning Research, 12:2493–2537. find our card
  8. Laurence Danlos and Owen Rambow. 2011. find
  9. Discourse Relations and Propositional Attitudes. In CID 2011 -Fourth International workshop on Constraints in Discourse. find
  10. Hal Daumé III and Daniel Marcu. 2009. A noisy-channel model for document compression. In Proceedings of ACL. find
  11. John W Du Bois. 2000. Santa Barbara Corpus of Spoken American English. University of California, Santa Barbara Center for the Study of Discourse. find
  12. Vanessa Wei Feng and Graeme Hirst. 2012. Text-level discourse parsing with rich linguistic features. In Proceedings of ACL. find our card
  13. Vanessa Wei Feng and Graeme Hirst. 2014. A linear-time bottom-up discourse parser with constraints and post-editing. In Proceedings of ACL. find our card
  14. Hugo Hernault, Helmut Prendinger, David A. duVerle, and Mitsuru Ishizuka. 2010. Hilda: A discourse parser using support vector machine classification. Dialogue and Discourse, 1:1–33. find our card
  15. Derrick Higgins, Jill Burstein, Daniel Marcu, and Claudia Gentile. 2004. Evaluating multiple aspects of coherence in student essays. In Proceedings of HLT-NAACL. find
  16. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735– 1780. find
  17. Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. Ontonotes: The 90% solution. In Proceedings of HLT-NAACL. find
  18. Zhiheng Huang, Wei Xu, and Kai Yu. arXiv:1508.01991. 2015. find our card
  19. Bidirectional lstm-crf models for sequence tagging. find our card
  20. Yangfeng Ji and Jacob Eisenstein. 2014a. One vector is not enough: Entity-augmented distributional semantics for discourse relations. TACL. find
  21. Yangfeng Ji and Jacob Eisenstein. 2014b. Representation learning for text-level discourse parsing. In Proceedings of ACL. find our card
  22. Xin Jin, Fuzhen Zhuang, Shuhui Wang, Qing He, and Zhongzhi Shi. 2013. Shared structure learning for multiple tasks with multiple views. In Machine Learning and Knowledge Discovery in Databases, pages 353–368. Springer. find
  23. Shafiq R. Joty, Giuseppe Carenini, and Raymond T. Ng. 2012. A novel discriminative framework for sentence-level discourse analysis. In Proceedings of EMNLP. find
  24. Sigrid Klerke, Yoav Goldberg, and Anders Søgaard. 2016. Improving sentence compression by learning to predict gaze. In Proceedings of NAACL. find
  25. Man Lan, Yu Xu, and Zhengyu Niu. 2013. Leveraging synthetic discourse data via multi-task learning for implicit discourse relation recognition. In Proceedings of ACL. find our card
  26. Huong Le Thanh, Geetha Abeysinghe, and Christian Huyck. 2004. Generating discourse structures for written text. In Proceedings of COLING. find
  27. Sujian Li, Liang Wang, Ziqiang Cao, and Wenjie Li. 2014. Text-level discourse dependency parsing. In Proceedings of ACL. find
  28. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing implicit discourse relations in the penn discourse treebank. In Proceedings of EMNLP. find our card
  29. Annie Louis, Aravind Joshi, and Ani Nenkova. 2010. Discourse indicators for content selection in summarization. In Proceedings of SIGDIAL. find
  30. William C. Mann and Sandra A. Thompson. 1988. Rhetorical structure theory : Toward a functional theory of text organization. Text, 8:243–281. find our card
  31. Daniel Marcu. 1997. The Rhetorical Parsing, Summarization, and Generation of Natural Language Texts. Ph.D. thesis, University of Toronto. find
  32. Daniel Marcu. 2000a. The rhetorical parsing of unrestricted texts: A surface-based approach. Computational Linguistics. find our card
  33. Daniel Marcu. 2000b. The Theory and Practice of Discourse Parsing and Summarization. MIT Press, Cambridge, MA, USA. find
  34. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of english: The penn treebank. Computational Linguistics, 19(2):313–330. find our card
  35. Philippe Muller, Stergos Afantenos, Pascal Denis, and Nicholas Asher. 2012. Constrained decoding for text-level discourse parsing. In Proceedings of COLING. find our card
  36. Emily Pitler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of ACL-IJCNLP. find our card
  37. Barbara Plank, Anders Søgaard, and Yoav Goldberg. 2016. Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss. In Proceedings of ACL. find our card
  38. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The penn discourse treebank 2.0. In Proceedings of LREC. find our card
  39. James Pustejovsky, Patrick Hanks, Roser Saurı́, Andrew See, Rob Gaizauskas, Andrea Setzer, Dragomir R. Radev, Beth Sundheim, David S. Day, Lisa Ferro, and Marcia Lazo. 2003. The TIMEBANK corpus. In Proceedings of Corpus Linguistics. find
  40. James Pustejovsky, Robert Knippen, Jessica Littman, and Roser Saurı́. 2005. Temporal and event information in natural language text. Language Resources and Evaluation, 39(2):123–164. find
  41. Attapol Rutherford and Nianwen Xue. 2014. Discovering implicit discourse relations through brown cluster pair representation and coreference patterns. In Proceedings of EACL. find our card
  42. Roser Saurı́ and James Pustejovsky. 2009. Factbank: a corpus annotated with event factuality. In Proceedings of LREC, volume 43, pages 227–268. find
  43. Caroline Sporleder and Mirella Lapata. 2005. Discourse chunking and its application to sentence compression. In Proceedings of HLT/EMNLP. find
  44. Manfred Stede. 2011. Discourse Processing. Morgan & Claypool. find
  45. Maite Taboada and Debopam Das. 2013. Annotation upon annotation: Adding signalling information to a corpus of discourse relations. Dialogue and Discourse, 4(2):249–281. find our card
  46. Maite Taboada and William C. Mann. 2006. Applications of rhetorical structure theory. Discourse Studies, 8:567–588. find
  47. Gian Lorenzo Thione, Martin Van den Berg, Livia Polanyi, and Chris Culy. 2004. Hybrid text summarization: Combining external relevance measures with structural analysis. In Proceedings of the ACL Workshop Text Summarization Branches Out. find
  48. Ngo Xuan Bach, Nguyen Le Minh, and Akira Shimazu. 2012. A reranking model for discourse segmentation using subtree features. In Proceedings of Sigdial. find
  49. Shu Zhang, Dequan Zheng, Xinchen Hu, and Ming Yang. 2015. Bidirectional long short-term memory networks for relation classification. In Proceedings of PACLIC. find
  50. Jie Zhou and Wei Xu. 2015. End-to-end learning of semantic role labeling using recurrent neural networks. In Proceedings of ACL. find

↑ back to top