Elamur Research⌕
☰
home / papers / 2020 / MEGA RST Discourse Treebanks with Structure and Nuclearity from Scalable Distant Sentiment Supervision

MEGA RST Discourse Treebanks with Structure and Nuclearity from Scalable Distant Sentiment Supervision

Summary

RST-style discourse parsing lacks training data. Each human-annotated treebank holds only a few hundred documents, which limits deep-learning models. The authors' earlier distant-supervision method derived tree structure from document-level sentiment. However, it relied on exhaustive CKY search, which fails on long documents, and it produced no nuclearity labels.

This paper replaces CKY with a heuristic beam search. The search keeps only candidate subtrees whose aggregated sentiment stays close to the document's gold label. It adds a stochastic exploration–exploitation element borrowed from reinforcement learning and predicts nuclearity together with structure. EDU-level sentiment and attention scores come from a neural MIL model.

Applied to Yelp '13 reviews, the method yields MEGA-DT, a public silver-standard treebank of about 250,000 documents. A parser trained on MEGA-DT does not match in-domain training. In cross-domain tests, however, it usually beats the same parser trained on human-annotated or earlier silver-standard treebanks, often significantly. As a result, any sentiment-labelled corpus can be turned into a domain-specific discourse treebank.

Key points

  • Resource: MEGA-DT contains roughly 250,000 Yelp '13 documents with RST-style trees that include structure and nuclearity but not relation labels. The treebank and the generation code are publicly released.
  • Method:
    • A neural MIL model (Angelidis and Lapata, 2018) assigns each EDU a sentiment polarity and an attention weight.
    • These values are combined bottom-up with an attention-weighted aggregation function.
    • A stochastic beam search keeps tree building tractable for documents of any length.
  • Evaluation setup: The parser of Wang et al. (2017) was trained on MEGA-DT and compared against training on:
    • Yelp13-DT (about 100,000 documents with at most 20 EDUs);
    • RST-DT (news);
    • Instr-DT (instructional texts).
  • Results:
    • For inter-domain parsing, MEGA-DT gave statistically significant structure gains over the best baseline.
    • It also gave significant nuclearity gains on Instr-DT under RST-Parseval, and matched the best baseline on RST-DT nuclearity.
    • The ablation (MEGA-DT-Base, +Stoch, +Nuc) showed the full combination works best.
    • Performance kept improving as larger subsets of the treebank were used.
  • Weakness: Confusion matrices show the parser assigns the N-N class too often and frequently mistakes gold N-S for S-N. The authors list better nuclearity assignment as future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis · Treebanks

Cited by

14 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 10 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.emnlp-main.603 · arXiv 2011.03017

@inproceedings{huber2020mega,
  title = {{MEGA RST Discourse Treebanks with Structure and Nuclearity from Scalable Distant Sentiment Supervision}},
  author = {Huber, Patrick and Carenini, Giuseppe},
  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.603},
  eprint = {2011.03017},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2020.emnlp-main.603}
}

References

40 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Stefanos Angelidis and Mirella Lapata. 2018. Multiple instance learning networks for fine-grained sentiment analysis. Transactions of the Association for Computational Linguistics, 6:17–31. ↗
  2. Parminder Bhatia, Yangfeng Ji, and Jacob Eisenstein. 2015. Better document-level sentiment analysis from rst discourse parsing. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2212–2218. ↗ our card
  3. Lynn Carlson, Mary Ellen Okurowski, and Daniel Marcu. 2002. RST discourse treebank. Linguistic Data Consortium, University of Pennsylvania. find
  4. Tuhin Chakrabarty, Christopher Hidey, Smaranda Muresan, Kathleen McKeown, and Alyssa Hwang. 2019. Ampersand: Argument mining for persuasive online discussions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2926–2936. ↗
  5. Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. 2018. A discourse-aware attention model for abstractive summarization of long documents. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 615–621, New Orleans, Louisiana. Association for Computational Linguistics. find our card
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. ↗
  7. Qiming Diao, Minghui Qiu, Chao-Yuan Wu, Alexander J Smola, Jing Jiang, and Chong Wang. 2014. Jointly modeling aspects, ratings and sentiments for movie recommendation (jmars). In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 193– 202. ACM. find
  8. Chris Dyer, Adhiguna Kuncoro, Miguel Ballesteros, and Noah A Smith. 2016. Recurrent neural network grammars. arXiv preprint arXiv:1602.07776. ↗
  9. Vanessa Wei Feng and Graeme Hirst. 2014. A linear-time bottom-up discourse parser with constraints and post-editing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 511–521. find our card
  10. Elisa Ferracane, Greg Durrett, Junyi Jessy Li, and Katrin Erk. 2019. Evaluating discourse in structured text representations. arXiv preprint arXiv:1906.01472. ↗
  11. Daniel Fried, Mitchell Stern, and Dan Klein. 2017. Improving neural parsing by disentangling model combination and reranking effects. arXiv preprint arXiv:1707.03058. ↗
  12. Shima Gerani, Yashar Mehdad, Giuseppe Carenini, Raymond T Ng, and Bita Nejat. 2014. Abstractive summarization of product reviews using discourse structure. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1602–1613. find
  13. Alexander Hogenboom, Flavius Frasincar, Franciska De Jong, and Uzay Kaymak. 2015. Using rhetorical structure in sentiment analysis. Commun. ACM, 58(7):69–77. ↗
  14. Patrick Huber and Giuseppe Carenini. 2019. Predicting discourse structure using distant supervision from sentiment. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2306–2316. ↗
  15. Yangfeng Ji and Jacob Eisenstein. 2014. Representation learning for text-level discourse parsing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 13–24. find our card
  16. Yangfeng Ji and Noah A Smith. 2017. Neural discourse structure for text categorization. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 996–1005. ↗
  17. Shafiq Joty, Giuseppe Carenini, and Raymond T Ng. 2015. Codra: A novel discriminative framework for rhetorical analysis. Computational Linguistics, 41(3). ↗ our card
  18. Dan Jurafsky and James H Martin. 2014. Speech and language processing, volume 3. Pearson London. find
  19. Hamid Karimi and Jiliang Tang. 2019. Learning hierarchical discourse-level structure for fake news detection. arXiv preprint arXiv:1903.07389. ↗
  20. James D Keeler, David E Rumelhart, and Wee Kheng Leow. 1991. Integrated segmentation and recognition of hand-printed numerals. In Advances in neural information processing systems, pages 557–563. find
  21. Qi Li, Tianshi Li, and Baobao Chang. 2016. Discourse parsing with attention-based hierarchical neural networks. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 362–371. find our card
  22. Yang Liu, Ivan Titov, and Mirella Lapata. 2019. Single document summarization as tree induction. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1745–1755. find
  23. Amandla Mabona, Laura Rimell, Stephen Clark, and Andreas Vlachos. 2019. Neural generative rhetorical structure parsing. arXiv preprint arXiv:1909.11049. ↗ our card
  24. William C Mann and Sandra A Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text-Interdisciplinary Journal for the Study of Discourse, 8(3):243–281. ↗ our card
  25. Daniel Marcu. 2000. The theory and practice of discourse parsing and summarization. MIT press. ↗
  26. Mathieu Morey, Philippe Muller, and Nicholas Asher. 2017. How much progress have we made on rst discourse parsing? a replication study of recent results on the rst-dt. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1319–1324. find our card
  27. Mathieu Morey, Philippe Muller, and Nicholas Asher. 2018. A dependency perspective on rst discourse parsing and evaluation. Computational Linguistics, 44(2):197–235. ↗ our card
  28. Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Çağlar Gu̇lçehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence RNNs and beyond. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 280–290, Berlin, Germany. Association for Computational Linguistics. American Chapter of the Association for Computational Linguistics, pages 566–574. Association for Computational Linguistics. find
  29. Duyu Tang, Bing Qin, and Ting Liu. 2015. Document modeling with gated recurrent neural network for sentiment classification. In Proceedings of the 2015 conference on empirical methods in natural language processing, pages 1422–1432. find
  30. Oriol Vinyals, Łukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2015. Grammar as a foreign language. In Advances in neural information processing systems, pages 2773–2781. ↗
  31. Yizhong Wang, Sujian Li, and Houfeng Wang. 2017. A two-stage parsing method for text-level discourse analysis. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 184–188. ↗ our card
  32. Nan Yu, Meishan Zhang, and Guohong Fu. 2018. Transition-based neural rst parsing with implicit syntax features. In Proceedings of the 27th International Conference on Computational Linguistics, pages 559–570. find
  33. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Advances in neural information processing systems, pages 649–657. ↗ our card
  34. Bita Nejat, Giuseppe Carenini, and Raymond Ng. 2017. Exploring joint neural model for sentence level discourse parsing and sentiment analysis. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 289–298. find our card
  35. David L Poole and Alan K Mackworth. 2010. Artificial Intelligence: foundations of computational agents. Cambridge University Press. find
  36. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The penn discourse treebank 2.0. LREC. find our card
  37. Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don’t know: Unanswerable questions for squad. arXiv preprint arXiv:1806.03822. ↗ our card
  38. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250. ↗ our card
  39. Vighnesh Shiv and Chris Quirk. 2019. Novel positional encodings to enable tree-based transformers. In Advances in Neural Information Processing Systems, pages 12058–12068. find
  40. Rajen Subba and Barbara Di Eugenio. 2009. An effective discourse parser that uses rich linguistic information. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North find our card

↑ back to top