Elamur Research⌕
☰
home / papers / 2015 / Better Document-level Sentiment Analysis from RST Discourse Parsing

Better Document-level Sentiment Analysis from RST Discourse Parsing

Summary

Document-level sentiment classifiers usually treat a review as a bag of words. This fails when many positive remarks are overridden by one decisive negative conclusion. The authors revisit Rhetorical Structure Theory (RST), since earlier work found automatic RST parses unhelpful but parsers have improved considerably since then. They obtain discourse trees from the off-the-shelf DPLP parser and use them in two ways.

The first method converts each RST tree into a dependency-style discourse tree (DEP-DT). It then down-weights elementary discourse units (EDUs) that are nested deeply inside satellites.

The second method, the Rhetorical Recursive Neural Network (R2N2), passes scalar sentiment scores up the RST tree toward the root. It learns separate composition weights for nuclei and satellites and for contrastive versus non-contrastive relations.

Both methods were tested on two movie-review polarity datasets. Depth reweighting gives large gains for lexicon-based analysis but adds little to a trained classifier. R2N2 achieves the best accuracy on both datasets. The work shows that automatic discourse parsing is now accurate enough to help document-level opinion mining, even with simple models.

Key points

  • Data: the Pang & Lee (2004) set of 2,000 reviews, evaluated with ten-fold cross-validation, and a 50,000-review set (attributed to Socher et al., 2013) with a fixed 50/50 train/test split. Ratings ≤4 count as negative, ≥7 as positive, and 5–6 are discarded.
  • Depth reweighting: each EDU's weight is λ = max(0.5, 1 − d/6), where d is its DEP-DT depth.
    • With the Wilson et al. (2005) lexicon, accuracy rises from 68.3 to 72.6 on Pang & Lee and from 74.9 to 78.9 on the larger set.
    • With a logistic regression classifier, the gain is under 1 point.
  • R2N2 results: 84.1 on Pang & Lee and 85.6 on the larger set, using relation-specific weights. The bag-of-words classifier baseline scores 82.4 and 81.5. Without relation distinctions, R2N2 reaches 83.4 and 85.5.
  • R2N2 design: it is trained with a hinge loss, backpropagation through structure and SGD. Relations are grouped only into contrastive (e.g., CONCESSION, CONTRAST, ANTITHESIS) versus non-contrastive. The authors suggest that limited relation-detection accuracy (~60%) explains the small benefit from relations.
  • Code: released at github.com/parry2403/R2N2. Evaluation on gold RST trees was not possible, because the only candidate corpus, the SFU Review Corpus, contains just 30 texts.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

68 papers of our library cite it (of the 140673 whose bibliographies we read); 22 of them have a page here, 46 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d15-1263 · arXiv 1509.01599

@inproceedings{bhatia2015better,
  title = {{Better Document-level Sentiment Analysis from RST Discourse Parsing}},
  author = {Bhatia, Parminder and Ji, Yangfeng and Eisenstein, Jacob},
  booktitle = {Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing},
  year = {2015},
  doi = {10.18653/v1/d15-1263},
  eprint = {1509.01599},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d15-1263}
}

References

45 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. José M Chenlo, Alexander Hogenboom, and David E Losada. 2014. Rhetorical structure theory for polarity estimation: An experimental study. Data & Knowledge Engineering. find
  2. Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A computational approach to politeness with application to social factors. In Proceedings of the Association for Computational Linguistics (ACL), pages 250–259, Sophia, Bulgaria. find our card
  3. Ronen Feldman. 2013. Techniques and applications for sentiment analysis. Communications of the ACM, 56(4):82–89. find
  4. Vanessa Wei Feng and Graeme Hirst. 2012. Text-level Discourse Parsing with Rich Linguistic Features. In Proceedings of the Association for Computational Linguistics (ACL), Jeju, Korea. find our card
  5. Shima Gerani, Yashar Mehdad, Giuseppe Carenini, Raymond T Ng, and Bita Nejat. 2014. Abstractive summarization of product reviews using discourse structure. In Proceedings of the Association for Computational Linguistics (ACL), Baltimore, MD. find
  6. Christoph Goller and Andreas Kuchler. 1996. Learning task-dependent distributed representations by backpropagation through structure. In Neural Networks, IEEE International Conference on, pages 347–352. IEEE. find
  7. Vasileios Hatzivassiloglou and Kathleen R. McKeown. 1997. Predicting the semantic orientation of adjectives. In Proceedings of the Association for Computational Linguistics (ACL), pages 174–181, Madrid, Spain. find
  8. Bas Heerschop, Frank Goossen, Alexander Hogenboom, Flavius Frasincar, Uzay Kaymak, and Franciska de Jong. 2011. Polarity analysis of texts using discourse structure. In Proceedings of the 20th ACM international conference on Information and knowledge management, pages 1061–1070. ACM. find
  9. Hugo Hernault, Helmut Prendinger, David A. duVerle, and Mitsuru Ishizuka. 2010. HILDA: A Discourse Parser Using Support Vector Machine Classification. Dialogue and Discourse, 1(3):1–33. find our card
  10. Tsutomu Hirao, Yasuhisa Yoshida, Masaaki Nishino, Norihito Yasuda, and Masaaki Nagata. Single-document summarization as a tree knapsack problem. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP), pages 1515–1520. find
  11. Alexander Hogenboom, Flavius Frasincar, Franciska de Jong, and Uzay Kaymak. 2015. Using rhetorical structure in sentiment analysis. Communications of the ACM, 58(7):69–77. find
  12. Yangfeng Ji and Jacob Eisenstein. 2014. Representation learning for text-level discourse parsing. In Proceedings of the Association for Computational Linguistics (ACL), Baltimore, MD. find our card
  13. Shafiq Joty, Giuseppe Carenini, and Raymond Ng. 2015. CODRA: A novel discriminative framework for rhetorical analysis. Computational Linguistics, 41(3). find our card
  14. Dan Jurafsky, Victor Chahuneau, Bryan R Routledge, and Noah A Smith. 2014. Narrative framing of consumer sentiment in online restaurant reviews. First Monday, 19(4). find
  15. Suin Kim, JinYeong Bak, and Alice Haeyun Oh. 2012. Do you feel what i feel? social aspects of emotions in twitter conversations. In Proceedings of the International Conference on Web and Social Media (ICWSM), Menlo Park, California. AAAI Publications. find
  16. Sandra Kübler, Ryan McDonald, and Joakim Nivre. 2009. Dependency parsing. Synthesis Lectures on Human Language Technologies, 1(1):1–127. find
  17. Angeliki Lazaridou, Ivan Titov, and Caroline Sporleder. 2013. A bayesian model for joint unsupervised induction of sentiment, aspect and discourse representations. In Proceedings of the Association for Computational Linguistics (ACL), pages 1630–1639, Sophia, Bulgaria. find
  18. Jiwei Li, Rumeng Li, and Eduard Hovy. 2014. Recursive deep models for discourse parsing. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP). find our card
  19. Bing Liu. 2012. Sentiment analysis and opinion mining. Synthesis Lectures on Human Language Technologies, 5(1):1–167. find
  20. William Mann. 1984. Discourse structures for text generation. In Proceedings of the 10th International Conference on Computational Linguistics and 22nd annual meeting on Association for Computational Linguistics, pages 367–375. Association for Computational Linguistics. find
  21. Daniel Marcu. 1997. The rhetorical parsing of natural language texts. In Proceedings of the European Chapter of the Association for Computational Linguistics (EACL), pages 96–103. find
  22. Daniel Marcu. 1999. The automatic construction of 2013. large-scale corpora for summarization research. In Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, pages 137–144. ACM. find
  23. Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the Association for Computational Linguistics (ACL), pages 271–278. find
  24. Bo Pang and Lillian Lee. 2008. Opinion mining and sentiment analysis. Foundations and trends in information retrieval, 2(1-2):1–135. find
  25. Kenji Sagae. 2009. Analysis of Discourse Structure with Syntactic Dependencies and Data-Driven Shift-Reduce Parsing. In Proceedings of the 11th International Conference on Parsing Technologies (IWPT’09), pages 81–84, Paris, France, October. Association for Computational Linguistics. find
  26. Paul Smolensky. 1990. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial intelligence, 46(1):159–216. find
  27. Richard Socher, Cliff C. Lin, Andrew Y. Ng, and Christopher D. Manning. 2011. Parsing Natural Scenes and Natural Language with Recursive Neural Networks. In Proceedings of the International Conference on Machine Learning (ICML), Seattle, WA. find
  28. Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP). find
  29. Swapna Somasundaran, Galileo Namata, Janyce Wiebe, and Lise Getoor. 2009. Supervised and unsupervised methods in employing discourse relations for improving opinion polarity classification. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP), Singapore. find our card
  30. Radu Soricut and Daniel Marcu. 2003. Sentence level discourse parsing using syntactic and lexical information. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 149–156. find our card
  31. Philip J. Stone. 1966. The General Inquirer: A Computer Approach to Content Analysis. The MIT Press. find
  32. Maite Taboada, Julian Brooke, Milan Tofiloski, Kimberly Voll, and Manfred Stede. 2011. Lexicon-based methods for sentiment analysis. Computational linguistics, 37(2):267–307. find our card
  33. Maite Taboada. 2008. SFU review corpus. http://www.sfu.ca/˜mtaboada/research/SFU_Review_Corpus.html. ↗
  34. Yla R Tausczik and James W Pennebaker. 2010. The psychological meaning of words: LIWC and computerized text analysis methods. Journal of Language and Social Psychology, 29(1):24–54. find
  35. Rakshit Trivedi and Jacob Eisenstein. 2013. Discourse connectors for latent subjectivity in sentiment analysis. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 808–813, Stroudsburg, Pennsylvania. Association for Computational Linguistics. find
  36. Vinı́cius Rodrigues Uzêda, Thiago Alexandre Salgueiro Pardo, and Maria Das Graças Volpe Nunes. 2010. A comprehensive comparative evaluation of rst-based summarization methods. ACM Transactions on Speech and Language Processing (TSLP), 6(4):4. find
  37. Kimberly Voll and Maite Taboada. 2007. Not all words are created equal: Extracting semantic orientation as a function of adjective relevance. In Proceedings of Australian Conference on Artificial Intelligence. find
  38. Henning Wachsmuth, Martin Trenkmann, Benno Stein, and Gregor Engels. 2014. Modeling review argumentation for robust sentiment analysis. In Proceedings of the International Conference on Computational Linguistics (COLING). find
  39. Fei Wang and Yunfang Wu. 2013. Exploiting hierarchical discourse structure for review sentiment analysis. In Proceedings of the Conference on Asian Language Processing (IALP), pages 121–124. find
  40. Fei Wang, Yunfang Wu, and Likun Qiu. 2012. Exploiting discourse relations for sentiment analysis. In Proceedings of the International Conference on Computational Linguistics (COLING), pages 1311– 1320, Mumbai, India. find our card
  41. Theresa Wilson, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP), pages 347–354. find
  42. Bishan Yang and Claire Cardie. 2014. Context-aware learning for sentence-level sentiment analysis with posterior regularization. In Proceedings of the Association for Computational Linguistics (ACL), Baltimore, MD. find
  43. Yasuhisa Yoshida, Jun Suzuki, Tsutomu Hirao, and Masaaki Nagata. 2014. Dependency-based Discourse Parser for Single-Document Summarization. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP). find our card
  44. Lanjun Zhou, Binyang Li, Wei Gao, Zhongyu Wei, and Kam-Fai Wong. 2011. Unsupervised discovery of discourse relations for eliminating intra-sentence polarity ambiguities. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP), pages 162–171. find our card
  45. Cäcilia Zirn, Mathias Niepert, Heiner Stuckenschmidt, and Michael Strube. 2011. Fine-grained sentiment analysis with structural features. In Proceedings of the International Joint Conference on Natural Language Processing (IJCNLP), pages 336–344, Chiang Mai, Thailand. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bhatia-2015-better-document-level-sentiment-analysis/