Elamur Research⌕
☰
home / papers / 2017 / Exploring Joint Neural Model for Sentence Level Discourse Parsing and Sentiment Analysis

Exploring Joint Neural Model for Sentence Level Discourse Parsing and Sentiment Analysis

Summary

The paper investigates whether discourse parsing and sentiment analysis can strengthen each other when learned together in one neural system at the sentence level. The authors reason that sentiment cues, such as opposing polarity in two spans, should point toward contrast-type relations, and that discourse structure should in turn help compute overall sentence polarity. They encode text spans with Skip-thought vectors, which task-specific neural compressors shrink to a low dimension. Three Recursive Neural Nets are built on top of these vectors: one predicts discourse structure, one predicts discourse relations, and one assigns sentiment over the Discourse Tree. They then combine the networks in two ways, Pre-training and Multi-tasking. Experiments on RST-DT and on the Sentiment Treebank, re-annotated over discourse units, show that both joint setups beat the individual models. Multi-tasking gives the larger gains, especially for contrastive relations. The work does not set a new state of the art; CODRA remains stronger at sentence-level parsing. Its value is in showing empirically that the two tasks share useful latent information.

Key points

  • Data: From RST-DT, the authors extracted 6846 sentence-level Discourse Trees. They dropped 2239 trees that contained only one EDU, leaving 4607 sentences. The Sentiment Treebank (11855 sentences) was parsed with CODRA (Joty et al., 2015), and each discourse node received the sentiment label of the matching syntactic subtree.
  • Representations: Compressed Skip-thought vectors with dimension d = 100 were used. Results stayed roughly stable for d between 100 and 1200 but dropped noticeably at 50 and 25.
  • Sentiment result: Multi-tasking reached 44.82% fine-grained root-level sentiment accuracy, close to the 45.7% that Socher et al. (2013) reported using costlier syntactic-level annotation.
  • Discourse result: Multi-tasking improved relation prediction mainly for the Contrastive group, which includes Contrast, Comparison, Antithesis, Concession and similar relations.
  • Training cost: Multi-tasking converged in about 6 epochs, compared with about 200 for the individual models. However, the individual and Pre-training models can be trained in parallel, so they run roughly an order of magnitude faster in practice.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

12 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 7 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w17-5535

@inproceedings{nejat2017exploring,
  title = {{Exploring Joint Neural Model for Sentence Level Discourse Parsing and Sentiment Analysis}},
  author = {Nejat, Bita and Carenini, Giuseppe and Ng, Raymond},
  booktitle = {Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue},
  year = {2017},
  doi = {10.18653/v1/w17-5535},
  url = {https://doi.org/10.18653/v1/w17-5535}
}

References

25 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Martı́n Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenbe… find
  2. Shafiq Joty, Giuseppe Carenini, and Raymond T Ng. 2015. Codra: A novel discriminative framework for rhetorical analysis. Computational Linguistics . find our card
  3. Takuhiro Kaneko, Kaoru Hiramatsu, and Kunio Kashino. 2016. Adaptive visual feedback generation for facial expression improvement with multi-task deep neural networks. In Proceedings of the 2016 ACM on Multimedia Conference. ACM, pages 327–331. find
  4. Ryan Kiros, Yukun Zhu, Ruslan R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Skip-thought vectors. In Advances in neural information processing systems. pages 3294–3302. find
  5. Parminder Bhatia, Yangfeng Ji, and Jacob Eisenstein. 2015. Better document-level sentiment analysis from rst discourse parsing. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP). http://www.aclweb.org/anthology/D/D15/D15-1263.pdf. ↗ our card
  6. Angeliki Lazaridou, Ivan Titov, and Caroline Sporleder. 2013. A bayesian model for joint unsupervised induction of sentiment, aspect and discourse representations. In ACL (1). pages 1630–1639. find
  7. Jiwei Li, Rumeng Li, and Eduard H Hovy. 2014. Recursive deep models for discourse parsing. In EMNLP. pages 2061–2069. find our card
  8. Lynn Carlson and Daniel Marcu. 2001. Discourse tagging reference manual. ISI Technical Report ISI-TR-545 54:56. find
  9. Lynn Carlson, Mary Ellen Okurowski, and Daniel Marcu. 2002. RST discourse treebank. Linguistic Data Consortium, University of Pennsylvania. find
  10. Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2016a. Recurrent neural network for text classification with multi-task learning. arXiv preprint arXiv:1605.05101 . — [arXiv:1605.05101](https://arxiv.org/abs/1605.05101) find
  11. Yu-An Chung, Hsuan-Tien Lin, and Shao-Wen Yang. 2015. Cost-aware pre-training for multi-class cost-sensitive deep learning. arXiv preprint arXiv:1511.09337 . — [arXiv:1511.09337](https://arxiv.org/abs/1511.09337) find
  12. Yang Liu, Sujian Li, Xiaodong Zhang, and Zhifang Sui. 2016b. Implicit discourse relation classification via multi-task neural networks. arXiv preprint arXiv:1603.02776 . find
  13. Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, Christopher Potts, et al. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. Proceedings of the conference on empirical methods in natural language processing (EMNLP) 1631:1… find
  14. Annie Louis, Aravind Joshi, and Ani Nenkova. 2010. Discourse indicators for content selection in summarization. In Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue. Association for Computational Linguistics, pages 147–156. find
  15. William C Mann and Sandra A Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text-Interdisciplinary Journal for the Study of Discourse 8(3):243–281. find our card
  16. Suzan Verberne, Lou Boves, Nelleke Oostdijk, and Peter-Arno Coppen. 2007. Evaluating discourse-based answer extraction for why-question answering. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, pages 735–736. find
  17. Daniel Marcu and Kevin Knight. 2001. Discourse parsing and summarization. US Patent App. 09/854,301. find
  18. Mitchell P Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of english: The penn treebank. Computational linguistics 19(2):313–330. find our card
  19. Thomas Meyer and Andrei Popescu-Belis. 2012. Using sense-labeled discourse connectives for statistical machine translation. In Proceedings of the Joint Workshop on Exploiting Synergies between Information Retrieval and Machine Translation (ESIRMT) and Hybrid Approaches to Machine Translation (HyTra)… find our card
  20. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. pages 3111–3119. find
  21. Tetsuji Nakagawa, Kentaro Inui, and Sadao Kurohashi. 2010. Dependency tree-based sentiment classification using crfs with hidden variables. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Association for Compu… find
  22. Bo Pang, Lillian Lee, et al. 2008. Opinion mining and sentiment analysis. Foundations and Trends R in Information Retrieval 2(1–2):1–135. find
  23. Vassiliki Rentoumi, Stefanos Petrakis, Manfred Klen-ner, George A Vouros, and Vangelis Karkaletsis. 2010. United we stand: Improving sentiment analysis by joining machine learning and rule based methods. In LREC. find
  24. Sara Rosenthal, Alan Ritter, Preslav Nakov, and Veselin Stoyanov. 2014. Semeval-2014 task 9: Sentiment analysis in twitter. In Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014). Dublin, Ireland, pages 73–80. find
  25. Seyyede Zohreh Seyyedsalehi and Seyyed Ali Seyyedsalehi. 2015. A fast and efficient pre-training method based on layer-by-layer maximum discrimination for deep neural networks. Neurocomputing 168:669–680. find

↑ back to top

This page in Markdown →