Elamur Research⌕
☰
home / papers / 2020 / It's absolutely divine! Can fine-grained sentiment analysis benefit from coreference resolution?

It's absolutely divine! Can fine-grained sentiment analysis benefit from coreference resolution?

Summary

Surveys of opinion mining often claim that resolving anaphora should help, but this has seldom been tested. The authors test it on aspect category classification, a subtask of aspect-based sentiment analysis (ABSA). They use SemEval restaurant reviews in English (350 reviews, SemEval 2015) and Dutch (400 reviews, SemEval 2016). Many sentiment-bearing aspects have no explicit target and are marked 'NULL'. The authors manually labelled each of these as one of three types:

  • COREF: an anaphor whose antecedent appears in the same review.
  • EMPTY: an anaphor without such an antecedent.
  • IMPLICIT: a truly implicit mention.

The classifier combines bag-of-words with WordNet and DBpedia features. A wrapper-based genetic algorithm tuned features and hyperparameters jointly. Each language was run with and without coreference resolution beforehand, using both gold-standard and automatic links. During optimisation, gold links gave small gains, larger for English, while automatic links lowered accuracy. On held-out test sets, coreference brought no benefit in either language. The study therefore challenges the assumption that coreference resolution is a ready improvement for fine-grained sentiment analysis.

Key points

  • Implicit ('NULL') targets make up 623 of 2,499 aspect categories in English (24.9%) and 773 of 2,445 in Dutch (31.6%). Most of them are truly implicit.
  • Referential pronouns with an in-review antecedent account for 20.7% of implicit targets in English but only 10.34% in Dutch. Dutch more often uses EMPTY anaphors.
  • With gold coreference, optimised accuracy rose from 67.23% to 68.20% for English and from 63.16% to 63.59% for Dutch. Automatically derived coreference reduced performance.
  • On held-out data, English fell to 57.75% without coreference and 56.92% with gold coreference. Dutch reached 66.42% in both settings, almost ten points above the best SemEval 2016 result on that dataset.
  • The English test set contained only 8 referential anaphors among 248 implicit targets, and just 4 had an in-review antecedent. The Dutch test set had 56 of 210, with 30 anaphor-antecedent pairs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{clercq2020it,
  title = {{It's absolutely divine! Can fine-grained sentiment analysis benefit from coreference resolution?}},
  author = {Clercq, Orphee De and Hoste, Veronique},
  booktitle = {Proceedings of the Third Workshop on Computational Models of Reference, Anaphora and Coreference},
  year = {2020},
  doi = {10.48448/hcdr-c690},
  url = {https://doi.org/10.48448/hcdr-c690}
}

References

35 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chih-Chung Chang and Chih-Jen Lin. 2011. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3):1–27. find
  2. Orphée De Clercq and Veronique Hoste. 2016. All mixed up? finding the optimal feature set for general readability prediction and its application to english and dutch. COMPUTATIONAL LINGUISTICS, 42(3):457–490. find
  3. O. De Clercq, I. Hendrickx, and V. Hoste. 2011. Cross-domain Dutch coreference resolution. In Proceedings of the 8th International Conference on Recent Advances in Natural Language Processing (RANLP-2011), pages 186–193. find
  4. B. Desmet and V. Hoste. 2013. Fine-grained Dutch named entity recognition. Language Resources and Evaluation, pages 307–343. ↗
  5. X. Ding and B. Liu. 2010. Resolving Object and Attribute Coreference in Opinion Mining. In Proceedings of the 23rd International Conference on Computational Linguistics (COLING2010), pages 268–276. find our card
  6. Hai Ha Do, PWC Prasad, Angelika Maag, and Abeer Alsadoon. 2019. Deep learning for aspect-based sentiment analysis: A comparative review. Expert Systems with Applications, 118:272 – 299. ↗
  7. Ronen Feldman. 2013. Techniques and applications for sentiment analysis. Communications of the ACM, 56(4):82–89. ↗
  8. C. Fellbaum. 1998. WordNet: an Electronic Lexical Database. MIT Press. find
  9. Vasileios Hatzivassiloglou and Kathleen R. McKeown. 1997. Predicting the semantic orientation of adjectives. In Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and the 8th Conference of the European Chapter of the Association for Computational Linguistics (ACL - EACL-1997), pages 174–181. find
  10. Veronique Hoste. 2016. The mention-pair model. In Massimo Poesio, Roland Stuckardt, and Yannick Versley, editors, Anaphora resolution : algorithms, resources and applications, pages 281–295. Springer-Verlag. ↗
  11. Eduard Hovy, Roberto Navigli, and Simone Paolo Ponzetto. 2013. Collaboratively built semi-structured content and Artificial Intelligence: The story so far. Artificial Intelligence, 194:2–27. ↗
  12. Jason S. Kessler and Nicolas Nicolov. 2009. Targeting sentiment expressions through supervised ranking of linguistic configurations. In The 3rd Association for the Advancement of Artificial Intelligence Conference on Weblogs and Social Media (ICWSM-2009), pages 90–97. find
  13. Soo-Min Kim and Eduard Hovy. 2006. Extracting opinions, opinion holders, and topics expressed in online news media text. In Proceedings of the Workshop on Sentiment and Subjectivity in Text (SST-2006), pages 1–8. find
  14. H. Lee, A. Chang, Y. Peirsman, N. Chambers, M. Surdeanu, and D. Jurafsky. 2013. Deterministic coreference resolution based on entity-centric, precision-ranked rules. Computational Linguistics, 39(4):885–916. ↗ our card
  15. Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo N. Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick van Kleef, Sôren Auer, and Christian Bizer. 2013. DBpedia – A Large-scale, Multilingual Knowledge Base Extracted from Wikipedia. Semantic Web Journal, 6:167–195. find
  16. Binyang Li, Lanjun Zhou, Shi Feng, and Kam-Fai Wong. 2010. A unified graph model for sentence-based opinion retrieval. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL-2010), pages 1367–1375. find
  17. Bing Liu. 2012. Sentiment analysis and opinion mining. Synthesis Lectures on Human Language Technologies, 5(1):1–167. ↗
  18. Bing Liu. 2015. Sentiment Analysis - Mining Opinions, Sentiments, and Emotions. Cambridge University Press. find
  19. Pablo N. Mendes, Max Jakob, Andrés Garcı́a-Silva, and Christian Bizer. 2011. DBpedia Spotlight: Shedding light on the web of documents. In Proceedings of the 7th International Conference on Semantic Systems (I-Semantics-2011), pages 1–8. find
  20. Saif M. Mohammad. 2016. Challenges in sentiment analysis. In D. Das, E. Cambria, and S. Bandyopadhyay, editors, A Practical Guide to Sentiment Analysis. Springer. ↗
  21. N. Nicolov, F. Salvetti, and S. Ivanova. 2008. Sentiment analysis: Does coreference matter? In Proceedings of the Symposium on Affective Language in Human and Machine, pages 37–40. find
  22. Neil O’Hare, Michael Davy, Adam Bermingham, Paul Ferguson, Páraic Sheridan, Cathal Gurrin, and Alan F. Smeaton. 2009. Topic-dependent sentiment analysis of financial blogs. In Proceedings of the 1st International Conference on Information and Knowledge Management Workshop on Topic-sentiment Analysis for Mass Opinion (TSA-2009), pages 9–16. find
  23. Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up?: Sentiment classification using machine learning techniques. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP-2002), pages 79–86. ↗
  24. Heiko Paulheim and Johannes Fürnkranz. 2012. Unsupervised Generation of Data Mining Features from Linked Open Data. In Proceedings of the 2nd International Conference on Web Intelligence, Mining and Semantics (WIMS-2012), page 31. find
  25. Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. Semeval-2014 task 4: Aspect based sentiment analysis. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval-2014), pages 27–35. find
  26. Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Suresh Manandhar, and Ion Androutsopoulos. 2015. Semeval-2015 task 12: Aspect based sentiment analysis. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval-2015), pages 486–495. find
  27. Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Ion Androutsopoulos, Suresh Manandhar, Mohammad AL-Smadi, Mahmoud Al-Ayyoub, Yanyan Zhao, Bing Qin, Orphée De Clercq, Véronique Hoste, Marianna Apidianaki, Xavier Tannier, Natalia Loukachevitch, Evgeniy Kotelnikov, Núria Bel, Salud Marı́a Jiménez-Zafra, and Gülşen Eryiğit. 2016. Semeval-2016 task 5: Aspect based sentiment analysis. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 19–30. ↗
  28. José Saias. 2015. Sentiue: Target and aspect based sentiment analysis in SemEval-2015 Task 12. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval-2015), pages 767–771, June. find
  29. Veselin Stoyanov and Claire Cardie. 2006. Partially supervised coreference resolution for opinion summarization through structured rule learning. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP-2006), pages 336–344. find our card
  30. Zhiqiang Toh and Jian Su. 2015. NLANGP: Supervised machine learning system for aspect category classification and opinion target extraction. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval-2015), pages 496–501, June. find
  31. Zhiqiang Toh and Jian Su. 2016. NLANGP at SemEval-2016 Task 5: Improving Aspect Based Sentiment Analysis using Neural Network Features. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 282–288. find
  32. P. Vossen, I. Maks, R. Segers, H. van der Vliet, M.F. Moens, K. Hofmann, E. Tjong Kim Sang, and M. de Rijke. 2013. Cornetto: a lexical semantic database for Dutch. In Peter Spyns and Jan Odijk, editors, Essential Speech and Language Technology for Dutch, Theory and Applications of Natural Language Processing, pages 165–184. Springer. find
  33. T. Wilson, J. Wiebe, and P. Hoffman. 2009. Recognizing contextual polarity: an exploration of features for phrase-level sentiment analysis. Computational Linguistics, 35(3):399–433. ↗
  34. Lei Zhang, Shuai Wang, and Bing Liu. 2018. Deep learning for sentiment analysis : A survey. ↗
  35. Y. Zhao, B. Qin, T. Liu, and W. Yang. 2015. Aspect-Object Alignment with Integer Linear Programming in Opinion Mining. PLOS One, 10(5). ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/clercq-2020-it-s-absolutely-divine-can/