Elamur Research⌕
☰
home / papers / 2006 / SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining

SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining

Summary

Most earlier opinion-mining lexicons label whole words, and they mainly decide whether a word is positive or negative. The harder question of whether a word carries opinion at all got less attention. Word senses were also ignored, even though one word can be neutral in one sense and evaluative in another. Esuli and Sebastiani address this with SentiWordNet 1.0, which annotates every synset of WordNet 2.0 with three graded values: Obj(s), Pos(s) and Neg(s). Each value lies between 0 and 1, and the three sum to 1.

To build it, the authors start from small hand-picked seed sets and grow them by following WordNet relations. They then represent synsets through their glosses and train eight ternary classifiers. The classifiers differ in training data and learning algorithm. A synset's scores are the normalized share of classifiers that voted for each label.

The result covers all of WordNet, roughly 115,000 synsets, and comes with a web interface. Accuracy is estimated only indirectly, and a human-annotated gold standard is in preparation. The resource matters because it gives sense-level, graded opinion information at wide coverage.

Key points

  • Scoring scheme: each WordNet 2.0 synset gets Obj, Pos and Neg scores that add up to 1.0. For example, the "deserving of respect" sense of estimable scores Pos 0.75 and Obj 0.25, while its "can be computed" sense is fully objective.
  • Seed sets: positive and negative seeds come from 14 paradigmatic terms used by Turney and Littman (e.g., nice, nasty). After manually selecting the intended senses, these yield 47 Positive and 58 Negative synsets.
  • Seed expansion: the seeds are expanded over K iterations using the relations adopted for WordNet-Affect: direct antonymy, similarity, derived-from, pertains-to, attribute and also-see. Antonymy flips polarity; the other relations keep it.
  • Score distribution: averaged over all parts of speech, scores are about 0.043 Pos, 0.054 Neg and 0.903 Obj. Adjectives are the most opinion-laden category, with Obj averaging 0.743.
  • Evaluation: quality is inferred indirectly from earlier results on the General Inquirer lexicon. A gold standard of 1000 synsets, each scored by five trained annotators, is under construction.

Interesting ideas

  • I1. Labels from committee agreement: the share of diverse classifiers voting for each class is turned into a soft score. This is a cheap way to get graded labels without any graded training data, and it could be applied to any lexicon or taxonomy.
  • I2. Annotate senses, not word forms: attaching labels to synsets avoids mixing up neutral and evaluative senses of the same word. The same design would suit polarity or other lexical properties in wordnets for other languages.
  • I3. Relation-aware seed propagation: relations that preserve a label spread it unchanged, while antonymy spreads the opposite label. This turns a few dozen hand-picked seeds into a large training set, which is handy for low-resource lexicon building.
  • I4. Glosses as features: representing a sense by its dictionary definition supplies text features for units that otherwise have no context. Any resource with definitions can be classified this way.
  • I5. Graded human annotation: the planned gold standard has annotators split one unit of mass across the three labels. This matches the soft output format and also allows measuring how consistently annotators agree, a useful evaluation design for inherently fuzzy categories.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

239 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 232 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{esuli2006sentiwordnet,
  title = {{SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining}},
  author = {Esuli, Andrea and Sebastiani, Fabrizio},
  booktitle = {Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC'06)},
  year = {2006},
  doi = {10.63317/54vo5wjya4cw},
  url = {https://doi.org/10.63317/54vo5wjya4cw}
}

References

19 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alina Andreevskaia and Sabine Bergler. 2006. Mining WordNet for fuzzy sentiment: Sentiment tag extraction from WordNet glosses. In Proceedings of EACL-06, 11th Conference of the European Chapter of the Association for Computational Linguistics, Trento, IT. Forthcoming. find
  2. Andrea Esuli and Fabrizio Sebastiani. 2005. Determining the semantic orientation of terms through gloss analysis. In Proceedings of CIKM-05, 14th ACM International Conference on Information and Knowledge Management, pages 617–624, Bremen, DE. find
  3. Andrea Esuli and Fabrizio Sebastiani. 2006. Determining term subjectivity and term orientation for opinion mining. In Proceedings of EACL-06, 11th Conference of the European Chapter of the Association for Computational Linguistics, Trento, IT. Forthcoming. find
  4. Vasileios Hatzivassiloglou and Kathleen R. McKeown. 1997. Predicting the semantic orientation of adjectives. In Proceedings of ACL-97, 35th Annual Meeting of the Association for Computational Linguistics, pages 174–181, Madrid, ES. find
  5. Vasileios Hatzivassiloglou and Janyce M. Wiebe. 2000. Effects of adjective orientation and gradability on sentence subjectivity. In Proceedings of COLING-00, 18th International Conference on Computational Linguistics, pages 174–181. find
  6. Jaap Kamps, Maarten Marx, Robert J. Mokken, and Maarten De Rijke. 2004. Using WordNet to measure semantic orientation of adjectives. In Proceedings of LREC-04, 4th International Conference on Language Resources and Evaluation, volume IV, pages 1115–1118, Lisbon, PT. find
  7. Soo-Min Kim and Eduard Hovy. 2004. Determining the sentiment of opinions. In Proceedings of COLING-04, 20th International Conference on Computational Linguistics, pages 1367–1373, Geneva, CH. find
  8. Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of ACL-04, 42nd Meeting of the Association for Computational Linguistics, pages 271–278, Barcelona, ES. find
  9. Bo Pang and Lillian Lee. 2005. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of ACL-05, 43rd Meeting of the Association for Computational Linguistics, pages 115–124, Ann Arbor, US. find
  10. Ellen Riloff, Janyce Wiebe, and Theresa Wilson. 2003. Learning subjective nouns using extraction pattern bootstrapping. In Proceedings of CONLL-03, 7th Conference on Natural Language Learning, pages 25–32, Edmonton, CA. find our card
  11. P. J. Stone, D. C. Dunphy, M. S. Smith, and D. M. Ogilvie. 1966. The General Inquirer: A Computer Approach to Content Analysis. MIT Press, Cambridge, US. find
  12. Hiroya Takamura, Takashi Inui, and Manabu Okumura. 2005. Extracting emotional polarity of words using spin model. In Proceedings of ACL-05, 43rd Annual Meeting of the Association for Computational Linguistics, pages 133–140, Ann Arbor, US. find
  13. Peter D. Turney and Michael L. Littman. 2003. Measuring praise and criticism: Inference of semantic orientation from association. ACM Transactions on Information Systems, 21(4):315– 346. find
  14. Peter Turney. 2002. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In Proceedings of ACL-02, 40th Annual Meeting of the Association for Computational Linguistics, pages 417–424, Philadelphia, US. find
  15. Alessandro Valitutti, Carlo Strapparava, and Oliviero Stock. 2004. Developing affective lexical resources. PsychNology Journal, 2(1):61–83. find
  16. Stefano Vegnaduzzo. 2004. Acquisition of subjective adjectives with limited resources. In Proceedings of the AAAI Spring Symposium on Exploring Attitude and Affect in Text: Theories and Applications, Stanford, US. find
  17. Casey Whitelaw, Navendu Garg, and Shlomo Argamon. 2005. Using appraisal groups for sentiment analysis. In Proceedings of CIKM-05, 14th ACM International Conference on Information and Knowledge Management, pages 625–631, Bremen, DE. find
  18. Theresa Wilson, Janyce Wiebe, and Rebecca Hwa. 2004. Just how mad are you? Finding strong and weak opinion clauses. In Proceedings of AAAI-04, 21st Conference of the American Association for Artificial Intelligence, pages 761–769, San Jose, US. find
  19. Hong Yu and Vasileios Hatzivassiloglou. 2003. Towards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences. In Proceedings of EMNLP-03, 8th Conference on Empirical Methods in Natural Language Processing, pages 129–136, Sapporo, JP. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/esuli-2006-sentiwordnet-publicly-available-lexical-resource/