📄 Determining the Sentiment of Opinions

Summary

Kim and Hovy address opinion analysis. Starting from a collection of documents and a subject of interest, the goal is to find who holds an opinion about the topic and whether that opinion is positive, negative or neutral. They model an opinion as a tuple of Topic, Holder, Claim and Sentiment. The work proceeds in two stages.

1. Word level. Small hand-picked seed lists of positive and negative adjectives and verbs are expanded through WordNet synonyms and antonyms. Two Naive-Bayes-style models then score how strongly an unseen word leans each way, based on how many of its synonyms fall into each list.
2. Sentence level. IdentiFinder proposes candidate holders. A region of text around the holder is delimited, and the word sentiments inside it are combined with one of three models: sign product, harmonic mean or geometric mean.

Sentence accuracy reached 81% when holders were given by annotators and 67% when holders were detected automatically. The study shows that a usable sentiment and holder system can be built from minimal manual seeding without annotated training corpora.

Key points

• Seed lists: 23 positive and 21 negative verbs, and 15 positive and 19 negative adjectives; nouns were added later.
• Expanded lexicon: two rounds of WordNet expansion produced 5880 positive and 6233 negative adjectives, plus 2840 positive and 3239 negative verbs. Adjectives used synonyms and antonyms; verbs used only synonyms.
• Word-level agreement: on 231 test adjectives, agreement with humans was about 68%, against about 58–59% for random selection.
• Holder detection: BBN's IdentiFinder supplied PERSON and ORGANIZATION entities as holder candidates. When several appeared, the one closest to the topic phrase was chosen.
• Model comparison:
  • The sign-only Model 0 gave the best single result.
  • With manual holders, the region from the holder to the end of the sentence (Window4) worked best.
  • Automatic holder detection caused roughly 7 more misclassified sentences, about 11%.

Interesting ideas

• I1. Lexicon from a thesaurus instead of a corpus. Growing a polarity lexicon from a few seeds via synonym and antonym links avoids annotated data entirely. This could bootstrap sentiment resources for lower-resourced languages that have a WordNet-like resource.
• I2. Scoring both polarities for each word. Giving every word a positive and a negative strength exposes ambiguous items such as "great" or "get", which appear in both lists. A threshold can then filter them out instead of forcing a single label.
• I3. Sweeping the context window. Comparing several region definitions (full sentence, holder-to-topic span, padded span, holder to sentence end) is a cheap stand-in for parsing. It shows how much context the combination step actually needs.
• I4. Testing with gold and with automatic upstream input. Evaluating once with human-annotated holders and once with automatic ones isolates the error added by holder detection from the error of sentiment classification. This is a good practice for any pipeline evaluation.
• I5. Trying the simple sign baseline first. Multiplying polarity signs, with explicit negation flipping, beat strength-weighted averaging. Simple baselines deserve a test before more elaborate scoring.