📄 Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis

Summary

Sentiment lexicons usually record a word's prior polarity, meaning how it reads out of context. Inside a real sentence, though, the word's effect can be reversed, cancelled, or made neutral. Negation, irrealis modality, word sense, syntactic role and diminishers all play a part. This paper addresses automatic recognition of this contextual polarity at the phrase level.

The authors extended the MPQA Opinion Corpus by going through the subjective expressions it already marks and assigning each one a contextual polarity of positive, negative, neutral or both. They validated this scheme with a two-annotator agreement study. They also built a lexicon of over 8,000 single-word subjectivity clues, each tagged with prior polarity and reliability.

Each clue occurrence is then classified in two machine-learned steps. The first separates neutral from polar instances. The second decides the polarity of the polar ones. In 10-fold cross-validation, the system significantly outperforms a baseline that relies on prior polarity alone, and it covers a large share of sentiment expressions. This matters for opinion question answering, summarization and review mining, which need sentiment pinned to individual expressions rather than whole documents.

Key points

• Annotation scheme: contextual polarity was added to the subjective expressions of the MPQA Opinion Corpus. In total, 15,991 expressions in 425 documents (8,984 sentences) were labelled positive, negative, both or neutral. Annotators judged the polarity conveyed after the whole sentence is interpreted.
• Agreement: on 10 documents with 447 expressions, the two annotators agreed 82% of the time (κ = 0.72). When cases marked uncertain (18%) are dropped, agreement rises to 90% (κ = 0.84).
• Corpus statistics: 47% of sentences contain two or more subjective expressions. Among those, 17% mix positive and negative expressions, and 62% mix polar and neutral ones. This supports analysis below the sentence level.
• Data split: a development set of 66 documents (2,808 expressions) was used for exploration and feature design. The remaining 359 documents (13,183 expressions) were used for 10-fold cross-validation.
• Lexicon: the lexicon starts from the Riloff and Wiebe clues and is expanded with a dictionary, a thesaurus and the General Inquirer lists. Each clue is tagged strongsubj or weaksubj. Prior polarities are 59.7% negative, 33.1% positive, 0.3% both and 6.9% neutral.
• Ablation: dropping individual groups of polarity features changed results significantly in only one case, neutral F-measure when the modification features were removed. Reaching significance over baseline required the combination of features.

Interesting ideas

• I1. Split the decision into two stages. Separate "is this sentiment at all?" from "which direction?". Most lexicon hits are actually neutral in context, so filtering them first keeps the polarity step from being swamped. This design transfers to stance or emotion tagging in other languages.
• I2. Derive token labels from span annotations with explicit merge rules. For example, a clue inside both a positive and a negative span becomes "both", while a negative-plus-neutral overlap becomes negative. This lets existing span-level corpora train word-level classifiers without new annotation.
• I3. Let annotators flag uncertainty. Agreement can then be reported both with and without borderline cases. This shows whether disagreement comes from the guidelines or from genuinely ambiguous items, which is useful for any new annotation project, such as Ukrainian sentiment data.
• I4. Keep neutral-prior words in a sentiment lexicon. Verbs like feel or look and intensifiers like deeply carry no polarity themselves but signal that sentiment is nearby. Including them raises coverage at little cost.
• I5. Use a prior-polarity-only classifier as the baseline, then ablate feature groups. Comparing against the lexicon alone, and then removing groups of features, shows exactly how much context modelling adds. It also shows whether any single feature family is doing the work, which is a cheap, interpretable evaluation pattern.