📄 Learning Extraction Patterns for Subjective Expressions

Summary

The paper tackles sentence-level subjectivity detection, i.e. deciding whether a sentence states an opinion, speculation or accusation or reports a fact. Manually labeled sentence data is scarce. Riloff and Wiebe build a bootstrapping pipeline around unannotated text. Two cautious classifiers, HP-Subj and HP-Obj, rely on known lexical subjectivity clues and label only the sentences they are confident about. These automatically labeled sentences then train an extraction pattern learner modeled on AutoSlog-TS. The learner yields lexico-syntactic patterns that are statistically tied to subjective sentences. The patterns label further sentences as subjective, which enlarges the training data for another learning round. They are also added as features to HP-Subj. Patterns alone reach 71–85% precision. Feeding them into HP-Subj lifts its recall from 32.9% to 40.1% while precision only slips from 91.3% to 90.2%. This shows that cheap, confident labeling of raw text can replace manual annotation for learning a broad subjective vocabulary. That vocabulary includes idiomatic, non-compositional expressions that single words or N-grams miss.

Key points

• HP-Subj and HP-Obj use word and N-gram clue lists drawn from earlier resources, such as Levin's verb classes, Ballmer and Brennenstuhl, Framenet experiencer lemmas and Hatzivassiloglou–McKeown polarity adjectives. They abstain on any sentence where they lack confidence.
• Each pattern is filtered by two thresholds. θ1 is its frequency in the training data, and θ2 is the share of its occurrences that fall in subjective sentences. Across threshold settings, precision ranges from 71% to 85%.
• The second bootstrapping cycle works as follows:
  • Patterns with θ1≥5 and θ2=1.0 labeled about 9,500 new subjective sentences.
  • These were combined with 7,500 earlier subjective sentences and the same ~17,000 HP-Obj objective sentences.
  • The cycle produced 4,248 new patterns at θ1≥2 and θ2≥.60, including 308 at θ1≥10 and θ2=1.0.
  • The new patterns raised recall by about 2–4% at a precision cost of 0.5–2%.
• On a 2,197-sentence test set, HP-Subj improved from 32.9 to 40.1 recall, with precision going from 91.3 to 90.2. The augmented version uses learned patterns only for sentences the original left unlabeled.
• A cited earlier study found that 44% of sentences in news (with editorials and reviews excluded) were subjective. This motivates classifying individual sentences rather than whole documents.

Interesting ideas

• I1. Abstaining classifiers as label generators. A classifier that labels only a minority of sentences, but labels them reliably, can still produce a big training set from abundant raw text. This is attractive for low-resource languages or tasks where annotation is expensive.
• I2. Patterns with slots instead of N-grams. Templates like <x> drives <y> up the wall capture flexible idioms whose meaning is not the sum of their words. Such constructions vary too much for fixed word sequences.
• I3. Voice and syntactic form as signal. The same verb can be subjective in one construction and neutral in another; "bombed" said of a comedian is subjective in the active voice but not in the passive. Encoding syntactic roles in features can therefore separate connotations that bag-of-words models merge.
• I4. Guarding against class drift during bootstrapping. The loop produces only new subjective examples, so the authors rebalanced each round. They kept the objective set fixed and swapped in new subjective sentences rather than letting that class grow. This is a simple safeguard for any self-training setup where one class grows faster.
• I5. Conservative fusion rules. New patterns could label a sentence only if two of them co-occurred, or if one appeared alongside an existing clue. This combination rule is a cheap way to gain coverage while keeping precision almost intact.