📄 Sentiment Analysis of Twitter Data

Summary

The paper addresses polarity classification of tweets in two settings: positive vs. negative, and a 3-way task that adds neutral. The authors use 11,875 tweets labelled by hand. A commercial provider collected them from the live Twitter stream without keyword filtering, so the sample is less biased than data gathered with search queries or emoticon-based distant labels. Preprocessing relies on two new resources, an emoticon polarity dictionary and an acronym expansion list. Three model types are compared:

• a unigram baseline;
• a feature-based model with about 100 "Senti-features" that combine prior word polarity with POS tags;
• an SVM tree kernel over a custom tree representation of each tweet.

The 100 features match unigrams, which need over 10,000 features. The tree kernel and the model combinations beat the baseline by roughly 4% absolute. Feature analysis shows that polarity-by-POS features matter most, while Twitter-specific cues add little. The authors conclude that standard NLP tools and lexicons transfer to tweets reasonably well.

Key points

• Data: 11,875 annotated tweets. After removing "junk" tweets, often caused by poor Google Translate output, 8,753 remain. Stratified sampling then gives a balanced set of 5,127 tweets, 1,709 per class.

• Resources:

  • an emoticon dictionary of 170 Wikipedia-listed emoticons with five labels, from Extremely-positive to Extremely-negative plus Neutral;
  • an acronym dictionary of 5,184 entries, e.g. lol → laughing out loud.

• Preprocessing: emoticons are replaced by their polarity, and URLs and @-targets are replaced by tags. Negations are normalised to NOT, and character runs are shortened to three letters.

• 3-way results (5-fold CV, chance 33.33%):

  Model	Accuracy
  Unigram	56.58%
  Senti-features	56.31%
  Unigram + Senti-features	60.50%
  Tree kernel	60.60%
  Tree kernel + Senti-features	≈60.83% (best)

• Feature ranking: the strongest unigrams include clearly polar words, emoticon tags and the URL tag. Unexpectedly, "for" also ranks high, through patterns like "for you".

Interesting ideas

• I1. Polarity crossed with POS: pairing a word's lexicon polarity with its part of speech gives a small, interpretable feature set that rivals large n-gram models. This could suit low-resource languages, where a sentiment lexicon plus a POS tagger are easier to obtain than lots of labelled data.
• I2. Tree kernels instead of feature engineering: encoding each tweet as a tree and letting an SVM kernel compare structures removes most manual feature design. It still performs about as well as the best hand-built features, which makes it a quick strong baseline for new genres.
• I3. Normalising noise into meaningful tags: mapping emoticons, URLs, mentions and negations to placeholder tokens, while keeping three-letter elongations, preserves emphasis and still shrinks the vocabulary. The step is cheap and reusable for any social-media corpus.
• I4. Incremental ablation plus learning curves: adding feature groups one at a time shows which families actually help. Plotting accuracy against training-set size showed the gains over unigrams grow when data is scarce. Both checks are worth copying when justifying compact feature sets.
• I5. Sampling from the stream rather than by query: a random sample of the stream avoids topic and emoticon biases in evaluation data. That is useful whenever a benchmark must reflect real-world language use.