📄 Citius: A Naive-Bayes Strategy for Sentiment Analysis on English Tweets

Summary

The paper addresses SemEval-2014 Task 9, subtask B: deciding whether an English tweet is positive, negative or neutral. The authors build a family of Naive Bayes classifiers in Perl. Before lemmatization, tweets are normalized: URLs, usernames and hashtags are removed, stretched letters are collapsed, and emoticons and interjections become sentiment words. Features are content-word lemma unigrams, POS-pattern multiwords, a merged polarity lexicon and negation handling that depends on part of speech. The central design choice splits the problem in two. A binary classifier is trained only on positive and negative tweets, and a simple lexicon test labels a tweet neutral when it contains no polarity lemma. This split beat a direct three-class model. Adding the lexicon and multiwords pushed F-score from 0.49 to 0.63. The best submission ranked 26th of 53 systems, which is mid-table. The work shows that a cheap, transparent pipeline built from linguistically motivated features can reach average competitive performance.

Key points

• Data: 6,408 SemEval training tweets (tweeti-b.dist.tsv), with Neutral, Objective and Neutral-or-Objective merged into one Neutral label. "Unconstrained" runs also used 5,050 external positive and negative tweets (Narr et al., 2012). Development testing used tweeti-b.devel.tsv.

• The polarity lexicon has 10,850 entries merged from AFINN-111 (1,520 lemmas), Hedonometer (top 300 positive), the Hu&Liu list (5,695) and SentiWordNet-3.0 (6,600 strongest lemmas).

• Multiwords came from POS patterns such as NOUN-ADJ, ADJ-NOUN, NOUN-NOUN, VERB-NOUN, NOUN-PRP-NOUN and VERB-PRP-NOUN, extracted with the authors' GPL tool GaleXtra.

• Development F-scores were:

  System	F-score
  CONSTR-BASE	.49
  CONSTR-BASE-LEX	.56
  CONSTR-BIN-LEX	.57
  CONSTR-BIN-LEX-MW	.61
  UNCONSTR-BIN-LEX	.58
  UNCONSTR-BIN-LEX-MW	.63

• On the official Tweets2014 test set, the constrained system scored 0.62 and the unconstrained one 0.63. A Spanish variant placed 3rd of 13 at TASS-2013.

Interesting ideas

• I1. Each lexicon entry is turned into a synthetic one-lemma training example that carries the entry's polarity, which lets a statistical model absorb a hand-built lexicon without any architecture change.
• I2. Instead of one three-way decision, a lexicon gate first decides whether there is any polarity, and a two-class model then picks the direction; the clean split outperformed the joint classifier.
• I3. Negation scope is chosen by the negator's category: "not" flips only nearby verbs and adjectives, while "no" and "none" flip only nouns, all within a short window.
• I4. POS-tag patterns pick out linguistically motivated multiword units, mostly syntactically linked pairs, and adding them gave the single biggest gain in the ablation.
• I5. Evaluation is run as a ladder of systems that each add one component, plus a split between constrained and unconstrained training data, so each component's contribution is isolated.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Dictionary knowledge such as VerbNet classes or WordNet senses could be injected as minimal synthetic training items, giving rare words coverage the treebank or silver data lacks.
• I2 (frames): Speech-act or irony detection could first ask a cheap, explainable "is anything evaluative here?" question before running a finer classifier, which keeps every decision traceable.
• I3 (frames): The same idea maps naturally onto UD arcs (advmod vs det negation attached to a head), so polarity reversal in frames can follow the tree rather than word distance.
• I4 (tokens): Pattern-based multiword candidates could feed the lemmatizer/tagger as compound units, so that later stages treat them as one concept.
• I5 (learn): Evaluation reports could adopt the same incremental-ablation layout and keep runs on in-house data separate from runs with external data, so data effects are not confused with method effects.