📄 Subjectivity and Sentiment Analysis of Modern Standard Arabic

Summary

The paper addresses the lack of subjectivity and sentiment analysis (SSA) resources and systems for morphologically rich languages, focusing on Modern Standard Arabic (MSA) newswire. The authors built a sentence-level SSA corpus on top of the Penn Arabic Treebank (PATB). Two native speakers labeled each sentence as objective or as positive, negative, or neutral subjective. They also compiled a hand-made polarity lexicon of adjectives.

Using gold PATB tokenization, they compared three ways of normalizing stem words: Surface, Lemma, and Stem. They then added Arabic inflectional features and several language-independent features. Classification runs in two binary steps with linear SVMs: subjective versus objective, then positive versus negative.

The Stem setting worked best for both tasks, and morphological features gave further small gains. The domain-specific adjective lexicon had by far the largest effect on sentiment. The work shows that explicitly modeling morphology helps SSA in a language like Arabic, and it contributes rare annotated resources for the field.

Key points

• Corpus: 2855 sentences from the first 400 documents of PATB Part 1 V 3.0, which is about 54.5% of that part.
  • Inter-annotator agreement was 88.06%.
  • Class counts: 1281 OBJ, 491 S-POS, 689 S-NEG, and 394 S-NEUT.
  • Each sentence also carries a newswire domain label.
• Polarity lexicon: 3982 newswire-domain adjectives, each tagged positive, negative, or neutral.
• Setup:
  • Text uses the PATB+Al tokenization scheme.
  • Classifier is SVM light with a linear kernel and binary presence vectors over N-grams up to 4.
  • Morphological features cover person, state, gender, tense, aspect, and number. No POS tags or diacritics are used.
• Subjectivity results:
  • On DEV, Stem reached 73.17% F, ahead of Lemma (72.74) and Surface (71.97).
  • The best TEST result was 71.54% F (Stem+Morph+ADJ), against a 55.13% majority baseline.
• Sentiment results:
  • On DEV, Stem reached 56.87% F, versus 55.01 for Lemma and 52.53 for Surface.
  • Adding the ADJ lexicon features raised results to 90.93% F on DEV and 95.52% F on TEST, against a 58.38% baseline.

Interesting ideas

• I1. Annotating on top of an existing treebank. Adding the sentiment layer to PATB sentences gives access to gold tokenization and morphology for free. This separates the effect of preprocessing from tagger errors, which is a useful design for any low-resource language that has a treebank.
• I2. Comparing normalization depth as an experimental variable. The authors tested surface forms, dictionary lemmas, and crude IR-style stems that may not be real words. The crude stems won, which suggests that for classification, aggressive normalization can beat linguistically "proper" lemmatization in inflection-heavy languages.
• I3. A small, domain-tailored lexicon as binary features. Simple has-positive-adjective and has-negative-adjective flags drove most of the sentiment performance. A focused hand-built word list can be worth more than elaborate feature engineering.
• I4. Checking features against the baseline. Without the lexicon, the sentiment models stayed below the majority baseline. Always reporting that baseline exposes when a feature-rich setup is not actually learning the task.
• I5. Separate test evaluation for the best configurations only. Settings were chosen by cross-validation on 80% of the data, then only the winners were scored on a held-out 20%. That split is larger than any single dev fold, which limits overfitting to the development data.