📄 Sentiment Analysis of Arabic Tweets Using Large Language Models

Summary

The paper looks at binary (positive vs. negative) sentiment classification of Arabic tweets, a setting made hard by dialect variety, rich morphology, informal spelling and Arabic–English code-switching. The authors use the Kaggle "Arabic Sentiment Twitter Corpus" and apply standard cleaning: removing punctuation, dates and times, tokenizing, and dropping stopwords. They then turn each tweet into a fixed sentence vector using three pretrained encoders: MiniLM-L12-v2, Universal Sentence Encoder (USE) and XLM-R base. These vectors feed five classical classifiers. Despite the "fine tuning" framing, the encoders are used frozen, as feature extractors. The central experiment runs every encoder–classifier pair twice, once with emojis kept and once with them removed. Keeping emojis helps every encoder. The best result, 85.98% accuracy, comes from MiniLM embeddings with Random Forest on the emoji-preserving data; XLM-R base peaks at 78.98% without emojis. The work is a simple baseline, but it shows that preprocessing choices about non-word symbols can matter as much as the choice of encoder.

Key points

• Data: "Arabic Sentiment Twitter Corpus" (Kaggle), 56,795 tweets, nearly balanced at 28,513 positive and 28,282 negative, collected 31 March – 29 April 2019. It contains emojis, code-mixed Arabic/English and informal writing.
• Encoders: MiniLM-L12-v2 (384-dim sentence vector), USE (512-dim), and XLM-R base (sentence vector from mean pooling of last-layer token states). All are used as frozen feature extractors.
• Classifiers: Logistic Regression, SVM, Random Forest, Decision Tree and KNN. Each is evaluated on accuracy, F1, precision and recall.
• Headline results:
  • Best overall: MiniLM + Random Forest with emojis, 85.98%.
  • With emojis, XLM-R + Random Forest reaches 82.48%.
  • Without emojis, XLM-R's best is 78.98%.
  • Random Forest is reported as the strongest classifier across encoders, though the paper's prose and tables are not fully consistent on some rankings.
• USE ranks lowest: the authors blame its general-purpose similarity training and thin Arabic coverage, but note that emojis partly compensate.

Interesting ideas

• I1. Same pipeline, one input class removed: Running the identical pipeline with and without one kind of input (here emojis) isolates how much signal that kind carries, more cleanly than comparing different systems.
• I2. Non-word symbols as content: Emojis are treated as meaningful tokens rather than noise to clean away, because they often carry the polarity of a short informal text.
• I3. Encoder × classifier grid as a probe: Pairing frozen encoders with simple classifiers is a cheap way to ask how much of a task is linearly or shallowly recoverable from each representation.
• I4. Explicit cue as a fallback: USE improved notably once emojis were present, suggesting that explicit surface cues can rescue a weak semantic model on short texts.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Use single-feature ablations as negative controls, e.g. rerun an evaluation with punctuation, emoticons or casing stripped to measure what each step actually depends on.
• I2 (tokens): Keep emojis and emoticons as distinct tokens with their own tag and lemma, so later stages can see them instead of losing them in normalization.
• I3 (learn): Probe gparse's internal states with simple classifiers for properties like polarity or speech act, as a quick diagnostic of what the parser representation already encodes.
• I4 (frames): When the parse-based reading of attitude or irony is uncertain, an emoji or emoticon attached to the clause could serve as an explainable, separately logged cue for the speaker's stance.