📄 A Comparative Study of Arabic Sentiment Swap Models for AraSentEval 2026

Summary

The paper describes the winning entry for AraSentEval 2026 Subtask 2, Arabic sentiment swap. In this task a sentence must be rewritten so that its polarity flips while its content and fluency stay intact. The authors compared several model families on the MA’aks parallel corpus, which mixes MSA with Gulf, Levantine, Egyptian and Moroccan varieties:

• Arabic-pretrained encoder–decoders: AraT5v2 and AraBART
• a multilingual encoder–decoder: mT5
• a decoder-only model: GPT-J-6B

The Arabic seq2seq models clearly beat both the multilingual and the decoder-only baselines. They also tried rebalancing the dialect mix. A dialect classifier labelled the training data, and cheap surface perturbations were then added to the minority dialects. This made the distribution more even, but by the paper's own account it gave no reliable improvement.

The final system lets AraT5v2 and AraBART each produce a candidate rewrite. A sentiment classifier, CAMeLBERT-DA, then keeps whichever candidate better matches the target polarity. This ensemble placed first in the subtask. The work matters because it gives a strong baseline for controlled generation in Arabic, a language with rich morphology and many dialects.

Key points

• MA’aks splits:
  • training: 6,262 pairs (about 55% positive)
  • validation: 1,315 pairs
  • test: 647 unlabeled sentences
• Individual fine-tuned models:
  • AraT5v2: BLEU 42.71, chrF 65.0, sentiment accuracy 0.7461
  • AraBART: BLEU 42.82, chrF 65.53, accuracy 0.6966
• The ensemble with CAMeLBERT-DA selection reached BLEU 43.0, chrF 65.36 and accuracy 0.7554, which ranked first.
• Dialect labelling used the MARBERTv2 dialect classifier.
  • Before augmentation, MSA made up 49.77% of the data.
  • After augmentation the data grew to 8,157 instances and MSA's share fell to 38.22%.
  • The perturbations were tatweel removal, elongation, punctuation changes and emoji insertion.
• mT5 and GPT-J-6B performed noticeably worse. The remaining errors cluster around:
  • MSA/Gulf ambiguity
  • emojis
  • idioms
  • sentiment that is implied rather than stated

Interesting ideas

• I1. Generate-then-judge ensembling: each model proposes one output, and an independent classifier picks the candidate that best has the desired property; no output merging is needed.
• I2. Score the two goals separately: reference overlap (BLEU/chrF) and target-attribute correctness (classifier accuracy) are reported side by side, and they can disagree, since AraBART had higher overlap but worse polarity.
• I3. Audit before you augment: an automatic tagger was run over the training data to measure how skewed the dialect coverage was, and only then were minority slices targeted.
• I4. An honest negative result: balancing the label statistics with surface-only noise did not translate into better robustness, a reminder that a balanced distribution does not mean new capability.
• I5. The error analysis singles out polarity carried by implicit cues, idioms and emojis instead of explicit evaluative words.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (back): when generating from a meaning graph, produce several surface variants, re-parse each one, and keep the variant whose recovered graph best matches the source graph, giving an explainable round-trip selection.
• I2 (learn): evaluate generation and translation with both a surface-overlap metric and a meaning-level check, so that fluent but wrong outputs are not rewarded.
• I3 (learn): tag treebanks and silver data by genre or register to find thin regions before deciding where to add or synthesize training material.
• I4 (tokens): such perturbations (elongation, punctuation, emoji) are better used as robustness probes for the tokenizer and tagger than as training data that is assumed to help.
• I5 (frames): the implicature and irony components could explicitly flag evaluative content that has no overt polarity marker, so that downstream steps know a sentiment judgment rests on inference.