📄 Dialect-Aware Sentiment Analysis for Ahasis Challenge

Summary

The paper tackles three-way sentiment classification (positive, negative, neutral) of Arabic hospitality reviews written in two very different dialects, Saudi and Moroccan Darija, for the AHaSIS shared task. Instead of building a separate model per dialect, the authors train one model and give it the dialect label as an extra input. They fine-tune Qarib, a BERT-style Arabic model. A learned 64-dimensional dialect vector is joined to the 768-dimensional sentence representation, and the result feeds a small residual classifier with GELU and layer normalization. Before encoding, text goes through general Arabic orthographic normalization plus hand-written mappings for each dialect, which unify common intensifiers and negators. Focal loss and class weighting handle the uneven class sizes. On the test set the system reaches 0.770 macro F1. The same model without dialect features scores 0.72, MARBERT 0.70 and ARBERT 0.67. This suggests that telling a shared model which language variety it is reading is a cheap and effective way to handle variation with very little data.

Key points

• Data: 860 training samples (430 per dialect) and 216 test samples (108 per dialect). The training labels are 336 negative, 308 positive and 216 neutral.
• Setup: ahmedabdelali/bert-base-qarib with max length 128 and LR 2e-6, using linear warmup over 15% of steps. Batch size is 16, made effectively 64 by accumulating gradients over 4 steps. Training runs 10 epochs with early-stopping patience 3, dropout 0.2 and focal loss with gamma 2.0. Lower layers get smaller learning rates than the task layers.
• Overall test results: macro F1 0.770 and balanced accuracy 0.775. Adding dialect features gives a 5-point absolute gain over the plain model.
• Results by dialect: Saudi scores 0.787 F1 and Darija 0.752. The authors attribute the gap to Darija's mixing with Berber, French and Spanish.
• Results by class: the neutral class is weakest (F1 0.734). It has the lowest precision (0.678) but the highest recall (0.800), so ambiguous items tend to be pushed into neutral.
• Error analysis: the main failure sources were
  • sarcasm, especially in Darija;
  • sentiment that depends on cultural context;
  • wrongly identified dialect;
  • doubtful neutral labels that actually carry implicit polarity.

Interesting ideas

• I1. Feed the language variety to one shared model as a small learned vector instead of training a separate model per variety, so common patterns are shared while variety-specific behaviour can still be learned.
• I2. Normalize variant spellings of a few high-value function words (intensifiers, negators) to one canonical form, while deliberately keeping the words that signal sentiment rather than stripping them.
• I3. Judge a feature by removing it from an otherwise identical model, then break the scores down by variety and by class instead of reporting a single number.
• I4. A neutral or default class with high recall but low precision acts as a sink for uncertain inputs, and this asymmetry is a useful diagnostic.
• I5. Sort errors into named causes (sarcasm, missing world context, wrong variety detection, label noise) instead of treating them as one mass.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tree): gparse could take a treebank/register embedding (e.g. EWT vs GUM vs web text, or a source-dialect tag in UD dialect conversion) so a single parser learns both shared and variety-specific attachment preferences.
• I2 (tokens): the en crate could map informal variants ("sooo", "nope", "gonna") to canonical lemmas while keeping a feature recording the original marker, so later intensity or negation logic still sees it.
• I3 (learn): every new feature in training could be checked with a same-model-without-it control plus per-genre and per-class tables, so a gain cannot hide a regression in one slice.
• I4 (frames): the speech-act and irony detectors in prag could watch whether the "literal/neutral" reading silently absorbs hard cases, and send those to explicit uncertainty instead.
• I5 (answer): failed school-test or fable-QA items could be tagged with a cause from a similar fixed taxonomy, pointing each failure to the stage (frames, world, or data) that needs repair.