📄 Rasid at StanceEval: Fine-tuning MARBERT for Arabic Stance Detection

Summary

This paper describes a team's entry to the StanceEval shared task on Arabic stance detection. The task is to decide whether a tweet's author is for, against, or neutral toward one of three topics: the COVID-19 vaccine, digital transformation, and women empowerment. The data comes from the MAWQIF Twitter corpus. It is informal, dialect-heavy, and has unevenly distributed classes. The authors compared three systems: a fine-tuned MARBERT, a fine-tuned AraBERT, and a classical ensemble built on TF-IDF features. They picked MARBERT because its pretraining data includes tweets in both dialects and MSA, which fits the task's register. They fed the models unprocessed text, since normalization gave no measurable gain. MARBERT scored highest overall with a macro F1 of 0.7566, ahead of AraBERT and the ensemble. However, AraBERT did better on the digital transformation target. The work shows that matching the pretraining domain to the target text matters more than heavy preprocessing. It also shows that contextual models clearly outperform n-gram classifiers on this kind of target-directed opinion task.

Key points

• The data is the MAWQIF corpus, whose tweets are labeled for stance, sentiment and sarcasm. The training split has 3,502 tweets: 1,167 Covid Vaccine, 1,145 Digital Transformation and 1,190 Women Empowerment. The authors held out 20% of it as a validation set.
• MARBERT has a BERT-Base-sized architecture (~163M parameters, 12 layers, hidden size 768, 12 heads). It was fine-tuned for 5 epochs with learning rate 1e-5, batch size 64 and a maximum length of 256 tokens, using Hugging Face tooling.
• The baseline ensemble combines Logistic Regression, an SVM and Multinomial Naive Bayes. Its features are TF-IDF over word 1–2-grams plus character 2–5-grams.
• Overall macro F1 was 0.7566 for MARBERT, 0.7344 for AraBERT and 0.7033 for the ensemble. MARBERT's best single target was Women Empowerment, with F1 0.8191.
• The authors say MARBERT won everywhere, but the per-target table contradicts this on Digital Transformation, where AraBERT scored 0.7568 against MARBERT's 0.7394.

Interesting ideas

• I1. Pick the pretrained encoder by how closely its pretraining text resembles the target genre, not by size or general reputation; register match can outweigh other differences. — For Mova (tree): When choosing or adapting the gparse encoder for informal or social-media English, prefer a backbone pretrained on similar text, and compare it against a general one.
• I2. Do not assume text cleaning helps: run a controlled check with and without normalization, and drop the step if it adds nothing. — For Mova (tokens): Each normalization rule in the tokenizer and tagger pipeline could be justified by a with/without ablation rather than kept by habit.
• I3. Keep a cheap but strong classical baseline (word plus character n-gram TF-IDF) to show how much the heavier model actually adds. — For Mova (learn): Use a bag-of-n-grams classifier as a negative control in evaluations, to confirm that tree- and frame-based gains are not achievable by surface statistics alone.
• I4. Always break results down per target, because an averaged score can hide cases where the "winner" loses, as happened here. — For Mova (answer): Report school-test and QA accuracy per course, task type or text genre, so regressions hidden by the overall mean become visible.
• I5. Stance is an attitude aimed at a specific target and differs from general emotional tone; treating the two as separate labels on the same text keeps them apart. — For Mova (frames): In prag, represent an author's stance as an explicit holder–target relation, distinct from sentiment and irony, so answers can explain whom or what the attitude concerns.