Elamur Research⌕
☰
home / papers / 2026 / CasbAI at AraSentEval 2026: Robust Dialectal Arabic Sentiment Classification via Multi-Seed Ensembling and Data Augmentation.

CasbAI at AraSentEval 2026: Robust Dialectal Arabic Sentiment Classification via Multi-Seed Ensembling and Data Augmentation.

Paper · 2026 · Chaima Abdelaziz, KahinaHouda Saadaoui, Faiza BELBACHIR et al.

Summary

This paper describes the third-place system in Subtask 1 of AraSentEval 2026 (OSACT7). The task is three-way sentiment classification (positive, negative, neutral) of hotel reviews in Moroccan Darija, Saudi, Jordanian and Egyptian Arabic. The training data is very small: 1,731 labelled reviews and 312 hidden test items. Part of it was produced by translating MSA with NLLB-200, and native speakers then corrected the output.

The authors fine-tune MARBERT and add three pieces on top:

  • an extra attention layer with a residual connection,
  • learned pooling across all tokens instead of using only [CLS],
  • a compact classification head.

Training uses a weighted mix of focal loss and label-smoothed cross-entropy. Robustness comes from several inexpensive techniques stacked together: random single-word swaps from a hand-made dialect lexicon to balance the classes, seven random seeds averaged at the probability level, Stochastic Weight Averaging within each seed, and test-time augmentation.

The system reaches 84.62% macro F1 on the official test set, ranking 3rd of 15 teams. Ablations show that every component helps slightly, with augmentation helping most. The work offers a practical recipe for tiny dialectal datasets. However, the large gap between internal validation (~96%) and the test score suggests the in-house split overestimates performance.

Key points

  • Data: the training set has 657 negative, 609 positive and 465 neutral items, split almost exactly 25% per dialect. Test labels are withheld. Preprocessing is limited to whitespace normalisation.
  • Backbone choice: in preliminary validation, MARBERT scored 94.47%, ahead of CAMeLBERT-MSA (87.84%), CAMeLBERT-Mix (86.87%), DziriBERT (84.21%) and AraBERTv2 (81.87%). The final model has about 168M parameters, of which roughly 3M are new.
  • Recipe details:
    • With probability 0.5, one word per sentence is swapped from the lexicon. Classes are filled up to 657 each, giving 1,971 items (+13.9%).
    • Seven seeds are trained. SWA starts at epoch 10 and averages the last 5 epochs at lr 4×10⁻⁶.
    • Each seed's output blends 40% regular weights and 60% SWA weights.
    • TTA averages predictions over 5 augmented copies of each input.
  • Ablation (validation macro F1): baseline 0.9447 → +augmentation 0.9509 → +7-seed ensemble 0.9560 → +SWA 0.9619 → +TTA 0.9625. The total gain is +1.78 points. On the test set the team scored 84.62%, versus 94.23% for the winner.
  • Error analysis (single model, 198 validation items):
    • The model makes 13 errors in total.
    • Negative recall is only 0.879, while neutral recall is 0.970.
    • No positive item was ever labelled negative.
    • Error rates by dialect: Darija 9.0%, Egyptian 8.2%, Jordanian 5.3%, Saudi 2.3%.
    • Pseudo-labeling was dropped because no test prediction exceeded the 95% confidence threshold.

Interesting ideas

  • I1. Before self-training, the authors checked whether any unlabelled prediction actually cleared the confidence bar; none did, so they abandoned pseudo-labeling instead of lowering the threshold.
  • I2. Their errors cluster around concessive negation, where a later clause partly cancels an earlier negative marker, and around positive words under conditional or sarcastic framing.
  • I3. The per-class analysis shows the neutral class acting as an "attractor" that absorbs uncertain negatives. This is a useful diagnostic for any system that has a fallback or middle category.
  • I4. The authors train several seeds and average them, partly because small training sets make single runs sensitive to initialisation.
  • I5. Test-time augmentation averages predictions over lightly perturbed copies of the input, which reduces sensitivity to incidental wording.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): the silver-data pipeline should report how many items pass the witness-agreement gate, and skip self-training when the yield is near zero rather than quietly relaxing the gate.
  • I2 (frames): add explicit rules that let the clause after a concessive marker override the evaluation of the earlier clause, and flag conditional frames so their sentiment is not read as asserted.
  • I3 (answer): measure whether "cannot tell"-type answers on school tests and fable QA soak up items the reader should have resolved, using precision/recall per answer type.
  • I4 (tree): train gparse with several seeds and report the seed spread before accepting a claimed LAS improvement; averaging seed outputs could also stabilise silver parses.
  • I5 (learn): use synonym-substituted variants as a consistency check, where the UD tree shape and frame roles should stay the same, and flag sentences whose analysis flips as candidates for repair rules.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓École Nationale Supérieure d'InformatiqueAlgiers · Algeria8 works · 430 citations

Also on the byline: École Centrale d'Électronique.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 works

Cite

Cite as Chaima Abdelaziz, KahinaHouda Saadaoui, Faiza BELBACHIR and Lynda Said Lhadj. 2026. CasbAI at AraSentEval 2026: Robust Dialectal Arabic Sentiment Classification via Multi-Seed Ensembling and Data Augmentation. The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/32cq3gywnhjs

@inproceedings{abdelaziz2026casbai,
  title = {{CasbAI at AraSentEval 2026: Robust Dialectal Arabic Sentiment Classification via Multi-Seed Ensembling and Data Augmentation.}},
  author = {Chaima Abdelaziz, KahinaHouda Saadaoui, Faiza BELBACHIR and Lynda Said Lhadj},
  booktitle = {The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks},
  year = {2026},
  doi = {10.63317/32cq3gywnhjs},
  url = {https://doi.org/10.63317/32cq3gywnhjs}
}

References

18 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mustafa Abdalaa, Richard Sutcliffe, Xia Sun, Jun Feng, Eiad Almekhlafi, and Ephrem Retta. 2021. Improving arabic sentiment analysis using cnn-based architectures and text preprocessing. Computational Intelligence and Neuroscience. find
  2. Amine Abdaoui, Mohamed Berrimi, Mourad Oussalah, and Abdelouahab Moussaoui. 2022. Dziribert: a pre-trained language model for the algerian dialect. ↗
  3. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. Marbert: A robust pre-trained model for arabic social media. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL 2021), pages 7088–7105. find
  4. Maram Alharbi, Saad Ezzini, Hansi Hettiarachchi, Tharindu Ranasinghe, and Ruslan Mitkov. 2025a. Evaluating large language models on arabic dialect sentiment analysis. In Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing (RANLP). find
  5. Maram I Alharbi, Salmane Chafik, Saad Ezzini, Ruslan Mitkov, Tharindu Ranasinghe, and Hansi Hettiarachchi. 2025b. Ahasis: Shared task on sentiment analysis for arabic dialects. In Proceedings of the Shared Task on Sentiment Analysis for Arabic Dialects, pages 1–6.
  6. Maram I Alharbi, Saad Ezzini, Hansi Hettiarachchi, Tharindu Ranasinghe, and Ruslan Mitkov. 2025c. Ador: Dataset for arabic dialects in hotel reviews: A human benchmark for sentiment analysis. In Proceedings of the First Workshop on Advancing NLP for Low-Resource Languages, pages 187–191.
  7. Wissam Antoun, Fady Baly, and Hazem Hajj. 2021. Arabert: Transformer-based model for arabic language understanding. ↗
  8. Gilbert Badaro, Ramy Baly, Hazem Hajj, Nizar Habash, and Wassim El-Hajj. 2014. A large scale arabic sentiment lexicon for arabic opinion mining. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 165–173.
  9. Faiza Belbachir. 2023. Foul at semeval-2023 task 12: Marbert language model and lexical filtering for sentiment analysis of tweets in algerian arabic. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval 2023), pages 389–396. find
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT 2019.
  11. Saad Ezzini, Shadi Abudalfa, Maram Alharbi, Salmane Chafik, Hamzah Luqman, Mo El-Haj, Paul Rayson, Samaher Alghamdi, and Reem Alotaibi. 2026. AraSentEval: A shared task on sentiment analysis and swapping in arabic. In Proceedings of the 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7), co-located with the Language Resources and Evaluation Conference (LREC 2026), Palma, Mallorca, Spain. find
  12. Huang Huang, Fei Yu, Jianqing Zhu, Xuming Sun, Qi Zhang, Jizhong Wang, and Fei Huang. 2024. AceGPT: Localizing large language models in Arabic. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 6383–6396, Mexico City, Mexico. ↗
  13. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. The interplay of variant, size, and task type in arabic pre-trained language models.
  14. Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. 2014. A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404.2188. ↗
  15. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. ↗
  16. Saif M. Mohammad, Mohammad Salameh, and Svetlana Kiritchenko. 2016. Sentiment lexicons for arabic social media. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016).
  17. Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, William Marshall, Gurpreet Xu, Haewon Kim, Hitesh Bhatt, et al. 2023. Jais and Jais-chat: Arabic-centric foundation and instruction-tuned open generative large language models. In arXiv preprint arXiv:2308.16149. ↗
  18. Qixuan Yin. 2024. Three-class text sentiment analysis based on lstm. arXiv preprint arXiv:2412.17347. 8. Appendix ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/abdelaziz-2026-casbai-arasenteval-2026-robust-dialectal/