Elamur Research⌕
☰
home / papers / 2025 / Sentiment Analysis on Arabic Dialects: A Multi-Dialect Benchmark

Sentiment Analysis on Arabic Dialects: A Multi-Dialect Benchmark

Paper · 2025 · Abdusalam F. Ahmad Nwesri, Nabila Almabrouk S. Shinbir and Amani Bahlul Sharif

Summary

This system paper reports on a team's entry in the AHASIS shared task, held at RANLP 2025, which targets three-way sentiment classification (positive, negative, neutral) of hotel reviews written in Arabic dialects rather than Modern Standard Arabic. The organizers supplied 860 labelled training reviews split evenly between Saudi and Moroccan Darija, plus a 216-review test set. The authors fine-tuned six pretrained Arabic encoders: three AraBERT variants, MARBERTv2, QARiB and DarijaBERT. They ran three setups: the original mixed training set, each dialect trained separately with fixed hyperparameters, and a synthetic set where same-ID reviews from both dialects were concatenated into one longer input. MARBERTv2 performed best on the official test set with a 79% F1, which placed the team third of 14. Across setups, models pretrained on large, varied social-media corpora consistently beat both the base AraBERT and the single-dialect DarijaBERT. The work matters because it suggests that broad exposure to informal, multi-dialect text helps more than narrow dialect specialization, even when the target data come from one dialect.

Key points

  • Training data: 860 hotel reviews, 430 Saudi and 430 Darija, each subset with 154 positive, 168 negative and 108 neutral items. Test data: 216 reviews (108 per dialect), with at most five submissions per team.
  • Pipeline: HuggingFace Trainer on Google Colab, an 80/20 train/dev split, and maximum length 128 tokens (raised to 512 for the merged-dialect inputs). The official baseline was AraBERT.
  • Dev F1 ranged from 0.849 (DarijaBERT) to 0.884 (MARBERTv2). On the official test set, MARBERTv2 scored F1 0.792 (balanced accuracy 0.784), bert-base-qarib 0.773 and DarijaBERT 0.759.
  • In single-dialect training, bert-large-arabertv02-twitter led on Saudi (F1 0.849) and MARBERTv2 led on Darija (0.779). DarijaBERT unexpectedly did better on Saudi (0.733) than on Darija (0.698).
  • On the concatenated two-dialect inputs, the AraBERT twitter variants scored highest (around 0.83–0.84 F1) and DarijaBERT scored lowest (0.732).

Interesting ideas

  • I1. Build a harder test condition by gluing together aligned items that share an ID and a label, producing longer, mixed-variety inputs whose gold answer is already known, with no new annotation.
  • I2. Freeze all hyperparameters and retrain on each dialect subset separately, so that per-variety differences can be blamed on the model's pretraining rather than on tuning.
  • I3. Report balanced accuracy next to F1 when classes are uneven, so a model cannot look good by favouring the majority class.
  • I4. Pretraining on broad, informal, multi-variety text beat a narrow single-dialect specialist, even on that specialist's own dialect.
  • I5. The dataset pairs reviews across two dialects under shared IDs and labels, which gives a natural check that the same meaning survives a change of surface variety.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Concatenating short labelled sentences into multi-sentence passages could stress-test whether frame and world extraction stays stable as input length and mixing grow.
  • I2 (tree): Evaluating the perceptron parser and gparse per UD treebank genre or dialect under identical settings would show where each parser carries a built-in bias.
  • I3 (learn): Evaluations of speech-act or label classifiers could add a per-class-averaged score so that rare labels are not hidden behind frequent ones.
  • I4 (tokens): When choosing training text for the tagger and lemmatizer, mixing diverse informal sources may generalize better than a narrowly matched corpus.
  • I5 (back): Paired paraphrases could serve as a test that the text → graph → text path yields the same meaning graph for both surface forms.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of TripoliTripoli · Libya5 works · 8 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 3Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 worksEvaluation & benchmarks · 1,220 works

Cite

Cite as Abdusalam F. Ahmad Nwesri, Nabila Almabrouk S. Shinbir and Amani Bahlul Sharif. 2025. Sentiment Analysis on Arabic Dialects: A Multi-Dialect Benchmark. Proceedings of the Shared Task on Sentiment Analysis for Arabic Dialects.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{nwesri2025sentiment,
  title = {{Sentiment Analysis on Arabic Dialects: A Multi-Dialect Benchmark}},
  author = {Abdusalam F. Ahmad Nwesri, Nabila Almabrouk S. Shinbir and Amani Bahlul Sharif},
  booktitle = {Proceedings of the Shared Task on Sentiment Analysis for Arabic Dialects},
  year = {2025},
  doi = {10.26615/978-954-452-109-7-013},
  url = {https://doi.org/10.26615/978-954-452-109-7-013}
}

References

19 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Israe Abdellaoui, Anass Ibrahimi, Mohamed Amine El Bouni, Asmaa Mourhir, Saad Driouech, and Mohamed Aghzal. 2024. Investigating offensive language detection in a low-resource setting with a robustness perspective. Big Data and Cognitive Computing, 8(12). *Table 5: Results of fine-tuning models on the mixed long reviews dataset.Recall and Precision columns have the same values across all models.* Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. ARBERT & MARBERT: Deep bidirectional transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. ness on multi-dialect hotel review sentiment analysis. Our key findings are: • Models pre-trained on large, multi-dialect Twitter corpora (Arabertv02-twitter and MARBERTv2) consistently outperform both standard AraBERT and dialect-specific BERTs. ↗
  2. Tareq Al-Moslmi, Mohammed Albared, Adel Al-Shabi, Nazlia Omar, and Salwani Abdullah. 2017. Arabic senti-lexicon: Constructing publicly available language resources for arabic sentiment analysis. Journal of Information Science, 44:345–362. • Increasing model capacity (large vs. base) and careful hyperparameter tuning yield reliable performance improvements across varied input scenarios. ↗
  3. Maram Alharbi, Salmane Chafik, Saad Ezzini, Ruslan Mitkov, Tharindu Ranasinghe, and Hansi Hettiarachchi. 2025a. Ahasis: Shared task on sentiment analysis for arabic dialects. In Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing (RANLP). • While certain models exhibit dialectal biases (e.g. MARBERTv2 on Darija, bert-large-arabertv02-twitter on Saudi), all top models maintain high accuracy and F1 ( > 81%) even on mixed, longer inputs.
  4. Maram Alharbi, Saad Ezzini, Hansi Hettiarachchi, Tharindu Ranasinghe, and Ruslan Mitkov. 2025b. Evaluating large language models on arabic dialect sentiment analysis. In Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing (RANLP). • Narrowly trained dialect BERTs (DarijaBERT) lag behind, highlighting the value of broad, multi-dialectal pre-training. Our best submission, MARBERTv2, achieved 79% F1 on the blind test set, ranking third among 14 teams. Future directions include exploring adapter-based fine-tuning to reduce resource demands, incorporating explicit dialect identifiers, and extending experiments to additional dialects Felipe Almeida and Geraldo Xexéo. 2019. Word embeddings: A survey. CoRR, abs/1901.09069. ↗
  5. Wissam Antoun, Fady Baly, and Hazem Hajj. 2021. Arabert: Transformer-based model for arabic language understanding. ↗
  6. Gilbert Badaro, Ramy Baly, Hazem Hajj, Nizar Habash, and Wassim El-Hajj. 2014. A large scale Arabic sentiment lexicon for Arabic opinion mining. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 165– 173, Doha, Qatar. Association for Computational Linguistics.
  7. Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  8. Abduslam F A Nwesri, Nabila A S Shinbir, and Hassan Ebrahem. 2023. UoT at NADI 2023 shared task: Automatic Arabic dialect identification is made possible. In Proceedings of ArabicNLP 2023, pages 620–624, Singapore (Hybrid). Association for Computational Linguistics.
  9. Ramy Baly, Georgi Karadzhov, Abdelrhman Saleh, James Glass, and Preslav Nakov. 2019. Multi-task ordinal regression for jointly predicting the trustworthiness and the leading political ideology of news media. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2109–2116, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  10. Raheel Qader, François Portet, and Cyril Labbé. 2019. Semi-supervised neural text generation by joint learning of natural language generation and natural language understanding models. In Proceedings of the 12th International Conference on Natural Language Generation, pages 552–562, Tokyo, Japan. Association for Computational Linguistics. ↗
  11. Claude Sammut and Geoffrey I. Webb, editors. 2010. Encyclopedia of Machine Learning. Springer. find
  12. Ekaba Bisong. 2019. Google Colaboratory, pages 59– 64. Apress, Berkeley, CA. find
  13. Zhiqiang Shi and Ruchit Agrawal. 2025. A comprehensive survey of contemporary Arabic sentiment analysis: Methods, challenges, and future directions. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 3760–3772, Albuquerque, New Mexico. Association for Computational Linguistics.
  14. Naaima Boudad, Rdouan Faizi, Rachid Oulad Haj Thami, and Raddouane Chiheb. 2018. Sentiment analysis in arabic: A review of the literature. Ain Shams Engineering Journal, 9(4):2479–2490. ↗
  15. Ghizlane Bourahouat, Manar Abourezq, and Najima Daoudi. 2023. Leveraging moroccan arabic sentiment analysis using arabert and qarib. In Innovations in Smart Cities Applications Volume 6, pages 299– 310, Cham. Springer International Publishing. find
  16. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  17. Obeida ElJundi, Wissam Antoun, Nour El Droubi, Hazem Hajj, Wassim El-Hajj, and Khaled Shaban. 2019. hULMonA: The universal language model in Arabic. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 68–77, Florence, Italy. Association for Computational Linguistics.
  18. Ashraf Elnagar, Yasmin S. Khalifa, and Anas Einea. 2018. Hotel Arabic-Reviews Dataset Construction for Sentiment Analysis Applications, pages 35–52. Springer International Publishing, Cham. find
  19. Nizar Habash, Fadhl Eryani, Salam Khalifa, Owen Rambow, Dana Abdulrahim, Alexander Erdmann, Reem Faraj, Wajdi Zaghouani, Houda Bouamor, Nasser Zalmout, Sara Hassan, Faisal Al-Shargi, Sakhar Alkhereyf, Basma Abdulkareem, Ramy Eskander, Mohammad Salameh, and Hind Saddiki. 2018. Unified guidelines and resources for Arabic dialect orthography. In Proceedings of the Eleventh International Conference on Language Resources and

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nwesri-2025-sentiment-analysis-arabic-dialects-multi/