Elamur Research⌕
☰
home / papers / 2025 / AHaSIS: Shared Task on Sentiment Analysis for Arabic Dialects

AHaSIS: Shared Task on Sentiment Analysis for Arabic Dialects

Paper · 2025 · Maram I. Alharbi, Salmane Chafik, Saad Ezzini et al.

Summary

The AHaSIS 2025 shared task addresses a gap in Arabic sentiment analysis. Most labelled resources cover Modern Standard Arabic (MSA), and models trained on them transfer poorly to regional dialects. The organisers took MSA hotel reviews from the SemEval-2016 ABSA-Hotels data, collected from Booking.com and TripAdvisor. They rendered these reviews into Saudi Arabic and Moroccan Darija, and native speakers checked both dialect accuracy and that the original sentiment survived translation. Each item carries one of three labels: positive, neutral or negative. Teams received a small labelled training split and an unlabelled test split, and were ranked by micro-F1. Over 40 teams registered and 12 submitted test runs. Scores fell in a narrow band from 0.73 to 0.81. The winning system, iWAN-NLP, was an ensemble of dialect-oriented BERT encoders. The task matters because it offers a compact, domain-grounded benchmark for dialect-aware sentiment classification under low-resource conditions. It also shows that fine-tuned encoders still tend to outperform prompted LLMs in this setting.

Key points

  • The source data is MSA hotel reviews from SemEval-2016 ABSA-Hotels, translated into Saudi and Darija and validated by native speakers. The paper gives two sizes: 538 sentiment-balanced reviews in the abstract and 860 training entries in Table 1.
  • The winner, Hend (iWAN-NLP), scored micro-F1 0.81. It fine-tuned MARBERTv2, SaudiBERT and DarijaBERT with stratified 5-fold CV and averaged logits across folds and models. It also used label smoothing, warmup and early stopping.
  • ISHFMG TUN placed second with 0.7916, using a fine-tuned AraBERTv02 with frozen lower layers and class weighting and no external data. LBY also reached 0.79, with MARBERTv2 as its best single model.
  • Prompted LLMs trailed fine-tuned encoders:
    • Lab17's few-shot GPT-4o scored 0.69, against 0.75 for its fine-tuned MARBERT.
    • BirLee's CAMeLBERT-DA scored 0.75, beating zero-shot Allam (0.70), ACeGPT (0.68) and Jais (0.65).
    • Lubbad's Gemini Pro 1.5 prompting reached 0.7361.
  • Other approaches included:
    • SetFit on Arabic-SBERT-100K with 64 examples per class (MAPROC, 0.73).
    • AraELECTRA combined with TF-IDF features (AraNLP).
    • Augmentation through FANAR or AraT5 paraphrasing and lexical perturbation (MARSAD, MARSAD AI).

Interesting ideas

  • I1. Build a dialect resource by carrying an existing labelled standard-language corpus into the dialect, then have native speakers confirm the label still holds; this yields parallel data with inherited annotation at low cost.
  • I2. Average raw logits over several differently pretrained encoders and over cross-validation folds; mixing specialised models makes the result more robust than any single one.
  • I3. Report results per dialect and sort errors into kinds, such as wrong polarity, dialect confusion or truly ambiguous input, so a single score does not hide where a system fails.
  • I4. Combine a contextual encoder with explicit lexical cues, such as TF-IDF terms or a hand-built sentiment lexicon, so decisions rest partly on inspectable evidence.
  • I5. Small fine-tuned models repeatedly beat zero- and few-shot LLMs here, which is a reminder to measure, not assume, the teacher's quality.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): When converting treebank sentences into another English variety or register, the gold UD tree could be carried along and then checked by a verifier. This would give parallel dialect treebanks without annotating from scratch.
  • I2 (learn): The witness parsers producing silver data could merge their arc and label scores before voting, not just their final trees. Cross-fold copies of gparse would also expose how unstable each arc is.
  • I3 (learn): The evaluation could split parser and reader scores by text genre and tag each failure by cause. Repair rules could then target one error class at a time.
  • I4 (frames): Irony and implicature detection in prag could pair tree-based features with an explicit polarity lexicon. Each judgement could then cite the words that triggered it.
  • I5 (learn): Each LLM teacher tier should be scored against a held-out gold set before its labels enter training. This would keep a weaker teacher from contaminating the silver data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓King Fahd University of Petroleum and MineralsDhahran · Saudi Arabia52 works · 4,666 citations🎓Lancaster UniversityLancaster · United Kingdom35 works · 1,615 citations🎓Université Mohammed VI PolytechniqueMarrakesh · Morocco18 works · 171 citations🎓Jazan UniversityGazan · Saudi Arabia9 works · 98 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 works

Cite

Cite as Maram I. Alharbi, Salmane Chafik, Saad Ezzini, Ruslan Mitkov, Tharindu Ranasinghe and Hansi Hettiarachchi. 2025. AHaSIS: Shared Task on Sentiment Analysis for Arabic Dialects. Proceedings of the Shared Task on Sentiment Analysis for Arabic Dialects.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{alharbi2025ahasis,
  title = {{AHaSIS: Shared Task on Sentiment Analysis for Arabic Dialects}},
  author = {Maram I. Alharbi, Salmane Chafik, Saad Ezzini, Ruslan Mitkov, Tharindu Ranasinghe and Hansi Hettiarachchi},
  booktitle = {Proceedings of the Shared Task on Sentiment Analysis for Arabic Dialects},
  year = {2025},
  doi = {10.26615/978-954-452-109-7-001},
  url = {https://doi.org/10.26615/978-954-452-109-7-001}
}

References

18 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. ARBERT & MARBERT: Deep bidirectional transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. ↗
  2. Saja Al-Dabet, Sara Tedmori, and Mohammad AL-Smadi. 2021. Enhancing arabic aspect-based sentiment analysis using deep learning models. Computer Speech & Language, 69:101224. ↗
  3. M. Nabil, M. Aly, and A. F. Atiya. 2015. Astd: Arabic sentiment tweets dataset. In Proceedings of the 2015 IEEE International Conference on Data Mining Workshop (ICDMW), pages 697–703. ↗
  4. Amani A. Aladeemy, Ali Alzahrani, Mohammad H. Algarni, Saleh Nagi Alsubari, Theyazn H.H. Aldhyani, Sachin N. Deshmukh, Osamah Ibrahim Khalaf, Wing-Keung Wong, and Sameer Aqburi. 2024. Advancements and challenges in arabic sentiment analysis: A decade of methodologies, applications, and resource development. Heliyon, 10(21):e39786. ↗
  5. Alberto Poncelas, Pintu Lohar, James Hadley, and Andy Way. 2020. The impact of indirect machine translation on sentiment classification. In Proceedings of the 14th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track), pages 78–88, Virtual. Association for Machine Translation in the Americas. find
  6. Maram I. Alharbi, Saad Ezzini, Hansi Hettiarachchi, Tharindu Ranasinghe, and Ruslan Mitkov. 2025. Evaluating large language models on arabic dialect sentiment analysis. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2025). INCOMA Ltd. find
  7. Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Ion Androutsopoulos, Suresh Manandhar, Mohammad AL-Smadi, Mahmoud Al-Ayyoub, Yanyan Zhao, Bing Qin, Orphée De Clercq, Véronique Hoste, Marianna Apidianaki, Xavier Tannier, Natalia Loukachevitch, Evgeniy Kotelnikov, Nuria Bel, Salud Marı́a Jiménez-Zafra, and Gülşen Eryiğit. 2016. SemEval-2016 task 5: Aspect based sentiment analysis. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 19–30, San Diego, California. Association for Computational Linguistics. ↗
  8. Wissam Antoun, Fady Baly, and Hazem Hajj. 2021. AraELECTRA: Pre-training text discriminators for Arabic language understanding. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 191–195, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. ↗
  9. M. Baali and N. Ghneim. 2019. Emotion analysis of arabic tweets using deep learning approach. Journal of Big Data, 6(1):89. ↗
  10. Faisal Qarah. 2024. Saudibert: A large language model pretrained on saudi dialect corpora. arXiv preprint arXiv:2405.06239. ↗
  11. Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555. ↗
  12. Hadeel Saadany and Constantin Orasan. 2020. Is it great or terrible? preserving sentiment in neural machine translation of Arabic reviews. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 24–37, Barcelona, Spain (Online). Association for Computational Linguistics. ↗
  13. Kamel Gaanoun, Abdou Mohamed Naira, Anass Allak, and Imade Benelallam. 2024. Darijabert: a step forward in nlp for the written moroccan dialect. International Journal of Data Science and Analytics, pages 1–13. ↗
  14. I. Guellil, A. Adeel, F. Azouaou, F. Benali, A. Hachani, and A. Hussain. 2020. Sentiment analysis of arabic dialects. Technical report, Ecole Superieure des Sciences Appliquées d’Alger, Laboratoire des Méthodes de Conception des Systèmes, Ecole nationale Supérieure d’Informatique, & University of Stirling. find
  15. Nizar Habash, Owen Rambow, and Ryan Roth. 2013. Morphological analysis and generation for arabic dialects: Challenges and solutions. Computational Linguistics, 39(3):453–495. find
  16. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735– 1780. ↗
  17. A. Khrisat and Z. Al-Harthy. 2015. Arabic dialects and classical arabic language. Advances in Social Sciences Research Journal, 2(3):254–260. ↗
  18. Y. Matrane, F. Benabbou, and N. Sael. 2023. A systematic literature review of arabic dialect sentiment analysis. Journal of King Saud University-Computer and Information Sciences, 35(6):101570. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alharbi-2025-ahasis-shared-task-sentiment-analysis/