Elamur Research⌕
☰
home / papers / 2025 / Arabic-Centric Large Language Models for Dialectal Arabic Sentiment Analysis Task

Arabic-Centric Large Language Models for Dialectal Arabic Sentiment Analysis Task

Paper · 2025 · Salwa Saad Alahmari, Eric Atwell, Hadeel Saadany et al.

Summary

Sentiment classification for Arabic dialects is difficult. Spelling is not standardized, vocabulary shifts from region to region, speakers switch between dialect and MSA, and labelled data is scarce. For the AHaSIS shared task on hotel reviews in Saudi and Moroccan (Darija) dialects, the authors did three things:

  • New resource. They built the Saudi Proverbs Dataset (SPD), 698 proverbs from five Saudi dialect regions, each labelled Positive, Negative or Neutral by three native annotators.
  • Fine-tuning. They trained the CAMeLBERT-DA-SA model on the official review training set plus the proverbs.
  • Zero-shot comparison. They tested three open Arabic-centric LLMs (ALLaM, ACeGPT, Jais) on the same test set without any task training.

The fine-tuned BERT model scored best at 75% F1. The zero-shot LLMs scored between 65% and 70%. The authors read this as evidence that adapting to the domain and dialect matters more than model size. The work matters because it adds a culturally grounded, freely available dialect resource and offers a simple reference point: a small specialist against larger generalists in a low-resource setting.

Key points

  • SPD sources and coverage: a 1983 printed proverb collection by Abdulkareem Aljuhaiman, interviews with elderly speakers, and online forums or social media. It spans the Najdi, Hijazi, Shamali, Janoubi and Sharqawi sub-dialects and is available on GitHub.
  • SPD labels: inter-annotator agreement was Fleiss' Kappa = 0.85. The labels are heavily skewed, with 511 Negative, 99 Positive and 88 Neutral proverbs.
  • AHaSIS data: the training set has 860 hotel reviews (430 Saudi, 430 Darija). The test set has 216 reviews (108 per dialect). Together with SPD, training used 1558 sentences.
  • Fine-tuning settings: CAMeLBERT-DA-SA was trained with learning rate 5e-5, max_length 512, batch size 8 and 2 epochs. Hyperparameters were not explored further because of limited compute.
  • F1 results on the test set:
ModelSettingF1
CAMeLBERT-DA-SAfine-tuned75%
ALLaM-7B-Instruct-previewzero-shot70%
AceGPTv2-8Bzero-shot68%
jais-family-13bzero-shot65%

Interesting ideas

  • I1. Proverbs are short, culturally loaded texts whose polarity often differs from the literal words, which makes them a hard test of sentiment understanding; the authors gathered some orally to capture sayings that are rarely written down.
  • I2. Labels come from three independent annotators, and Fleiss' Kappa is reported before the data is used, so label quality is measured rather than assumed.
  • I3. A modest fine-tuned encoder beat 7–13B instruction models used zero-shot, which suggests targeted adaptation can outweigh scale on narrow, low-resource tasks.
  • I4. Normalization was deliberately light: diacritics were removed, but dialect spellings were kept, so variation stays visible instead of being erased.
  • I5. An out-of-domain auxiliary corpus (proverbs) was mixed into in-domain review training, but its separate contribution was never isolated, which shows why such additions need a with/without comparison.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): an English set of proverbs and idioms labelled for polarity and intended meaning would stress-test the implicature and irony detectors in prag, where literal frames give the wrong reading.
  • I2 (learn): Fleiss' Kappa over several witness parsers or LLM teacher tiers could set a numeric acceptance threshold for silver sentences, so low-agreement items are routed to review instead of training.
  • I3 (learn): this supports keeping LLMs as offline teachers and running small, trained, explainable components per stage at inference time.
  • I4 (tokens): the en tokenizer could strip only true noise and map informal or variant spellings to lemmas in the tagger, keeping the surface form for explanation.
  • I5 (learn): every auxiliary corpus added to parser or frame training should come with an ablation run as a negative control, so its effect is shown rather than assumed.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of LeedsLeeds · United Kingdom42 works · 1,823 citations🎓Birmingham City UniversityBirmingham · United Kingdom6 works · 26 citations

Also on the byline: University of Hafr Al-Batin.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 3Research subjects and languages the work is about.Language models & pretraining · 3,894 worksSentiment & emotion · 1,035 worksSemitic languages · 3,066 works

Cite

Cite as Salwa Saad Alahmari, Eric Atwell, Hadeel Saadany and Mohammad Alsalka. 2025. Arabic-Centric Large Language Models for Dialectal Arabic Sentiment Analysis Task. Proceedings of the Shared Task on Sentiment Analysis for Arabic Dialects.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{alahmari2025arabic,
  title = {{Arabic-Centric Large Language Models for Dialectal Arabic Sentiment Analysis Task}},
  author = {Salwa Saad Alahmari, Eric Atwell, Hadeel Saadany and Mohammad Alsalka},
  booktitle = {Proceedings of the Shared Task on Sentiment Analysis for Arabic Dialects},
  year = {2025},
  doi = {10.26615/978-954-452-109-7-011},
  url = {https://doi.org/10.26615/978-954-452-109-7-011}
}

References

23 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ibrahim Abu Farha, Wajdi Zaghouani, and Walid Magdy. 2021. Overview of the wanlp 2021 shared task on sarcasm and sentiment detection in arabic. In Proceedings of the Sixth Arabic Natural Language Processing Workshop.
  2. Latifa Al-Sulaiti and Eric Atwell. 2006. The design of a corpus of contemporary arabic. International Journal of Corpus Linguistics, 11(2):135–171. ↗
  3. Salwa Alahmari. 2025. SADSLyC: A corpus for saudi Arabian multi-dialect identification through song lyrics. In Proceedings of the 4th Workshop on Arabic Corpus Linguistics (WACL-4), pages 38–43, Abu Dhabi, UAE. Association for Computational Linguistics. find
  4. Kamel Gaanoun and Imade Benelallam. 2021. Sarcasm and sentiment detection in Arabic language a hybrid approach combining embeddings and rule-based features. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 351–356, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  5. Salwa Alahmari, Eric Atwell, and Mohammad r Alsalka. 2024. Saudi arabic multi-dialects identification in social media texts. In Intelligent Computing, pages 209–217, Cham. Springer Nature Switzerland. find
  6. Imane Guellil, Houda Saâdane, Faical Azouaou, Billel Gueni, and Damien Nouvel. 2021. Arabic natural language processing: An overview. Journal of King Saud University - Computer and Information Sciences, 33(5):497–507.
  7. Basma Alharbi, Hind Alamro, Manal Alshehri, Zuhair Khayyat, Manal Kalkatawi, Inji Ibrahim Jaber, and Xiangliang Zhang. 2021. Asad: A twitter-based benchmark arabic sentiment analysis dataset. ↗
  8. Inception. 2024. Jais family model card. find
  9. Maram Alharbi, Salmane Chafik, Saad Ezzini, Ruslan Mitkov, Tharindu Ranasinghe, and Hansi Hettiarachchi. 2025a. Ahasis: Shared task on sentiment analysis for arabic dialects. In Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing (RANLP).
  10. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. The interplay of variant, size, and task type in Arabic pre-trained language models. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, Kyiv, Ukraine (Online). Association for Computational Linguistics.
  11. Maram Alharbi, Saad Ezzini, Hansi Hettiarachchi, Tharindu Ranasinghe, and Ruslan Mitkov. 2025b. Evaluating large language models on arabic dialect sentiment analysis. In Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing (RANLP). find
  12. Ikram El Karfi and Sanaa El Fkihi. 2022. An ensemble of arabic transformer-based models for arabic sentiment analysis. International Journal of Advanced Computer Science and Applications, 13(8). ↗
  13. Wael Alosaimi, Hager Saleh, Ali A. Hamzah, Nora El-Rashidy, Abdullah Alharb, Ahmed Elaraby, and Sherif Mostafa. 2024. Arabbert-lstm: improving arabic sentiment analysis based on transformer model and long short-term memory. Frontiers in Artificial Intelligence, Volume 7 - 2024. ↗
  14. Md Tawkat Islam Khondaker, Abdul Waheed, El Moatez Billah Nagoudi, and Muhammad Abdul-Mageed. 2023. GPTAraEval: A comprehensive evaluation of ChatGPT on Arabic NLP. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 220–247, Singapore. Association for Computational Linguistics.
  15. Sarah N. Alyami and Sunday O. Olatunji. 2020. Application of support vector machine for arabic sentiment classification using twitter-based dataset. Journal of Information & Knowledge Management, 19(01):2040018. ↗
  16. Juhao Liang, Zhenyang Cai, Jianqing Zhu, Huang Huang, Kewei Zong, Bang An, Mosen Alharthi, Juncai He, Lian Zhang, Haizhou Li, Benyou Wang, and Jinchao Xu. 2024. Alignment at pre-training! towards native alignment for arabic LLMs. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. M Saiful Bari, Yazeed Alnumay, Norah A. Alzahrani, Nouf M. Alotaibi, Hisham Abdullah Alyahya, Sultan AlRashed, Faisal Abdulrahman Mirza, Shaykhah Z. Alsubaie, Hassan A. Alahmed, Ghadah Alabduljabbar, Raghad Alkhathran, Yousef Almushayqih, Raneem Alnajim, Salman Alsubaihi, Maryam Al Mansour, Saad Amin Hassan, Dr.
  17. Majed Alrubaian, Ali Alammari, Zaki Alawami, Abdulmohsen Al-Thubaity, Ahmed Abdelali, Jeril Kuriakose, Abdalghani Abujabal, Nora Al-Twairesh, Areeb Alowisheq, and Haidar Khan. 2025. ALLam: Large language models for arabic and english. In The Thirteenth International Conference on Learning Representations.
  18. Omar Mohamed, Aly M Kassem, Ali Ashraf, Salma Jamal, and Ensaf Hussein Mohamed. 2022. An ensemble transformer-based model for arabic sentiment analysis. Social Network Analysis and Mining, 13(1):11. ↗
  19. Sara Rosenthal, Noura Farra, and Preslav Nakov. 2017. SemEval-2017 task 4: Sentiment analysis in Twitter. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 502– 518, Vancouver, Canada. Association for Computational Linguistics.
  20. Ali Farghaly and Khaled Shaalan. 2009. Arabic natural language processing: Challenges and solutions. ACM Transactions on Asian Language Information Processing, 8(4). ↗
  21. Anshul Wadhawan. 2021. AraBERT and farasa segmentation based approach for sarcasm and sentiment detection in Arabic tweets. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 395–400, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. ↗
  22. Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378. ↗
  23. Jianqing Zhu, Huang Huang, Zhihang Lin, Juhao Liang, Zhengyang Tang, Khalid Almubarak, Abdulmohsen Alharthik, Bang An, Juncai He, Xiangbo Wu, Fei Yu, Junying Chen, Zhuoheng Ma, Yuhao Du, He Zhang, Emad Alghamdi, Lian Zhang, Ruoyu Sun, Haizhou Li, and Jinchao Xu. 2024. Second language (arabic) acquisition of llms via progressive vocabulary expansion. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alahmari-2025-arabic-centric-large-language-models/