Elamur Research⌕
☰
home / papers / 2025 / Fine-tuning AraBert model for arabic sentiment detection

Fine-tuning AraBert model for arabic sentiment detection

Paper · 2025 · Mustapha Jaballah, Dhaou Ghoul and Ammar Mars

Summary

The paper describes a system for the Ahasis Shared Task, which asks for three-way sentiment classification (positive, negative, neutral) of hotel reviews written in several Arabic dialects, such as Darija and Saudi. Arabic is hard for this task because Modern Standard Arabic coexists with regional varieties that differ in vocabulary, syntax and idiom, and because sentiment is often carried by colloquial phrasing. The authors fine-tuned the Twitter variant of AraBERT (aubmindlab/bert-base-arabertv02) on the Ahasis training data. To handle class imbalance, they used frequency-derived class weights, gave the scarce neutral class an extra boost, and fed these weights into the training loss. The model scored 0.93 F1 on validation and 0.88 on an internal test split, and it achieved 79.16% on the organizers' blind evaluation data, ranking second. Their error analysis found two main problems: a bias toward predicting neutral and difficulty with sarcasm. It also found apparently wrong gold labels in the training data. The work shows how far a well-tuned dialect-aware encoder can go, and where its weaknesses lie.

Key points

  • Base model: aubmindlab/bert-base-arabertv02 (Twitter variant), fine-tuned on the Ahasis hotel-review dataset, which is labeled for both sentiment and dialect.
  • Class imbalance handling: weights are computed from class frequency, the neutral class weight is then doubled, and the weights are applied in the loss.
  • Scores: peak validation F1 of 0.93 at epoch 11, 0.88 on the internal test split, and 79.16% on the hidden test set, earning second place.
  • Error analysis found positive reviews predicted as neutral with very high confidence (e.g., 0.99), as well as missed irony and sarcasm.
  • Table 5 lists training items the authors believe are mislabeled, such as "the breakfast buffet is good" (Darija) labeled neutral and a complaint about a cramped room labeled positive.

Interesting ideas

  • I1. Class weights are set in two layers: an automatic inverse-frequency base, then a manual extra multiplier for the one class known to be neglected; this lets rare categories be protected without hand-tuning every weight.
  • I2. Performance is reported on three levels (validation, internal held-out split, official blind test), and the drop between levels is treated as a diagnostic of overfitting and distribution shift rather than hidden.
  • I3. The authors look at the model's confidence on its wrong answers; near-certain neutral predictions for positive texts exposed a systematic bias rather than random noise.
  • I4. Error analysis is turned into a label audit: suspicious gold labels are collected with a proposed correction, which treats the dataset itself as fallible.
  • I5. The authors propose a curated dialect sentiment lexicon used to adjust classifier confidence, either as extra input features or as a post-processing step, which brings in explicit, inspectable evidence.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): When training the tagger or parser on silver data, rare relations or tags could get frequency-based loss weights plus a targeted boost for labels that the error analysis shows are systematically missed.
  • I2 (learn): Evaluation reports could always show the gap between in-distribution dev, a held-out internal split, and a truly unseen text set, so that shrinking or growing gaps become a tracked signal.
  • I3 (answer): Students answering school tests or fable questions could log the confidence on failed items, so that confidently wrong answers are flagged as structural gaps worth repair rules.
  • I4 (learn): When witness parsers agree against a treebank or silver label, the item could be queued as a possible annotation error with a proposed correction, instead of being counted only as a model mistake.
  • I5 (frames): Polarity entries tied to lemmas and frames could adjust or explain speech-act and irony judgments in prag, so that every evaluative decision points to a concrete lexical source.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tunis UniversityTunis · Tunisia4 works · 28 citations

Also on the byline: ENSIT.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 1,035 worksSemitic languages · 3,066 works

Cite

Cite as Mustapha Jaballah, Dhaou Ghoul and Ammar Mars. 2025. Fine-tuning AraBert model for arabic sentiment detection. Proceedings of the Shared Task on Sentiment Analysis for Arabic Dialects.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{jaballah2025fine,
  title = {{Fine-tuning AraBert model for arabic sentiment detection}},
  author = {Mustapha Jaballah, Dhaou Ghoul and Ammar Mars},
  booktitle = {Proceedings of the Shared Task on Sentiment Analysis for Arabic Dialects},
  year = {2025},
  doi = {10.26615/978-954-452-109-7-003},
  url = {https://doi.org/10.26615/978-954-452-109-7-003}
}

References

31 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Jalel Akaichi. 2013. Social networks’ facebook’ statutes updates mining for sentiment classification. In 2013 International Conference on Social Computing, pages 886–891. find
  2. Sadam Al-Azani and El-Sayed M. El-Alfy. 2017. Hybrid deep learning for sentiment polarity determination of arabic microblogs. In Neural Information Processing, pages 491–500, Cham. Springer International Publishing. find
  3. Ramy Baly, Georges El-Khoury, Rawan Moukalled, Rita Aoun, Hazem Hajj, Khaled Bashir Shaban, and Wassim El-Hajj. 2017. Comparative evaluation of sentiment analysis methods across arabic dialects. Procedia Computer Science, 117:266–273. Arabic Computational Linguistics. find
  4. Abdulmohsen Al-Thubaity, Sakhar Alkhereyf, Hanan Murayshid, Nouf Alshalawi, Maha Omirah, Raghad Alateeq, Rawabi Almutairi, Razan Alsuwailem, Manal Alhassoun, and Imaan Alkhanen. 2023. Evaluating ChatGPT and bard AI on Arabic sentiment analysis. In Proceedings of ArabicNLP 2023, pages 335–349, Singapore (Hybrid). Association for Computational Linguistics.
  5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019a. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  6. Abdulaziz M. Alayba, Vasile Palade, Matthew England, and Rahat Iqbal. 2018. A combined cnn and lstm model for arabic sentiment analysis. In Machine Learning and Knowledge Extraction, pages 179–191, Cham. Springer International Publishing.
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019b. Bert: Pre-training of deep bidirectional transformers for language understanding.
  8. Maram Alharbi, Salmane Chafik, Saad Ezzini, Ruslan Mitkov, Tharindu Ranasinghe, and Hansi Hettiarachchi. 2025a. Ahasis: Shared task on sentiment analysis for arabic dialects. In Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing (RANLP).
  9. Ghoul Dhaou and Gaël Lejeune. 2020. Comparison between voting classifier and deep learning methods for Arabic dialect identification. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 243–249, Barcelona, Spain (Online). Association for Computational Linguistics.
  10. Maram Alharbi, Saad Ezzini, Hansi Hettiarachchi, Tharindu Ranasinghe, and Ruslan Mitkov. 2025b. Evaluating large language models on arabic dialect sentiment analysis. In Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing (RANLP). find
  11. AbdelRahim Elmadany, Muhammad Abdul-Mageed, et al. 2022. Arat5: Text-to-text transformers for arabic language generation. In Proceedings of the 60th annual meeting of the association for computational linguistics (Volume 1: Long papers), pages 628–647.
  12. Sana Alowaidi, Mustafa Saleh, and Osama Abulnaja. 2017. Semantic sentiment analysis of arabic texts. International Journal of Advanced Computer Science and Applications, 8(2). ↗
  13. Ashraf Elnagar, Yasmin S. Khalifa, and Anas Einea. 2018. Hotel Arabic-Reviews Dataset Construction for Sentiment Analysis Applications, pages 35–52. Springer International Publishing, Cham. find
  14. A. Aziz Altowayan and Lixin Tao. 2016. Word embeddings for arabic sentiment analysis. In 2016 IEEE International Conference on Big Data (Big Data), pages 3820–3825. find
  15. Dhaou Ghoul, Jérémy Patrix, Gaël Lejeune, and Jérôme Verny. 2024. A combined arabert and voting ensemble classifier model for arabic sentiment analysis. Natural Language Processing Journal, 8:100100. ↗
  16. Wissam Antoun, Fady Baly, and Hazem Hajj. Arabert: Transformer-based model for arabic language understanding. In LREC 2020 Workshop Language Resources and Evaluation Conference 11–16 May 2020, page 9. ↗
  17. Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2012. Improving neural networks by preventing co-adaptation of feature detectors. ArXiv, abs/1207.0580. ↗
  18. Huang Huang, Fei Yu, Jianqing Zhu, Xuening Sun, Hao Cheng, Song Dingjie, Zhihong Chen, Mosen Alharthi, Bang An, Juncai He, Ziche Liu, Junying Chen, Jianquan Li, Benyou Wang, Lian Zhang, Ruoyu Sun, Xiang Wan, Haizhou Li, and Jinchao Xu. 2024. AceGPT, localizing large language models in Arabic. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8139–8163, Mexico City, Mexico. Association for Computational Linguistics. ↗
  19. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. ↗
  20. Wissam Antoun, Fady Baly, and Hazem Hajj. 2021. AraGPT2: Pre-trained transformer for Arabic language generation. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 196–207, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. ↗
  21. Karima Kadaoui, Samar Magdy, Abdul Waheed, Md Tawkat Islam Khondaker, Ahmed El-Shangiti, El Moatez Billah Nagoudi, and Muhammad Abdul-Mageed. 2023. TARJAMAT: Evaluation of bard and ChatGPT on machine translation of ten Arabic varieties. In Proceedings of ArabicNLP 2023, pages 52–75, Singapore (Hybrid). Association for Computational Linguistics.
  22. Hussein Al-Natsheh. 2021. ALUE: Arabic language understanding evaluation. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 173–184, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  23. Sachin Kumar, Shuly Wintner, Noah A. Smith, and Yulia Tsvetkov. 2021. Topics to avoid: Demoting latent confounds in text classification. find
  24. Quoc Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machine Learning Research, pages 1188–1196, Bejing, China. PMLR. ↗
  25. Ammar Mars, Mohamed Salah Gouider, and Lam-jed Ben Saïd. 2015. A new big data framework for customer opinions polarity extraction. In International Conference: Beyond Databases, Architectures and Structures, pages 518–531. Springer. find
  26. Ammar Mars, Sihem Hamem, and Mohamed Salah Gouider. 2017. New ontological approach for opinion polarity extraction from twitter. In Computational Collective Intelligence: 9th International Conference, ICCCI 2017, Nicosia, Cyprus, September 27-29, 2017, Proceedings, Part II 9, pages 448–458. Springer. find
  27. Ammar Mars, Mustapha Jaballah, and Dhaou Ghoul. 2024. Ishfmg_tun at stanceeval: Ensemble method for arabic stance evaluation system. In Proceedings of The Second Arabic Natural Language Processing Conference, ArabicNLP 2024, Bangkok, Thailand, August 16, 2024, pages 832–836. Association for Computational Linguistics.
  28. M’hamed Mataoui, Tadj Eddine Bendali Hacine, Imad Tellache, Abdelghani Bakhtouchi, and Omar Zelmati. 2018. A new syntax-based aspect detection approach for sentiment analysis in arabic reviews. In 2018 2nd International Conference on Natural Language and Speech Processing (ICNLSP), pages 1–6. find
  29. Md Saef Ullah Miah, Md Mohsin Kabir, Talha Bin Sarwar, Mejdl Safran, Sultan Alfarhood, and MF Mridha. 2024. A multimodal approach to cross-lingual sentiment analysis with ensemble of transformer and llm. Scientific Reports, 14(1):9603. ↗
  30. Soukaina MIHI, Brahim AIT BEN ALI, Ismail EL BAZI, Sara AREZKI, and Nabil LAACHFOUBI. 2020. Mstd: Moroccan sentiment twitter dataset. International Journal of Advanced Computer Science and Applications, 11(10). ↗
  31. Alec Radford and Karthik Narasimhan. 2018. Improving language understanding by generative pre-training. Haitham Seelawi, Ibraheem Tuffaha, Mahmoud Gzawi, Wael Farhan, Bashar Talafha, Riham Badawi, Zyad Sober, Oday Al-Dweik, Abed Alhakim Freihat, and find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/jaballah-2025-fine-tuning-arabert-model-arabic/