Elamur Research⌕
☰
home / papers / 2025 / Dhati+: Fine-tuned Large Language Models for Arabic Subjectivity Evaluation

Dhati+: Fine-tuned Large Language Models for Arabic Subjectivity Evaluation

Summary

Arabic has few large annotated corpora for deciding whether a text states facts or expresses opinion. This paper extends the authors' earlier Dhati tool for Arabic subjectivity analysis. The original tweet corpus ASTD was combined with outside collections: LABR (book reviews), HARD (hotel reviews) and SANAD (news articles). Together they form AraDhati+, which supplies more subjective and objective examples. Three pretrained models were fine-tuned on this data for binary subjective/objective classification: multilingual XLM-RoBERTa, Arabic-specific AraBERT, and the generative ArabianGPT. Their outputs were then merged with an ensemble decision. The best configuration reached 97.79% accuracy. The authors also studied the errors the ensemble still makes and found that most come from texts mixing fact and opinion or from very short tweets. The work shows that a low-resource classification task can be improved substantially by assembling data from existing corpora and combining diverse transformer models, without building a new manual annotation campaign.

Key points

  • AraDhati+ is built from four existing Arabic resources: ASTD, LABR, HARD and SANAD.
  • The fine-tuned models are named AraSubjXLM-R, AraSubjBERT and AraSubjGPT, based on XLM-RoBERTa, AraBERT and ArabianGPT.
  • The ensemble decision achieved the top accuracy of 97.79%.
  • Results are reported separately per source dataset. The tweet part (ASTD) is clearly harder than the review and news parts.
  • Errors on which all three models agreed were sorted into three types:
    • mixed fact-and-opinion tweets, 41%;
    • unexplained model errors, 33%;
    • short tweets lacking context, 26%.

Interesting ideas

  • I1. A small, noisy target set (tweets) is enlarged by importing whole corpora whose genre strongly leans toward one class, such as reviews or news, instead of annotating new items. — For Mova (learn): silver training data for a rare category could be drawn from genres where that category dominates, with the genre bias checked by a held-out in-domain control.
  • I2. The ensemble mixes deliberately different model families (a multilingual encoder, a monolingual encoder and a decoder-only GPT), so that their mistakes overlap less. — For Mova (learn): when choosing witness parsers for silver agreement, preferring architecturally unlike witnesses makes their consensus more informative than agreement among near-copies.
  • I3. Errors where every ensemble member agrees on the wrong answer are treated as a separate diagnostic group, because they point to flaws in the data or the shared design rather than random noise. — For Mova (learn): unanimous-but-wrong silver cases could be routed to the LLM teacher tier or to repair-rule induction, since agreement alone would never flag them.
  • I4. The largest error class is texts that state a fact and then comment on it, which suggests a single label per sentence is too coarse. — For Mova (frames): speech-act tagging could label each clause as assertion or evaluation separately, so "X is natural, but its downside is Y" yields one factual and one evaluative act.
  • I5. Reporting scores per source corpus shows that the homogeneous review and news parts are close to trivially easy and could hide weak performance on tweets. — For Mova (answer): test results should be broken down by text source or genre, so easy subsets do not inflate the overall reading-comprehension score.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1007/s10579-026-09906-y · arXiv 2508.19966

@misc{bellaouar2025dhati,
  title = {{Dhati+: Fine-tuned Large Language Models for Arabic Subjectivity Evaluation}},
  author = {Bellaouar, Slimane and Nehar, Attia and Souffi, Soumia and Bouameur, Mounia},
  year = {2025},
  doi = {10.1007/s10579-026-09906-y},
  eprint = {2508.19966},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1007/s10579-026-09906-y}
}

References

31 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdul-Mageed, M., Diab, M., Korayem, M. (2011, June). Subjectivity and sentiment analysis of Modern Standard Arabic. D. Lin, Y. Matsumoto, & R. Mihalcea (Eds.), Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies (pp. 587–591). Portland, Oregon, USA: Association for Computational Linguistics. Retrieved from https://aclanthology.org/P11-2103 ↗
  2. Abdul-Mageed, M., Diab, M., Kübler, S. (2014). Samar: Subjectivity and sentiment analysis for arabic social media. Computer Speech & Language, 28 (1), 20–37, Alduailej, A., & Alothaim, A. (2022, May 31). Araxlnet: pre-trained language model for sentiment analysis of arabic. Journal of Big Data, 9 (1), 72, https://doi.org/10.1186/s40537-022-00625-z Retrieved from https://doi.org/10.1186/s40537-022-00625-z ↗
  3. Alharbi, A., Kalkatawi, M., Taileb, M. (2021, Sep 01). Arabic sentiment analysis using deep learning and ensemble methods. Arabian Journal for Science and Engineering, 46 (9), 8913-8923, https://doi.org/10.1007/s13369-021-05475-0 Retrieved from https://doi.org/10.1007/s13369-021-05475-0 ↗
  4. Alhumoud, S.O., & Al Wazrah, A.A. (2022, Jan 01). Arabic sentiment analysis using recurrent neural networks: a review. Artificial Intelligence Review , 55 (1), 707-748, https://doi.org/10.1007/s10462-021-09989-9 Retrieved from https://doi.org/10.1007/s10462-021-09989-9 ↗
  5. Aly, M., & Atiya, A. (2013, August). LABR: A large scale Arabic book reviews dataset. Proceedings of the 51st annual meeting of the association for computational linguistics (volume 2: Short papers) (pp. 494–498). Sofia, Bulgaria: Association for Computational Linguistics. Retrieved from https://aclanthology.org/P13-2088 ↗
  6. Antoun, W., Baly, F., Hajj, H. (2020). Arabert: Transformer-based model for arabic language understanding. arXiv preprint arXiv:2003.00104 , , Awwad, H., & Alpkocak, A. (2016). Performance comparison of different lexicons for sentiment analysis in arabic. 2016 third european network intelligence conference (enic) (p. 127-133). ↗ our card
  7. Chen, H., Sun, M., Tu, C., Lin, Y., Liu, Z. (2016). Neural sentiment classification with user and product attention. Conference on empirical methods in natural language processing. find
  8. Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., . . . Stoyanov, V. (2019). Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116 , , Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., . . . Stoyanov, V. (2020). Unsupervised Cross-lingual Representation Learning at Scale. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 8440–8451). Online: Association for Computational Linguistics. Retrieved 2023-01-16, from https://www.aclweb.org/anthology/2020.acl-main.747 ↗
  9. Conneau, A., Schwenk, H., Barrault, L., Lecun, Y. (2017, April). Very deep convolutional networks for text classification. Proceedings of the 15th conference of the European chapter of the association for computational linguistics: Volume 1, long papers (pp. 1107–1116). Valencia, Spain: Association for Computational Linguistics. Retrieved from https://aclanthology.org/E17-1104 ↗ our card
  10. Devlin, J., Chang, M.-W., Lee, K., Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 , , Duwairi, R., & El-Orfali, M. (2014). A study of the effects of preprocessing strategies on sentiment analysis for arabic text. Journal of Information Science, 40 (4), 501-513, https://doi.org/10.1177/0165551514534143 Retrieved from https://doi.org/10.1177/0165551514534143 ↗
  11. Einea, O., Elnagar, A., Al Debsi, R. (2019). Sanad: Single-label arabic news articles dataset for automatic text categorization. Data in brief , 25 , 104076, Elnagar, A., Lulu, L., Einea, O. (2018). An annotated huge dataset for standard and colloquial arabic reviews for subjective sentiment analysis. Procedia Computer Science, 142 , 182-189, https://doi.org/https://doi.org/10.1016/j.procs.2018.10.474 Retrieved from https://www.sciencedirect.com/science/article/pii/S1877050918321781 (Arabic Computational Linguistics) ↗
  12. Ghosal, D., Akhtar, M.S., Chauhan, D., Poria, S., Ekbal, A., Bhattacharyya, P. (2018, October-November). Contextual inter-modal attention for multi-modal sentiment analysis. Proceedings of the 2018 conference on empirical methods in natural language processing (pp. 3454–3466). Brussels, Belgium: Association for Computational Linguistics. Retrieved from https://aclanthology.org/D18-1382 ↗
  13. Giannakopoulos, A., Musat, C., Hossmann, A., Baeriswyl, M. (2017, September). Unsupervised aspect term extraction with B-LSTM & CRF using automatically labelled datasets. Proceedings of the 8th workshop on computational approaches to subjectivity, sentiment and social media analysis (pp. 180–188). Copenhagen, Denmark: Association for Computational Linguistics. Retrieved from https://aclanthology.org/W17-5224 ↗
  14. Habimana, O., Li, Y., Li, R., Gu, X., Yu, G.X. (2019). Sentiment analysis using deep learning approaches: an overview. Science China Information Sciences, 63 , , Johnson, R., & Zhang, T. (2017, July). Deep pyramid convolutional neural networks for text categorization. Proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: Long papers) (pp. 562–570). Vancouver, Canada: Association for Computational Linguistics. Retrieved from https://aclanthology.org/P17-1052 Koubaa, A., Ammar, A., Ghouti, L., Najar, O., Sibaee, S. biangpt: Native arabic gpt-based large language model. https://arxiv.org/abs/2402.15313 ↗ our card
  15. Kouloumpis, E., Wilson, T., Moore, J. (2021, Aug.). Twitter sentiment analysis: The good the bad and the omg! Proceedings of the International AAAI Conference on Web and Social Media, 5 (1), 538-541, https://doi.org/10.1609/icwsm.v5i1.14185 Retrieved from https://ojs.aaai.org/index.php/ICWSM/article/view/14185 ↗
  16. Liu, B. (2012, May). Sentiment analysis and opinion mining. Synthesis Lectures on Human Language Technologies, 5 (1), 1–167, https://doi.org/10.2200/s00416ed1v01y201204hlt016 Retrieved from http://dx.doi.org/10.2200/S00416ED1V01Y201204HLT016 ↗
  17. Loshchilov, I., & Hutter, F. (2019). Decoupled weight decay regularization. International conference on learning representations. Retrieved from https://openreview.net/forum?id=Bkg6RiCqY7 ↗
  18. Maamouri, M., Bies, A., Buckwalter, T., Mekki, W. (2004). The penn arabic treebank: Building a large-scale annotated arabic corpus.. Retrieved from https://api.semanticscholar.org/CorpusID:16205731 ↗
  19. Mohamed, O., Kassem, A.M., Ashraf, A., Jamal, S., Mohamed, E.H. (2022, Dec 24). An ensemble transformer-based model for arabic sentiment analysis. Social Network Analysis and Mining, 13 (1), 11, https://doi.org/10.1007/s13278-022 -01009-0 Retrieved from https://doi.org/10.1007/s13278-022-01009-0 ↗
  20. Nabil, M., Aly, M., Atiya, A. (2015, September). ASTD: Arabic sentiment tweets dataset. Proceedings of the 2015 conference on empirical methods in natural language processing (pp. 2515–2519). Lisbon, Portugal: Association for Computational Linguistics. Retrieved from https://aclanthology.org/D15-1299 ↗ our card
  21. (2024). Ara-Retrieved from Nehar, A., Bellaouar, S., Souffi, S., Bouameur, M. (2023). Dhati: a fine-tuned large language model for evaluating subjectivity in arabic textual data. 2023 5th International Conference on Pattern Analysis and Intelligent Systems (PAIS) (p. 1-7). find
  22. Oueslati, O., Cambria, E., HajHmida, M.B., Ounelli, H. (2020). A review of sentiment analysis research in arabic language. Future Generation Computer Systems, 112 , 408-430, https://doi.org/https://doi.org/10.1016/j.future.2020.05.034 Retrieved from https://www.sciencedirect.com/science/article/pii/S0167739X19311537 ↗ our card
  23. Pang, B., & Lee, L. (2008, jan). Opinion mining and sentiment analysis. Found. Trends Inf. Retr., 2 (1–2), 1–135, https://doi.org/10.1561/1500000011 Retrieved from https://doi.org/10.1561/1500000011 ↗
  24. Pang, B., Lee, L., Vaithyanathan, S. (2002, July). Thumbs up? sentiment classification using machine learning techniques. Proceedings of the 2002 conference on empirical methods in natural language processing (EMNLP 2002) (pp. 79–86). Association for Computational Linguistics. Retrieved from https://aclanthology.org/W02-1011 ↗ our card
  25. Rushdi-Saleh, M., Martı́n-Valdivia, M.T., Ureña López, L.A., Perea-Ortega, J.M. (2011, oct). Oca: Opinion corpus for arabic. J. Am. Soc. Inf. Sci. Technol., 62 (10), 2045–2054, https://doi.org/10.1002/asi.21598 Retrieved from https://doi.org/10.1002/asi.21598 S. ↗
  26. Alotaibi, S. (2016, Apr.). Sentiment analysis in arabic: An overview. International Journal of Sciences: Basic and Applied Research (IJSBAR), 26 (2), 147 à 165, Retrieved from https://gssrr.org/index.php/JournalOfBasicAndApplied/article/view/5521 ↗
  27. Tang, D., Qin, B., Liu, T. (2015, July). Learning semantic representations of users and products for document level sentiment classification. C. Zong & M. Strube (Eds.), Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 1: Long papers) (pp. 1014–1023). Beijing, China: Association for Computational Linguistics. Retrieved from https://aclanthology.org/P15-1098 ↗
  28. Wang, B., Spencer, B., Ling, C.X., Zhang, H. (2008). Semi-supervised self-training for sentence subjectivity classification. Proceedings of the canadian society for computational studies of intelligence, 21st conference on advances in artificial intelligence (p. 344–355). Berlin, Heidelberg: Springer-Verlag. find
  29. Wang, W., Pan, S.J., Dahlmeier, D., Xiao, X. (2017, Feb.). Coupled multi-layer attentions for co-extraction of aspect and opinion terms. Proceedings of the AAAI Conference on Artificial Intelligence, 31 (1), , https://doi.org/10.1609/aaai.v31i1.10974 Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/10974 ↗
  30. Wu, Y., Zhang, Q., Huang, X., Wu, L. (2009). Phrase dependency parsing for opinion mining. Proceedings of the 2009 conference on empirical methods in natural language processing: Volume 3 - volume 3 (p. 1533–1541). USA: Association for Computational Linguistics. find
  31. Zhang, X., Zhao, J., LeCun, Y. (2015). Character-level convolutional networks for text classification. C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, & R. Garnett (Eds.), Advances in neural information processing systems (Vol. 28). Curran Associates, Inc. Retrieved from https://proceedings.neurips.cc/paper files/paper/2015/ ↗ our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bellaouar-2025-dhati-fine-tuned-large-language/