Elamur Research⌕
☰
home / papers / 2025 / Fine-Grained Arabic Offensive Language Classification with Taxonomy, Sentiment, and Emotions

Fine-Grained Arabic Offensive Language Classification with Taxonomy, Sentiment, and Emotions

Paper · 2025 · Natalia Vanetik, Marina Litvak and Chaya Liebeskind

Summary

Detecting offensive Arabic text is hard. Existing resources also tend to stop at a coarse yes/no label, which hides who is targeted, how severe the offense is, and what kind it is. The authors start from FARAD-500, a 500-text collection of offensive posts in Modern Standard Arabic and Levantine. It was assembled from 16 earlier datasets and only partly fits the seven-level ArSOL taxonomy. Native speakers re-annotated it under tighter rules: they fixed mislabels, filled every taxonomy level, allowed several labels per text, and limited the catch-all "Other" category. The result, reFarad-500, covers the explicit levels 1–6.

The authors then trained classifiers per level and per category, comparing:

  • n-gram features with arBERT and multilingual BERT sentence embeddings,
  • LR, RF and XGB classifiers,
  • fine-tuned BERT models,
  • runs with and without added sentiment and emotion features.

Fine-tuned transformers beat traditional classifiers. Sentiment and emotion features raised scores for several categories. The released dataset and guidelines give low-resource Arabic NLP a reusable, fine-grained benchmark.

Key points

  • reFarad-500 contains 500 offensive texts from Facebook and Twitter, drawn from 16 prior Arabic datasets. It covers MSA and Levantine and is freely available on GitHub with its annotation guidelines.
  • The ArSOL taxonomy has seven levels:
    • levels 1–6 are explicit: offensiveness, target, target presence, vulgarity, offense strength and offense aspect;
    • level 7 covers implicit offense and was left for future work.
  • At level 5, each text is first labeled Hate speech or Insult, then separately marked for Threat and Discredit. This makes the level multi-label.
  • Adding sentiment raised several F1 scores:
    • target: from 0.4887 to 0.5802;
    • hate speech/insult: from 0.7029 to 0.7220.
  • Adding emotions also helped some categories:
    • target presence: from 0.6755 to 0.7137;
    • religion: from 0.6094 to 0.6842.
  • Many other categories were unchanged. Ableism and classism stayed near F1 0.45–0.47.

Interesting ideas

  • I1. Re-annotate an existing noisy corpus under explicit rules instead of collecting new data; a small but clean set can be worth more than a large inconsistent one.
  • I2. Allow "Other" only when no specific label fits; this keeps the vague bin from absorbing cases that belong to real categories.
  • I3. Split one overloaded decision into a primary exclusive choice plus independent binary flags (Hate speech vs Insult, then Threat and Discredit separately); one text can then carry several functions at once.
  • I4. Report results per taxonomy level and per category, choosing the best representation and classifier for each one; this shows which distinctions are easy and which are data-starved.
  • I5. Use sentiment and emotion as auxiliary signals; they help only some categories, so test each addition per category rather than assuming a global gain.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Silver treebanks could get a targeted correction pass with written rules for the recurring error types, rather than being regenerated from scratch.
  • I2 (frames): Semantic-role and speech-act labelling could forbid a fallback role whenever any VerbNet role is applicable, which would keep frames informative.
  • I3 (frames): Speech-act tagging could pick one main act and add independent flags such as threat or irony, rather than forcing a single label.
  • I4 (learn): Evaluation could be broken down by dependency relation or frame type, so weak categories are visible rather than averaged away.
  • I5 (world): Character states in tales, such as anger or fear, could be stored as explicit world facts and used as cues for interpreting hostile or ironic utterances.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Jerusalem College of TechnologyJerusalem · Israel14 works · 147 citations🎓Sami Shamoon College of EngineeringBeersheba · Israel6 works · 10 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksSocial Sciences · 1,954 worksCommunication · 104 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 1,035 worksSemitic languages · 3,066 works

Cite

Cite as Natalia Vanetik, Marina Litvak and Chaya Liebeskind. 2025. Fine-Grained Arabic Offensive Language Classification with Taxonomy, Sentiment, and Emotions. Proceedings of the Workshop on Beyond English: Natural Language Processing for all Languages in an Era of Large Language Models.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{vanetik2025fine,
  title = {{Fine-Grained Arabic Offensive Language Classification with Taxonomy, Sentiment, and Emotions}},
  author = {Natalia Vanetik, Marina Litvak and Chaya Liebeskind},
  booktitle = {Proceedings of the Workshop on Beyond English: Natural Language Processing for all Languages in an Era of Large Language Models},
  year = {2025},
  doi = {10.26615/978-954-452-105-9-013},
  url = {https://doi.org/10.26615/978-954-452-105-9-013}
}

References

43 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ashraf Ahmad, Mohammad Azzeh, Eman Elnagi, Qasem Abu Al-Haija, Dana Halabi, Abdullah Aref, and Yousef AbuHour. 2024. Hate Speech Detection in the Arabic Language: Corpus Design, Construction and Evaluation. Frontiers in Artificial Intelligence, 7:1345445. find
  2. Azalden Alakrot, Liam Murray, and Nikola S Nikolov. 2018. Dataset construction for the detection of antisocial behaviour in online communication in Arabic. Procedia Computer Science, 142:174–181. find
  3. Nuha Albadi, Maram Kurdi, and Shivakant Mishra. 2018. Are they our brothers? analysis and detection of religious hate speech in the Arabic twittersphere. In 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pages 69–76. IEEE. find
  4. Ali Alhazmi. 2023. Hate Speech Dataset for the Saudi Dialect. Mendeley Data. find
  5. Amer J Almahdawi and William J Teahan. 2019. A new arabic dataset for emotion recognition. In Intelligent Computing-Proceedings of the Computing Conference, pages 200–216. Springer. find
  6. Maha Jarallah Althobaiti. 2022. BERT-based approach to Arabic hate speech and offensive language detection in Twitter: exploiting emojis and sentiment analysis. International Journal of Advanced Computer Science and Applications, 13(5). find
  7. Mohamed Aly and Amir Atiya. 2013. LABR: A large scale arabic book reviews dataset. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 494–498, Sofia, Bulgaria. Association for Computational Linguistics.
  8. Amr Al-Khatib and Samhaa R. El-Beltagy. 2018. Emotional Tone Detection in Arabic Tweets, volume 10762 of Lecture Notes in Computer Science, page 105–114. Springer, Cham. find
  9. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. In LREC 2020 Workshop Language Resources and Evaluation Conference 11–16 May 2020, page 9.
  10. Flor Miriam Plaza-del Arco, Sercan Halat, Sebastian Padó, and Roman Klinger. 2021. Multi-task learning with sentiment, emotion, and target detection to recognize hate speech and offensive language. arXiv preprint arXiv:2109.10255. ↗
  11. Abdullah Aref, Rana Husni Al Mahmoud, Khaled Taha, Mahmoud Al-Sharif, et al. 2020. Hate speech detection of arabic shorttext. In 9th International Conference on Information Technology Convergence and Services (ITCSE 2020), pages 81–94. Computer Science & Information Technology. find
  12. Tianqi Chen. 2015. Xgboost: extreme gradient boosting. R package version 0.4-2, 1(4):1–4. find
  13. Shammur Absar Chowdhury, Hamdy Mubarak, Ahmed Abdelali, Soon-gyo Jung, Bernard J Jansen, and Joni Salminen. 2020. A multi-platform Arabic news comment dataset for offensive language detection. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 6203–6212.
  14. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1):37–46. ↗
  15. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171– 4186. Association for Computational Linguistics.
  16. Abdelrahim Elmadany, Chiyu Zhang, Muhammad Abdul-Mageed, and Azadeh Hashemi. 2020. Leveraging affective bidirectional transformers for offensive language detection. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 102–108.
  17. Kabil Essefar, Hassan Ait Baha, Abdelkader El Mahdaouy, Abdellah El Mekki, and Ismail Berrada. 2023. OMCD: Offensive Moroccan Comments Dataset. Language Resources and Evaluation, pages 1–21. find
  18. Hatem Haddad, Hala Mulki, and Asma Oueslati. 2019. T-HSAB: A Tunisian hate speech and abusive dataset. In International Conference on Arabic Language Processing, pages 251–263. Springer. find
  19. Go Inoue, Hassan Sajjad, Fath Elrahman Saleh, Abdelrahim Elmadany, and Preslav Nakov. 2021. Camelbert: Open large-scale language models for arabic. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4706–4721. Association for Computational Linguistics. find
  20. Kirolos Khela. 2023. Sentiment-bert: Arabic sentiment analysis model. https://huggingface.co/kir oloskhela/Sentiment-Bert . Accessed: 2025-07-21. ↗
  21. David G Kleinbaum, K Dietz, M Gail, Mitchel Klein, and Mitchell Klein. 2002. Logistic regression. Springer. find
  22. Barbara Lewandowska-Tomaszczyk. 2023. A simplified taxonomy of offensive language (SOL) for computational applications. Konin Language Studies, 10(3):213–227. find
  23. Barbara Lewandowska-Tomaszczyk, Anna B aczkowska, Chaya Liebeskind, Giedre Valunaite Oleskeviciene, and Slavko Žitnik. 2023. An integrated explicit and implicit offensive language taxonomy. Lodz Papers in Pragmatics, 19(1):7–48. find
  24. Barbara Lewandowska-Tomaszczyk, Slavko Žitnik, Anna B aczkowska, Chaya Liebeskind, Jelena Mitrović, and Giedrė Valūnaitė Oleškevičienė. 2021. LOD-connected offensive language ontology and tagset enrichment. In R. Carvalho & R. Rocha Souza, R. (Eds.), Proceedings of the Workshop and Tutorials held at LDK 2021 co-located with the 3rd Language, Data and Knowledge Conference, volume 3064, pages 135–150. CEUR Workshop Proceedings. find
  25. Barbara Lewandowska-Tomaszczyk, Slavko Žitnik, Chaya Liebeskind, Giedrė Valūnaitė-Oleškevičienė, Anna B aczkowska, Paul A Wilson, Marcin Trojszczak, Ivana Brač, Lobel Filipić, and Ana Ostroški Anić. 2022. Annotation scheme and evaluation: The case of offensive language. Rasprave. find
  26. Mahmoud Nabil, Mohamed Aly, and Amir Atiya. 2015. ASTD: Arabic sentiment tweets dataset. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2515–2519, Lisbon, Portugal. Association for Computational Linguistics.
  27. Hatem Noaman. 2023. Improved emotion detection framework for arabic text using transformer models. Advanced Engineering Technology and Application, 12(2):1–11. find
  28. Nedjma Ousidhoum, Zizheng Lin, Hongming Zhang, Yangqiu Song, and Dit-Yan Yeung. 2019. Multilingual and multi-aspect hate speech analysis. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4675–4684.
  29. Chaya Liebeskind, Ali Afawi, Marina Litvak, and Natalia Vanetik. 2024. Classifying offensive language in arabic: a novel taxonomy and dataset. Lodz Papers in Pragmatics, 20(2):433–462. find
  30. Mahesh Pal. 2005. Random forest classifier for remote sensing classification. Journal of Remote Sensing, 26(1):217–222. find
  31. Chaya Liebeskind, Natalia Vanetik, and Marina Litvak. 2023. Hebrew offensive language taxonomy and dataset. Lodz Papers in Pragmatics, 19(2):325–351. find
  32. Niloofar Safi Samghabadi, Afsheen Hatami, Mahsa Shafaei, Sudipta Kar, and Thamar Solorio. 2020. Attending the emotions to detect online abusive language. In Proceedings of the Fourth Workshop on Online Abuse and Harms, pages 79–88. find
  33. Marina Litvak, Natalia Vanetik, Yaser Nimer, Abdulrhman Skout, and Israel Beer-Sheba. 2021. Offensive language detection in semitic languages. In Multimodal Hate Speech Workshop, volume 2021, pages 7–12. find
  34. Khouloud Mnassri, Praboda Rajapaksha, Reza Farahbakhsh, and Noel Crespi. 2023a. Hate speech and offensive language detection using an emotion-aware shared encoder. In ICC 2023-IEEE International Conference on Communications, pages 2852–2857. IEEE. find
  35. Khouloud Mnassri, Praboda Rajapaksha, Reza Farahbakhsh, and Noël Crespi. 2023b. Hate speech and offensive language detection using an emotion-aware shared encoder. CoRR, abs/2302.08777. ↗
  36. Hamdy Mubarak, Kareem Darwish, and Walid Magdy. 2017. Abusive language detection on Arabic social media. In Proceedings of the First Workshop on Abusive Language Online, pages 52–56.
  37. Hamdy Mubarak, Ammar Rashed, Kareem Darwish, Younes Samih, and Ahmed Abdelali. 2020. Arabic offensive language on Twitter: Analysis and experiments. arXiv preprint arXiv:2004.02192. ↗
  38. Hala Mulki and Bilal Ghanem. 2021. Let-Mi: An Arabic Levantine Twitter Dataset for Misogynistic Language. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 154–163.
  39. Hala Mulki, Hatem Haddad, Chedi Bechikh Ali, and Halima Alshabani. 2019. L-hsab: A Levantine Twitter dataset for hate speech and abusive language. In Proceedings of the third workshop on abusive language online, pages 111–118.
  40. Lewis Tunstall, Nils Reimers, Unso Eun Seo Jo, Luke Bates, Daniel Korat, Moshe Wasserblat, and Oren Pereg. 2022. Efficient few-shot learning without prompts. arXiv. find
  41. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019a. Predicting the Type and Target of Offensive Posts in Social Media. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1415–1420.
  42. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019b. Semeval-2019 task 6: Identifying and categorizing offensive language in social media (offenseval). In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 75–86.
  43. Marcos Zampieri, Preslav Nakov, Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Hamdy Mubarak, Leon Derczynski, Zeses Pitenis, and Cagri Coltekin. 2020. Semeval-2020 task 12: Multilingual offensive language identification in social media (offenseval 2020). In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1425–1447. Association for Computational Linguistics. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/vanetik-2025-fine-grained-arabic-offensive-language/