Elamur Research⌕
☰
home / papers / 2024 / Is it Offensive or Abusive? An Empirical Study of Hateful Language Detection of Arabic Social Media Texts

Is it Offensive or Abusive? An Empirical Study of Hateful Language Detection of Arabic Social Media Texts

Paper · 2024 · Salim Al Mandhari, Mo El-Haj and Paul Rayson

Summary

The paper looks at harmful language in Arabic YouTube comments, where Modern Standard Arabic (MSA) and regional dialects are mixed. It asks whether "offensive" and "abusive" content can be reliably told apart. Earlier studies use these labels inconsistently, so the authors first review that literature and settle on working definitions. Offensive covers hateful and aggressive remarks. Abusive covers vulgar, pornographic or sexual remarks. They then train NB, SVM, fastText, Keras and multilingual RoBERTa XML (XLM-R) embedding models on two separate YouTube corpora with about 47K comments in total. The experiments test TF-IDF, grid search, n-gram features and data cleaning. Every model except NB reaches roughly 82% accuracy, and fastText with word tri-grams is the strongest setup. Pre-processing changed the results very little. Class imbalance between offensive and abusive comments held accuracy down. The work matters for moderation in dialect-heavy Arabic, where keyword filters fail on slang, misspellings and words that change meaning by dialect. It also shows the class definitions can be learned by standard classifiers.

Key points

  • Two corpora of YouTube comments, about 47K in total, were used independently:
    • an open multi-class set of 32K comments from the Aljazeera channel, labelled offensive, abusive and clean, covering politics, society and economics;
    • a separate binary set.
  • Models tested: Naïve Bayes and SVM, the fastText and Keras classifiers, and a multilingual embedding model based on RoBERTa XML.
  • All models except NB reached about 82% accuracy; fastText was best at 82%.
  • Word tri-grams improved fastText results. TF-IDF and grid search were also applied, while cleaning and pre-processing brought no significant gain.
  • Dialectal polysemy is a core difficulty:
    • mamhún means "afflicted" in MSA but "libidinous" in a Gulf-influenced comment;
    • yubaqi has a distinct sense in Omani dialects of Buhla and Al-Hamra.

Interesting ideas

  • I1. The paper fixes class definitions from established categories of swearing (vulgar, pornographic, odious, after Jay and Janschewitz 2008) before modelling, which keeps labels consistent across a contested literature.
  • I2. One word form can be harmless in the standard language yet offensive in a particular dialect, so sense choice depends on the variety being used.
  • I3. Inspecting class keyword lists showed that frequent items such as "channel" and politicians' names are topic words, not markers of the class.
  • I4. The authors measured the effect of cleaning and found it negligible instead of assuming normalization helps.
  • I5. As future work they propose splitting the coarse binary "offensive" class into offensive and abusive, then merging both datasets into one larger, finer-labelled corpus.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): Speech-act labels like insult, threat and obscenity in prag could be defined by explicit criteria from such typologies, so each tag is explainable.
  • I2 (world): WordNet-derived lexicon entries could carry variety and register tags, letting the world builder pick a sense conditioned on detected dialect.
  • I3 (learn): Automatically induced cues or repair rules could be checked against topic-frequency negative controls so that domain vocabulary is not mistaken for signal.
  • I4 (tokens): Each normalization step in crate en could be ablated on downstream parse accuracy, keeping only steps with measurable benefit.
  • I5 (learn): Coarse silver labels from different sources could be refined into a shared finer scheme and pooled to enlarge training data for rare classes.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Lancaster UniversityLancaster · United Kingdom35 works · 1,615 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Salim Al Mandhari, Mo El-Haj and Paul Rayson. 2024. Is it Offensive or Abusive? An Empirical Study of Hateful Language Detection of Arabic Social Media Texts. Proceedings of the First International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{mandhari2024is,
  title = {{Is it Offensive or Abusive? An Empirical Study of Hateful Language Detection of Arabic Social Media Texts}},
  author = {Salim {Al Mandhari}, Mo El-Haj and Paul Rayson},
  booktitle = {Proceedings of the First International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security},
  year = {2024},
  url = {https://aclanthology.org/2024.nlpaics-1.16/}
}

References

19 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Atif Khan, Abrar Ahmed, Salman Jan, Muhammad Bilal, and Megat F Zuhairi. 2024. Abusive language detection in urdu text: Leveraging deep learning and attention mechanism. IEEE Access. ↗
  2. Tina R Patil, SS Sherekar, et al. 2013. Performance analysis of naive bayes and j48 classification algorithm for data classification. International Journal of Computer Science and Applications, 6(2):256–261. find
  3. Mahi Kolla, Siddharth Salunkhe, Eshwar Chandrasekharan, and Koustuv Saha. 2024. Llm-mod: Can large language models assist content moderation? find
  4. R Roache. 2019. Offensive language. [Podcast]. find
  5. Evan Lutins. 2019. Grid searching in machine learning: Quick explanation and python implementation. [Online]. Available: https://medium.com/@elutins/grid-searching-in-machine-learning-quick-explanation-and-python-implementation-550552200596. [Accessed 06 September 2019]. ↗
  6. Motaz K Saad and Wesam M Ashour. 2010. Arabic text classification using decision trees. Arabic text classification using decision trees, 2. find
  7. Yasser Salem, Arnold Hensman, and Brian Nolan. 2008. Implementing arabic-to-english machine translation using the role and reference grammar linguistic model. find
  8. TF-IDF. 2019. What does tf-idf mean? [Online]. Available: http://www.tfidf.com. [Accessed 06 September 2019]. ↗
  9. Brigitte Mathiak and Silke Eckstein. 2004. Five steps to text mining in biomedical literature. In Proceedings of the second European workshop on data mining and text mining in bioinformatics, volume 24. find
  10. Puneet Mathur, Ramit Sawhney, Meghna Ayyar, and Rajiv Shah. 2018. Did you offend me? classification of offensive tweets in hinglish language. In Proceedings of the 2nd Workshop on Abusive Language Online (ALW2), pages 138–148. find
  11. Zeerak Waseem, Thomas Davidson, Dana Warmsley, and Ingmar Weber. 2017. Understanding abuse: A typology of abusive language detection subtasks. arXiv preprint arXiv:1705.09899. ↗
  12. Jerold D Miller and Shirley M Hufstedler. 2009. Cyberbullying knows no borders. Australian Teacher Education Association. find
  13. Andrea Millwood-Hargrave. 2000. Delete expletives? Advertising Standards Authority London. find
  14. Sayyed Z. Mohammed, E. 2019. Arabic-sos: Segmentation, stemming, and orthography standardization for classical and pre-modern standard arabic. [Forthcoming]. find
  15. Djedjiga Mouheb, Rutana Ismail, Shaheen Al Qaraghuli, Zaher Al Aghbari, and Ibrahim Kamel. 2018. Detection of offensive messages in arabic social media communications. In 2018 International Conference on Innovations in Information Technology (IIT), pages 24–29. IEEE. find
  16. Abdullah Y Muaad, Shaina Raza, Usman Naseem, and Hanumanthappa J Jayappa Davanagere. 2023. Arabic text detection: a survey of recent progress challenges and opportunities. Applied Intelligence, 53(24):29845–29862. ↗
  17. Hamdy Mubarak, Kareem Darwish, and Walid Magdy. 2017. Abusive language detection on arabic social media. In Proceedings of the First Workshop on Abusive Language Online, pages 52–56.
  18. Hamdy Mubarak, Ammar Rashed, Kareem Darwish, Younes Samih, and Ahmed Abdelali. 2020. Arabic offensive language on twitter: Analysis and experiments. arXiv preprint arXiv:2004.02192. ↗
  19. NCPC. 2019. Cyberbullying tip sheets. [Online]. Available: shorturl.at/atADJ. [Accessed 07 August 2019]. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mandhari-2024-is-it-offensive-or-abusive/