Elamur Research⌕
☰
home / papers / 2020 / Quick and Simple Approach for Detecting Hate Speech in Arabic Tweets

Quick and Simple Approach for Detecting Hate Speech in Arabic Tweets

Paper · 2020 · Abeer Abuzayed and Tamer Elsayed

Summary

This paper addresses the automatic detection of hate speech in Arabic tweets. The authors built their system for the hate speech subtask of the OSACT4 shared task, and they had only about three days to do it. They deliberately used generic, easy-to-compute unigram features and off-the-shelf frameworks such as Keras. The data was the organizers' set of 8,000 labelled tweets. After light cleaning and Arabic letter normalization, they compared 15 learners (7 classical, 8 neural) and two term representations: tf-idf weights and AraVec2.0 word embeddings.

Among the classical learners, tf-idf beat averaged embeddings. This contradicts a claim in the abstract, but the experiments and the conclusion support it. The best neural model was a joint CNN+LSTM network, which beat the strongest classical system. It reached 0.73 macro-F1 on dev and 0.69 on test, against a majority-class baseline of 0.49. The work matters as a reproducible baseline for Arabic hate speech detection, and it shows how far a modest, quickly assembled pipeline can go.

Key points

  • The dataset is the OSACT4 shared task set of 8,000 Arabic tweets labelled HS / NOT_HS. The labels are imbalanced, with hate speech as the minority class.
  • The embeddings come from the AraVec2.0 SkipGram 300D model, pre-trained on over 77M tweets. Classical models received the average of a tweet's word vectors as features. Neural models used the vectors to initialize their embedding layer.
  • The best classical setup was Extra Trees with tf-idf. Combining CNN with an RNN improved on every single-architecture network, and CNN+LSTM was best overall.
  • Oversampling the minority class did not help. On the submitted runs, CNN+LSTM scored 0.73 dev / 0.69 test macro-F1, while the oversampled version scored 0.70 / 0.65.
  • Preprocessing removed diacritics, tatweel, numbers, mentions, URLs and repeated letters. It also unified alef variants, yaa/alef maqsura, hamza carriers, taa marbuta→haa, and گ→ك. The code is open-source.

Interesting ideas

  • I1. The authors kept the classifier fixed and swapped only the representation (tf-idf vs averaged embeddings), which isolates the effect of features from the effect of the model.
  • I2. They reported a trivial majority-class baseline next to every result, which makes a modest score interpretable.
  • I3. Their error table showed that different models fail on different tweets, which suggests the models' disagreements carry information.
  • I4. They used aggressive but explicit orthographic normalization (letter variants, elongation, repeated characters) before any lookup against a pre-trained vocabulary.
  • I5. The related-work discussion separates hate speech from mere offensive language: hate speech requires a targeted group characteristic and an intent to harm, and it may contain no profanity at all.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Mova's evaluation could use the same controlled swap, changing one component (e.g., the lemma source or the parser) while freezing the rest, so that gains can be attributed honestly.
  • I2 (learn): every Mova benchmark could ship with a "predict the most frequent answer" control, used as a negative control that any explainable reader must clearly beat.
  • I3 (learn): disagreement between witness parsers could flag the sentences where silver data is unreliable and should go to an LLM teacher tier.
  • I4 (tokens): an analogous deterministic, logged normalization layer for English noisy text (elongations like "sooo", odd Unicode quotes) could raise lexicon hit rates while staying explainable.
  • I5 (frames): this suggests detecting hostile speech acts through frame structure (who attacks whom, on the basis of which identity attribute) rather than through lists of bad words.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 3Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 worksBias, fairness & safety · 517 works

Cite

Cite as Abeer Abuzayed and Tamer Elsayed. 2020. Quick and Simple Approach for Detecting Hate Speech in Arabic Tweets. Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{abuzayed2020quick,
  title = {{Quick and Simple Approach for Detecting Hate Speech in Arabic Tweets}},
  author = {Abeer Abuzayed and Tamer Elsayed},
  booktitle = {Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection},
  year = {2020},
  url = {https://aclanthology.org/2020.osact-1.18/}
}

References

18 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alakrot, A., Murray, L. and Nikolov, N.S., 2018. Towards accurate detection of offensive language in online communication in arabic. Procedia computer science , 142, pp.315-320. find
  2. Albadi, N., Kurdi, M. and Mishra, S., 2018, August. Are they our brothers? Analysis and detection of religious hate speech in the Arabic Twittersphere. In 2018 IEEE/ACM ASONAM (pp. 69-76). find
  3. Chowdhury, A.G., Didolkar, A., Sawhney, R. and Shah, R., 2019. ARHNet-Leveraging Community Interaction for Detection of Religious Hate Speech in Arabic. In Proceedings of ACL: Student Research Workshop (pp. 273-280).
  4. Devlin, J., Chang, M.W., Lee, K. and Toutanova, K., 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 . Malmasi, S. & Zampieri, M., 2018. Challenges in discriminating profanity from hate speech. Journal of Experimental & Theoretical Artificial Intelligence, 30(2), pp. 187-202. ↗
  5. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S. and Dean, J., 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems (pp. 3111-3119).
  6. Mohaouchane, H., Mourhir, A. and Nikolov, N.S., 2019, October. Detecting Offensive Language on Arabic Social Media Using Deep Learning. In IEEE SNAMS (pp. 466-471). find
  7. Mondal, M., Silva, L. A. & Benevenuto, F., 2017. A Measurement Study of Hate Speech in Social Media.. N ew York, USA, ACM, p. 85–94. find
  8. Mubarak, H. & Darwish, K., 2019. Arabic Offensive Language Classification on Twitter. In International Conference on Social Informatics (pp. 269-276). Springer. find
  9. Mubarak, H., Darwish, K. and Magdy, W., 2017, August. Abusive language detection on Arabic social media. In Proceedings of the First Workshop on Abusive Language Online (pp. 52-56).
  10. Mubarak, H., Darwish, K., Magdy, W., Elsayed, T. and Al-Khalifa, H., 2020. Overview of OSACT4 Arabic Offensive Language Detection Shared Task. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) , vol. 4.
  11. Ousidhoum, N., Lin, Z., Zhang, H., Song, Y. and Yeung, D.Y., 2019. Multilingual and Multi-Aspect Hate Speech Analysis. arXiv preprint arXiv:1908.11049. ↗
  12. Smedt, T., De Pauw, G. and Van Ostaeyen, P., 2018. Automatic detection of online jihadist hate speech. arXiv preprint arXiv:1803.04596. ↗
  13. Soliman, A.B., Eissa, K. and El-Beltagy, S.R., 2017. Aravec: A set of arabic word embedding models for use in arabic nlp. Procedia Computer Science, 117, pp.256-265. find
  14. Wang, X., Jiang, W. and Luo, Z., 2016. Combination of convolutional and recurrent neural network for sentiment analysis of short texts. In Proceedings of COLING 2016: Technical papers (pp. 2428-2437). find
  15. Watanabe, H., Bouazizi, M. & Ohtsuki, T., 2018. Hate speech on Twitter: A pragmatic approach to collect hateful and offensive expressions and perform hate speech detection. IEEE Access, 6 , pp. 13825-13835. find
  16. Zhang, Z. & Luo, L., 2018. Hate speech detection: A solved problem? the challenging case of long tail on Twitter. CoRR abs/1803.03662 (2018). ↗
  17. Zhang, Z., Robinson, D. & Tepper, a. J., 2018. Hate Speech Detection Using a Convolution-LSTM Based Deep Neural Network. In Proceedings of ACM WWW conference (WWW’2018). N ew York, NY, USA. find
  18. Zhou, C., Sun, C., Liu, Z. and Lau, F., 2015. A C-LSTM neural network for text classification. arXiv preprint arXiv:1511.08630. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/abuzayed-2020-quick-simple-approach-detecting-hate/