Elamur Research⌕
☰
home / papers / 2022 / AlexU-AIC at Arabic Hate Speech 2022: Contrast to Classify

AlexU-AIC at Arabic Hate Speech 2022: Contrast to Classify

Paper · 2022 · Ahmad Shapiro, Ayman Khalafallah and Marwan Torki

Summary

The authors tackle the three-level Arabic hate speech shared task at OSACT5 2022, using tweets annotated by Mubarak et al. (2022). The levels are: is a tweet offensive, is an offensive tweet hate speech, and which of six hate categories it falls into. The training split is small (about 8.9k tweets) and heavily skewed, so fine-tuned transformers overfit easily.

The team compared AraBERT and MarBERT encoders and settled on MarBERTv2, whose pre-training on tweets matched the data. They tested three training regimes on top of Arabic-specific text normalisation:

  • plain classification fine-tuning,
  • contrastive learning with several loss variants,
  • multi-task learning, with an extra balanced corpus pooled from earlier Arabic offensive-language datasets.

In the final submission, sub-task A was handled by an ensemble of five models, while B and C relied on a multi-task model. It reached macro-F1 of 0.841, 0.817 and 0.476, far above majority baselines. The study is useful as a comparison of which training regime fits which degree of class imbalance.

Key points

  • The dataset has 12.7K tweets, split 70/10/20 into 8887 train, 1270 dev and 2541 test.
    • Labels: OFF/NOT OFF, HS/NOT HS, six HS classes (Race, Religion, Ideology, Disability, Social Class, Gender), plus vulgar and violent flags.
  • Imbalance is severe:
    • about 35.7% OFF and 10.8% HS in training;
    • roughly 89% of training tweets are NOT HS in sub-task C;
    • HS4 (Disability) has no training examples at all.
  • MarBERTv2 was preferred for two reasons: it was pre-trained on 1B Arabic tweets, and its vocabulary keeps 567 emoji. This mattered because all but 2 train/dev tweets contain emoji.
  • The BALANCED resource was assembled from OSACT2020, OffensEval2020 and other corpora, and was used only for multi-task learning.
    • Sub-task A: 19,906 examples.
    • Sub-task B: 4800 examples, compared with only 959 HS tweets in the original training data.
  • Final macro-F1 scores were 0.841 (A), 0.817 (B) and 0.476 (C), against majority baselines of 0.394, 0.472 and 0.135.
    • Pre-processing helped.
    • Contrastive training edged out plain fine-tuning only when imbalance was moderate.

Interesting ideas

  • I1. Before picking an encoder, the authors checked a surface property of the data (near-universal emoji) and chose a model whose vocabulary keeps it; they later learned emoji had been the anchors used to collect the corpus.
  • I2. They ran a per-subtask comparison of training regimes and found that the best regime depends on how skewed the labels are, with contrastive objectives losing their edge under extreme imbalance.
  • I3. External data with a different distribution was pooled to rebalance rare classes, but only as an auxiliary multi-task signal, which limits the harm from distribution mismatch.
  • I4. The task is a cascade of labels, where each finer decision applies only to tweets that passed the coarser one, and the levels are learned jointly.
  • I5. Orthographic variants (Alef forms, digits, punctuation, hashtags split into words) are unified before encoding to shrink sparsity.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Add negative controls that test whether a trained component leans on artifacts of how the training data was gathered, rather than on structure.
  • I2 (learn): Choose training objectives per component based on measured label skew (e.g., rare dependency relations or rare frame roles) instead of applying one recipe everywhere.
  • I3 (tree): Treebanks in other UD dialects could feed gparse as auxiliary tasks rather than being merged into the main training set, keeping conversion mismatch from contaminating the primary parser.
  • I4 (frames): Speech-act, offence and irony judgments in prag could be organised as nested decisions trained together, so that coarse cues support the sparse fine-grained ones.
  • I5 (tokens): A comparable explicit normalisation layer in crate en (Unicode quotes, digit styles, hashtag or camel-case splitting) would reduce out-of-vocabulary noise for the tagger and lemmatiser.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Alexandria UniversityAlexandria · Egypt39 works · 287 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

🧭 Topics 3Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 worksBias, fairness & safety · 517 works

Cite

Cite as Ahmad Shapiro, Ayman Khalafallah and Marwan Torki. 2022. AlexU-AIC at Arabic Hate Speech 2022: Contrast to Classify. Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur'an QA and Fine-Grained Hate Speech Detection.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{shapiro2022alexu,
  title = {{AlexU-AIC at Arabic Hate Speech 2022: Contrast to Classify}},
  author = {Ahmad Shapiro, Ayman Khalafallah and Marwan Torki},
  booktitle = {Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur'an QA and Fine-Grained Hate Speech Detection},
  year = {2022},
  url = {https://aclanthology.org/2022.osact-1.26/}
}

References

30 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdul-Mageed, M., Alhuzali, H., and Elaraby, M. (2018). You tweet what you speak: A city-level dataset of Arabic dialects. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, May. European Language Resources Association (ELRA).
  2. Abdul-Mageed, M., Elmadany, A. A., and Nagoudi, E. M. B. (2021). ARBERT & MARBERT: deep bidirectional transformers for arabic. CoRR, abs/2101.01785. ↗
  3. Abu Farha, I. and Magdy, W. (2020). Multitask learning for Arabic offensive language and hate-speech detection. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 86–90, Marseille, France, May. European Language Resource Association.
  4. Aldjanabi, W., Dahou, A., Al-qaness, M. A. A., Elaziz, M. A., Helmi, A. M., and Damaševičius, R. (2021). Arabic offensive and hate speech detection using a cross-corpora multi-task learning model. Informatics, 8(4):69, October. ↗
  5. Antoun, W., Baly, F., and Hajj, H. M. (2020). Arabert: Transformer-based model for arabic language understanding. CoRR, abs/2003.00104. ↗
  6. Ashi, M. M., Siddiqui, M. A., and Nadeem, F. (2018). Pre-trained word embeddings for arabic aspect-based sentiment analysis of airline tweets. In Advances in Intelligent Systems and Computing, pages 241–251. Springer International Publishing, August. find
  7. Darwish, K. and Mubarak, H. (2016). Farasa: A new fast and accurate Arabic word segmenter. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1070–1074, Portorož, Slovenia, May. European Language Resources Association (ELRA).
  8. Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2018). BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805. ↗
  9. Djandji, M., Baly, F., Antoun, W., and Hajj, H. (2020). Multi-task learning using AraBert for offensive language detection. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 97–101, Marseille, France, May. European Language Resource Association.
  10. Feng, F., Yang, Y., Cer, D., Arivazhagan, N., and Wang, W. (2020). Language-agnostic BERT sentence embedding. CoRR, abs/2007.01852. ↗
  11. Haddad, B., Orabe, Z., Al-Abood, A., and Ghneim, N. (2020). Arabic offensive language detection with attention-based deep neural networks. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 76– 81, Marseille, France, May. European Language Resource Association. ↗
  12. Hadsell, R., Chopra, S., and LeCun, Y. (2006). Dimensionality reduction by learning an invariant mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), volume 2, pages 1735–1742. find
  13. Hassan, S., Samih, Y., Mubarak, H., Abdelali, A., Rashed, A., and Chowdhury, S. A. (2020). ALT submission for OSACT shared task on offensive language detection. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 61–65, Marseille, France, May. European Language Resource Association.
  14. Hermans, A., Beyer, L., and Leibe, B. (2017). In defense of the triplet loss for person re-identification. CoRR, abs/1703.07737. ↗
  15. Husain, F. (2020). OSACT4 shared task on offensive language detection: Intensive preprocessing-based approach. CoRR, abs/2005.07297. ↗
  16. Ibrahim, M., Torki, M., and El-Makky, N. (2018). Imbalanced toxic comments classification using data augmentation and deep learning. In 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), pages 875–878. find
  17. Ibrahim, M., Torki, M., and El-Makky, N. (2020). AlexU-BackTranslation-TL at SemEval-2020 task 12: Improving offensive language detection using data augmentation and transfer learning. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1881–1890, Barcelona (online), December. International Committee for Computational Linguistics. ↗
  18. Mubarak, H., Darwish, K., Magdy, W., Elsayed, T., and Al-Khalifa, H. (2020). Overview of OSACT4 Arabic offensive language detection shared task. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 48– 52, Marseille, France, May. European Language Resource Association.
  19. Mubarak, H., Hassan, S., and Chowdhury, S. A. (2022). Emojis as anchors to detect arabic offensive language and hate speech. arXiv preprint arXiv:2201.06723. ↗
  20. Müller, K. and Schwarz, C. (2020). Fanning the Flames of Hate: Social Media and Hate Crime. Journal of the European Economic Association, 19(4):2131–2167, 10. ↗
  21. Zampieri, M., Nakov, P., Rosenthal, S., Atanasova, P., Karadzhov, G., Mubarak, H., Derczynski, L., Pitenis, Z., and Çöltekin, c. (2020). SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020). In Proceedings of SemEval. 8. ↗
  22. Language Resource References Abu Farha, I. and Magdy, W. (2019). Mazajak: An online Arabic sentiment analyser. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 192–198, Florence, Italy, August. Association for Computational Linguistics.
  23. Alakrot, A., Murray, L., and Nikolov, N. S. (2018). Dataset construction for the detection of anti-social behaviour in online communication in arabic. Procedia Computer Science, 142:174–181. Arabic Computational Linguistics. find
  24. Albadi, N., Kurdi, M., and Mishra, S. (2018). Are they our brothers? analysis and detection of religious hate speech in the arabic twittersphere. In Proceedings of the 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM ’18, page 69–76. IEEE Press. find
  25. Chowdhury, S. A., Mubarak, H., Abdelali, A., Jung, S.-g., Jansen, B. J., and Salminen, J. (2020). A multi-platform arabic news comment dataset for offensive language detection. In Proceedings of the International Conference on Language Resources and Evaluation (LREC’20).
  26. Haddad, H., Mulki, H., and Oueslati, A. (2019). T-hsab: A tunisian hate speech and abusive dataset. In ICALP. find
  27. Mulki, H., Haddad, H., Bechikh Ali, C., and Alshabani, H. (2019). L-HSAB: A Levantine Twitter dataset for hate speech and abusive language. In Proceedings of the Third Workshop on Abusive Language Online, pages 111–118, Florence, Italy, August. Association for Computational Linguistics.
  28. Ousidhoum, N., Lin, Z., Zhang, H., Song, Y., and Yeung, D.-Y. (2019). Multilingual and multi-aspect hate speech analysis. In Proceedings of EMNLP. Association for Computational Linguistics.
  29. Reimers, N. and Gurevych, I. (2019). Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11.
  30. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., and Brew, J. (2019). Huggingface’s transformers: State-of-the-art natural language processing. CoRR, abs/1910.03771. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shapiro-2022-alexu-aic-arabic-hate-speech/