Elamur Research⌕
☰
home / papers / 2020 / SalamNET at SemEval-2020 Task12: Deep Learning Approach for Arabic Offensive Language Detection

SalamNET at SemEval-2020 Task12: Deep Learning Approach for Arabic Offensive Language Detection

Summary

This system paper tackles automatic detection of offensive Arabic tweets for SemEval-2020 Task 12 (OffensEval), a language with few such tools compared to English. The authors use the Arabic OffensEval 2020 dataset of 10,000 labelled tweets. Before classification, the text goes through a heavy normalization pipeline taken from Husain (2020): emojis become text labels, letter variants are unified, dialectal nouns are mapped to Modern Standard Arabic, and some hyponyms are replaced by their hypernyms. They pair two input representations, TF-IDF and AraVec embeddings, with a Logistic Regression baseline and with several recurrent networks (RNN, GRU, Bi-GRU, LSTM, Bi-LSTM), tuned by grid search. The strongest configuration, a Bi-GRU fed TF-IDF features and named SalamNET, reaches a macro-F1 of 0.83. The paper's main value is its error analysis. It identifies tweets that every model misclassifies, traces the failures to dialect, named people and sports talk, and uses these findings to propose concrete next steps for Arabic offensive-language systems.

Key points

  • The data is the Arabic OffensEval 2020 corpus: 10,000 tweets, of which only 1,900 (about 19%) are offensive, split 7,000 / 1,000 / 2,000 for train, dev and test.
  • The official submissions placed 34th and 38th out of 53 teams; the winning team scored 0.901 macro-F1.
    • The LR + AraVec model reached 0.81 macro-F1.
    • The Bi-GRU + TF-IDF model reached 0.80 macro-F1.
    • Later tuning raised Bi-GRU + TF-IDF (SalamNET) to 0.83.
  • The two feature types behaved in opposite ways:
    • AraVec (Twitter corpus, CBOW) beat character TF-IDF (2–5 grams) for LR.
    • TF-IDF beat AraVec for every recurrent model.
  • Upsampling the minority class was tried and then removed from the pipeline, because it lowered performance.
  • All deep models together missed 390 offensive tweets and wrongly flagged 11 harmless ones. Typical misses were Egyptian-dialect tweets that mention celebrities or sports teams.

Interesting ideas

  • I1. The authors examine only the items that every model gets wrong, so systematic data or representation gaps stand out from random model noise. — For Mova (learn): Sentences on which all witness parsers fail together could be routed to a separate "coverage gap" queue for teacher review, rather than treated as ordinary disagreements.
  • I2. They compare examples that one feature family solves and the other fails, then describe that slice by length, vocabulary size and dialect. — For Mova (tree): Disagreements between the perceptron parser and gparse could be sliced the same way (sentence length, genre, construction) to show where each parser is the more reliable witness.
  • I3. Normalization is aggressive: emojis become words, letter runs are capped at two, and dialect forms are mapped to a standard variety before any classification happens. — For Mova (tokens): A similar pre-tagging layer for informal English could cap elongated spellings and verbalize emojis, while keeping the original token for explanations.
  • I4. Collapsing specific nouns into a category such as "animal" helped earlier work, but here it appeared to erase distinctions that matter, since some offensive animal terms differ from harmless ones. — For Mova (world): WordNet hypernym backoff should be added alongside the original lemma, not substituted for it, so frames keep the fine-grained sense when it carries meaning.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Sentiment & emotion · Semitic languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2007.13974 · arXiv 2007.13974

@misc{henry2020salamnet,
  title = {{SalamNET at SemEval-2020 Task12: Deep Learning Approach for Arabic Offensive Language Detection}},
  author = {Henry;, Fatemah Husain; Jooyeon Lee; Samuel and Uzuner, Ozlem},
  year = {2020},
  doi = {10.48550/arxiv.2007.13974},
  eprint = {2007.13974},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2007.13974}
}

References

21 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ehab A. Abozinadah and James H. Jones. 2017. A statistical learning approach to detect abusive twitter accounts. In Proceedings of the International Conference on Compute and Data Analysis, ICCDA ’17, page 6–13, New York, NY, USA. Association for Computing Machinery. find
  2. Ibrahim Abu Farha and Walid Magdy. 2019. Mazajak: An online Arabic sentiment analyser. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 192–198, Florence, Italy, August. Association for Computational Linguistics. find
  3. Azalden Alakrot, Liam Murray, and Nikola Nikolov. 2018a. Dataset construction for the detection of anti-social behaviour in online communication in arabic. Procedia Computer Science, 142:174–181, 11. find
  4. Azalden Alakrot, Liam Murray, and Nikola S. Nikolov. 2018b. Dataset construction for the detection of anti-social behaviour in online communication in arabic. Procedia Computer Science, 142:174–181. find
  5. Nuha Albadi, Maram Kurdi, and Shivakant Mishra. 2018. Are they our brothers? analysis and detection of religious hate speech in the arabic twittersphere. In 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pages 69–76, August. find
  6. Nuha Albadi, Maram Kurdi, and Shivakant Mishra. 2019. Investigating the effect of combining GRU neural networks with handcrafted features for religious hatred detection on arabic twitter space. Social Network Analysis and Mining, 9(41):1–19, August. find
  7. Ali Alshehri, El Moatez Billah Nagoudi, Hassan Alhuzali, and Muhammad Abdul-Mageed. 2018. Think before your click: Data and models for adult content in arabic twitter. In Second Workshop on Text Analytics for Cybersecurity and Online Safety (TA-COS 2018). European Language Resources Association (ELRA), May. find
  8. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2016. Enriching word vectors with subword information. cite arxiv:1607.04606Comment: Accepted to TACL. The two first authors contributed equally. ↗
  9. Arijit Ghosh Chowdhury, Aniket Didolkar, Ramit Sawhney, and Rajiv Ratn Shah. 2019. ARHNet - leveraging community interaction for detection of religious hate speech in Arabic. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 273–280, Florence, Italy, July. Association for Computational Linguistics. find
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171– 4186, Minneapolis, Minnesota, June. Association for Computational Linguistics. find
  11. Batoul Haidar, Maroun Chamoun, and Ahmed Serhrouchni. 2017. Multilingual cyberbullying detection system: Detecting cyberbullying in arabic content. In 2017 1st Cyber Security in Networking Conference (CSNet), pages 1–8, October. find
  12. Batoul Haidar, Maroun Chamoun, and Ahmed Serhrouchni. 2018. Arabic cyberbullying detection: Using deep learning. In 2018 7th International Conference on Computer and Communication Engineering (ICCCE), pages 284–289, September. find
  13. Batoul Haidar, Maroun Chamoun, and Ahmed Serhrouchni. 2019. Arabic cyberbullying detection: Enhancing performance by using ensemble machine learning. In 2019 International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), pages 323–327, July. find
  14. Fatemah Husain. 2020. OSACT4 shared task on offensive language detection: Intensive preprocessing based approach. In The 4th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT4). Proceedings of the Twelfth International Conference on Language Resources and Evaluation (LREC 2020), Marseille, France, May. find
  15. Zachary Keyser. 2018. Gab, the ’alt-right twitter’ used by antisemitic pittsburgh shooter. The Jerusalem Post, October. find
  16. Hanane Mohaouchane, Asmaa Mourhir, and Nikola Nikolov. 2019. Detecting offensive language on arabic social media using deep learning. In 2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS), pages 466–471, October. find
  17. Hamdy Mubarak, Kareem Darwish, and Walid Magdy. 2017. Abusive language detection on Arabic social media. In Proceedings of the First Workshop on Abusive Language Online, pages 52–56, Vancouver, BC, Canada, August. Association for Computational Linguistics. find our card
  18. Hamdy Mubarak, Ammar Rashed, Kareem Darwish, Younes Samih, and Ahmed Abdelali. 2020. Arabic offensive language on twitter: Analysis and experiments. arXiv preprint arXiv:2004.02192. ↗ our card
  19. Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah Smith. 2019. The risk of racial bias in hate speech detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, page 1668–1678. Association for Computational Linguistics. find our card
  20. Abu Bakr Soliman, Kareem Eissa, and Samhaa R. El-Beltagy. 2017. Aravec: A set of arabic word embedding models for use in arabic nlp. Procedia Computer Science, 117:256 – 265. Arabic Computational Linguistics. find
  21. Marcos Zampieri, Preslav Nakov, Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Hamdy Mubarak, Leon Derczynski, Zeses Pitenis, and Çağrı Çöltekin. 2020. SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020). In Proceedings of SemEval. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/husain-2020-salamnet-semeval-2020-task12-deep/