Elamur Research⌕
☰
home / papers / 2022 / Offensive Language Detection in Under-resourced Algerian Dialectal Arabic Language

Offensive Language Detection in Under-resourced Algerian Dialectal Arabic Language

Summary

The paper tackles automatic detection of offensive and abusive Facebook comments written in Algerian dialectal Arabic. This under-resourced variety blends Arabic with Berber, French and other borrowings, and it appears in both Arabic script and Romanised Arabizi. The only earlier Algerian resource was small and limited to hate speech against women, so the authors built a new general-purpose corpus, DziriOFN. They crawled 10,258 comments from public sports and politics pages that covered contentious topics. Five native speakers labelled each comment as offensive, abusive or normal over three annotation rounds, and about 8.7k texts were kept. The authors then compared classic and neural text classifiers on binary and three-way versions of the task. Simple tf-idf models (SVM, Multinomial NB) came out on top, reaching roughly 0.75 accuracy for binary and 0.67 for three-class classification. Neural models trailed, which the authors attribute to the small data size and the dialect's unstable spelling. The corpus and crawler give a baseline resource for a poorly covered dialect.

Key points

  • DziriOFN keeps 3,227 offensive, 1,334 abusive and 4,188 normal texts. Another 1,509 of the 10,258 crawled comments were discarded as unclear, foreign-language or hard to label. Each text also carries a language tag: MSA, dialect or mixed.
  • Annotation ran in three rounds. Two independent passes covered the whole corpus, with the second pass split among annotators #3–#5. A third pass re-judged only the texts where the first two passes disagreed.
  • The authors compared each annotator's labels with the final labels. Agreement on normal texts was high. The annotator group from the second pass rarely matched the final labels for offensive (~27.5%) and abusive (~16.9%) texts, which the authors see as an effect of training or cultural norms.
  • The setup used a 90/10 train/test split, word unigrams with tf-idf, and scikit-learn defaults. Binary accuracy was Multinomial NB 0.752, SVM 0.744 and Gaussian NB 0.710. Three-class accuracy was SVM 0.669, Multinomial NB 0.662 and Gaussian NB 0.518.
  • Neural results were weaker. FastText scored 0.716 / 0.648 (binary / three-class), BiLSTM 0.520 / 0.400, and CNN (512 filters, sizes 3–5, dropout 0.5) 0.523 / 0.347. The corpus and the Javascript Facebook crawler are on GitHub.

Interesting ideas

  • I1. Two full independent labelling passes, followed by adjudication of only the conflicting items, spend expert effort just where it is needed. — For Mova (learn): Silver trees agreed by the witness parsers could be accepted directly, with only the contested arcs sent to the LLM teacher tier for a ruling.
  • I2. Comparing each annotator's labels with the final labels, broken down by category, exposes who is reliable for which kind of item. — For Mova (learn): Per-relation agreement of each witness parser with the final silver data would show which parser to trust for which dependency labels, and could be used to weight their votes.
  • I3. Running the same task at two granularities (merged vs. separate harmful classes) shows how much of the error comes from fine distinctions rather than the coarse split. — For Mova (frames): Speech-act and irony labels could be evaluated both coarsely and finely, to see whether mistakes sit at the top-level decision or between close subtypes.
  • I4. The authors suggest that character n-grams could handle non-standard spelling, since variants mostly differ in vowels while keeping shared characters; they did not test this. — For Mova (tokens): Character-level fallbacks in the tagger and lemmatiser could map misspelled or informal English forms to known lemmas instead of treating them as unknown words.
  • I5. With a few thousand examples, default tf-idf linear models beat CNN and BiLSTM, so cheap baselines are a necessary reference point. — For Mova (learn): A shallow lexical baseline could serve as a negative control, so that any explainable-pipeline component must show it beats it rather than merely matching it.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2203.10024 · arXiv 2203.10024

@misc{boucherit2022offensive,
  title = {{Offensive Language Detection in Under-resourced Algerian Dialectal Arabic Language}},
  author = {Boucherit, Oussama and Abainia, Kheireddine},
  year = {2022},
  doi = {10.48550/arxiv.2203.10024},
  eprint = {2203.10024},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2203.10024}
}

References

20 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alharbi, A. I., & Lee, M. (2020, May). Combining character and word embeddings for the detection of offensive language in Arabic. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection (pp. 91-96). find
  2. Alshaalan, R., & Al-Khalifa, H. (2020, December). Hate speech detection in saudi twittersphere: A deep learning approach. In Proceedings of the Fifth Arabic Natural Language Processing Workshop (pp. 12-23). find
  3. Badjatiya, P., Gupta, S., Gupta, M., & Varma, V. (2017, April). Deep learning for hate speech detection in tweets. In Proceedings of the 26th international conference on World Wide Web companion (pp. 759-760). find
  4. Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017). Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5, 135-146. find
  5. De Souza, G. A., & Da Costa-Abreu, M. (2020, July). Automatic offensive language detection from twitter data using machine learning and feature selection of metadata. In 2020 International Joint Conference on Neural Networks (IJCNN) (pp. 1-6). find
  6. Djandji, M., Baly, F., Antoun, W., & Hajj, H. (2020, May). Multi-task learning using AraBert for offensive language detection. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection (pp. 97-101). find
  7. Farha, I. A., & Magdy, W. (2020, May). Multitask learning for Arabic offensive language and hate-speech detection. In Proceedings of the 4th workshop on open-source Arabic corpora and processing tools, with a shared task on offensive language detection (pp. 86-90). find
  8. Gitari, N. D., Zuping, Z., Damien, H., & Long, J. (2015). A lexicon-based approach for hate speech detection. International Journal of Multimedia and Ubiquitous Engineering, 10(4), 215-230. find
  9. Guellil, I., Adeel, A., Azouaou, F., Boubred, M., Houichi, Y., & Moumna, A. A. (2021). Sexism detection: The first corpus in Algerian dialect with a code-switching in Arabic/French and English. arXiv preprint arXiv:2104.01443. ↗ our card
  10. Haddad, H., Mulki, H., & Oueslati, A. (2019, October). T-hsab: A tunisian hate speech and abusive dataset. In International Conference on Arabic Language Processing (pp. 251-263). find
  11. Haidar, B., Chamoun, M., & Serhrouchni, A. (2017). A multilingual system for cyberbullying detection: Arabic content detection using machine learning. Advances in Science, Technology and Engineering Systems Journal, 2(6), 275-284. find
  12. Husain, F. (2020). Arabic offensive language detection using machine learning and ensemble machine learning approaches. arXiv preprint arXiv:2005.08946. ↗ our card
  13. Mubarak, H., Darwish, K., & Magdy, W. (2017, August). Abusive language detection on Arabic social media. In Proceedings of the first workshop on abusive language online (pp. 52-56). find our card
  14. Mulki, H., Haddad, H., Ali, C. B., & Alshabani, H. (2019, August). L-hsab: A levantine twitter dataset for hate speech and abusive language. In Proceedings of the third workshop on abusive language online (pp. 111-118). find
  15. Nayel, H. A., & Shashirekha, H. L. (2019, December). DEEP at HASOC2019: A Machine Learning Framework for Hate Speech and Offensive Language Detection. In FIRE (Working Notes) (pp. 336-343). find
  16. Otiefy, Y., Abdelmalek, A., & Hosary, I. E. (2020). WOLI at SemEval-2020 Task 12: Arabic offensive language identification on different Twitter datasets. arXiv preprint arXiv:2009.05456. ↗
  17. Pitenis, Z., Zampieri, M., & Ranasinghe, T. (2020). Offensive language identification in Greek. arXiv preprint arXiv:2003.07459. ↗ our card
  18. Santucci, V., Spina, S., Milani, A., Biondi, G., & Di Bari, G. (2018). Detecting hate speech for Italian language in social media. In EVALITA 2018, co-located with the Fifth Italian Conference on Computational Linguistics (CLiC-it 2018) (Vol. 2263). find
  19. Sutejo, T. L., & Lestari, D. P. (2018, November). Indonesia hate speech detection using deep learning. In 2018 International Conference on Asian Language Processing (IALP) (pp. 39 -43). find
  20. Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., & Kumar, R. (2019). Predicting the type and target of offensive posts in social media. arXiv preprint arXiv:1902.09666. ↗ our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/boucherit-2022-offensive-language-detection-under-resourced/