Elamur Research⌕
☰
home / papers / 2020 / OSACT4 Shared Task on Offensive Language Detection: Intensive Preprocessing-Based Approach

OSACT4 Shared Task on Offensive Language Detection: Intensive Preprocessing-Based Approach

Paper · 2020 · Fatemah Husain

Summary

The paper addresses automatic detection of offensive language and hate speech in Arabic tweets, the two sub-tasks of the OSACT4 shared task. Social-media Arabic is informal and dialectal, and it is full of emoji, hashtags and spelling variants. The positive classes are also rare: about 19% of tweets are offensive and only about 5% are hate speech. The author argues that earlier Arabic work concentrated on features and classifiers, so this study puts the effort into cleaning and normalising the text before classification. The pipeline does the following:

  • converts emoji, emoticons and hashtags into text;
  • unifies variant Arabic letter forms;
  • maps dialectal words to Modern Standard Arabic (MSA);
  • replaces words with category labels;
  • performs basic cleaning.

A fairly simple SVM is then trained on character n-gram counts. A baseline without this preprocessing is compared against the full model. The system placed 3rd in offensive language detection and 1st in hate speech detection. This suggests that careful, language-aware normalisation can matter as much as model choice for noisy, imbalanced text.

Key points

  • Data come from the OSACT4 shared task. The training set has 7,000 tweets, with 1,410 offensive and 361 hate speech according to the table. The development set has 1,000 tweets, with 179 offensive and 44 hate speech. The test set has 2,000 unlabeled tweets.
  • Preprocessing steps:
    • emoji and emoticon conversion
    • hashtag conversion
    • Arabic letter normalisation
    • dialect-to-MSA normalisation
    • normalisation by word categorisation
    • general cleaning
  • The classifier is an SVM over a character-level count vectorizer using 2–5 character n-grams.
  • Results: F1 of 89.82% on Sub-Task A (offensive, 3rd place) and 95.16% on Sub-Task B (hate speech, 1st place).
  • The related work notes that a list-based obscenity approach topped out around 60% F1. The author takes this as a sign that word lists alone are weak for this task.

Interesting ideas

  • I1. Turn emoji and emoticons into word tokens instead of deleting them, so their emotional signal survives into later stages.
  • I2. Rewrite hashtags as ordinary words, because they often carry the core stance of a post.
  • I3. Normalise non-standard variants (dialect, alternative spellings) to one standard form before modelling, which reduces sparsity on noisy data.
  • I4. Collapse words that belong to the same category into a shared label, so rare members do not fragment the statistics.
  • I5. Measure the contribution of preprocessing by running the same classifier with and without it. This isolates the effect of the data pipeline from the model itself.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): The en tokenizer could map emoji and emoticons to dedicated lemmas with a fixed tag, so they can attach in the tree and feed tone or irony cues rather than vanishing.
  • I2 (tokens): Segment #BoycottThisNow-style tags into separate word tokens before tagging, so the parser and frames see real predicates instead of one opaque token.
  • I3 (tree): A pre-parse normaliser for informal English (gonna, u, cuz) would let the perceptron parser and gparse work on forms their treebanks actually cover. The original surface form can be kept for explainability.
  • I4 (learn): When building silver data or repair rules, abstract rare members of a class (numbers, names) into a class token to get more general rules from fewer examples.
  • I5 (learn): Add "pipeline-off" runs as a control in evaluation, so gains from normalisation steps are proven rather than assumed.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Kuwait UniversityKuwait City · Kuwait11 works · 147 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 works

Cite

Cite as Fatemah Husain. 2020. OSACT4 Shared Task on Offensive Language Detection: Intensive Preprocessing-Based Approach. Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{husain2020osact4,
  title = {{OSACT4 Shared Task on Offensive Language Detection: Intensive Preprocessing-Based Approach}},
  author = {Fatemah Husain},
  booktitle = {Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection},
  year = {2020},
  url = {https://aclanthology.org/2020.osact-1.8/}
}

References

26 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abozinadah, E. , Mbaziira, A., and Jones, J. (2015). Detection of Abusive Accounts with Arabic Tweets. International Journal of Knowledge Engineering, Vol. 1, No. 2. DOI: 10.7763/IJKE.2015.V1.19 ↗
  2. Abozinadah, E. (2017). Detecting Abusive Arabic Language Twitter Accounts Using a Multidimensional Analysis Model. ↗
  3. Abozinadah, E., and Jones, J. (2017). A Statistical Learning Approach to Detect Abusive Twitter Accounts. Proceedings of the International Conference on Compute and Data Analysis, 130280, 6–13. https://doi.org/10.1145/3093241.3093281 ↗
  4. Alakrot, A., Murray, L., and Nikolov, N. S. (2018a). Dataset Construction for the Detection of Anti-Social Behaviour in Online Communication in Arabic. Procedia Computer Science, 142, 174–181. https://doi.org/https://doi.org/10.1016/j.procs.2018.10.473 ↗
  5. Alakrot, A., Murray, L., and Nikolov, N. S. (2018b). Towards Accurate Detection of Offensive Language in Online Communication in Arabic. Procedia Computer Science, 142, 315–320. https://doi.org/https://doi.org/10.1016/j.procs.2018.10.491 ↗
  6. Albadi, N., Kurdi, M., and Mishra, S. (2018). Are they Our Brothers? Analysis and Detection of Religious Hate Speech in the Arabic Twittersphere. 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 69–76. https://doi.org/10.1109/ASONAM.2018.8508247 ↗
  7. Albadi, N., Kurdi, M., and Mishra, S. (2019). Investigating the effect of combining GRU neural networks with handcrafted features for religious hatred detection on Arabic Twitter space. Soc. Netw. Anal. Min., vol. 9, no. 1, p. 41. ↗
  8. Alrefaie, M. (2016). Arabic-stop-words [Github Repository]. Retrieved on January 24, 2020 from https://github.com/mohataher/arabic-stop-words ↗
  9. Bohra, A., Vijay, D., Singh, V., Akhtar, S. S., & Shrivastava, M. (2018). A Dataset of Hindi-English Code-Mixed Social Media Text for Hate Speech Detection. https://doi.org/10.18653/v1/W18-1105 ↗
  10. Chowdhury, A., Didolkar, A., Sawhney, R., and Shah, R. (2019). ARHNet - Leveraging Community Interaction for Detection of Religious Hate Speech in Arabic. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop. Association for Computational Linguistics, Florence, Italy, 273–280. https://doi.org/10.18653/v1/P19-2038 ↗
  11. Davidson, T., Warmsley, D., Macy, M., and Weber, I. (2017). Automated Hate Speech Detection and the Problem of Offensive Language. AAAI Publications, Eleventh International AAAI Conference on Web and Social Media About offensive content. arXiv.org. Retrieved from http://search.proquest.com/docview/2074118430/ ↗
  12. Derczynski, L. (2019). Offensive Language and Hate Speech Detection for Danish. arXiv.org. Retrieved from http://search.proquest.com/docview/2273023012/ ↗
  13. Goodfellow, I., Bengio, B., Courville, A. (2016). Deep Learning. MIT press. Retrieved from http://www.deeplearningbook.org/contents/ml.html ↗
  14. Habash, N. (2010). Introduction to Arabic natural language processing (Vol. 3, pp. 1–185). ↗
  15. Kapoor, R., Kumar, Y., Rajput, K., Shah, R., Kumaraguru, P., & Zimmermann, R. (2018). Mind Your Language: Abuse and Offense Detection for Code-Switched Languages. arXiv.org. Retrieved from http://search.proquest.com/docview/2111970667/ ↗
  16. Kent, S. (2018). German Hate Speech Detection on Twitter. Proceedings of GermEval 2018, 14th Conference on Natural Language Processing (KONVENS 2018) Vienna, Austria – September 21. find
  17. Kwok, I., and Wang, Y. (2013). Locate the Hate: Detecting Tweets Against Blacks. Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence, pp.1621-1622 find
  18. Mohaouchane, H., Mourhir, A., and Nikolov, N. S. (2019). Detecting Offensive Language on Arabic Social Media using Deep Learning, (December). https://doi.org/10.1109/SNAMS.2019.8931839 ↗
  19. Mubarak, H., Darwish, K., and Magdy, W. (2017). Abusive Language Detection on Arabic Social Media. Proceedings of the First Workshop on Abusive Language Online. Association for Computational Linguistics (ACL), 2017. pp. 52–56.
  20. Mustafa, R., Nawaz, M., Ferzund, J., Lali, M., Shahzad, B., and ournier-Viger, P. (2017). Early Detection of Controversial Urdu Speeches from Social Media. Data Science and Pattern Recognition, Ubiquitous International, Volume1, Number2. ISSN 2520-4165. find
  21. Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., and Chang, Y. (2016). Abusive language detection in online user content. In Proceedings of the 25th international conference on world wide web, pp. 145–153. find
  22. International World Wide Web Conferences Steering Committee, 2016 find
  23. Orasan, C. (2018). Aggressive language identification using word embeddings and sentiment features. Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying. Association for Computational Linguistics, http://hdl.handle.net/2436/621749 ↗
  24. Ozel, S., Sarac, E., Akdemir, S., and Aksu, H. (2017). Detection of cyberbullying on social media messages in Turkish. 2017 International Conference on Computer Science and Engineering (UBMK), 366– 370. https://doi.org/10.1109/UBMK.2017.8093411 ↗
  25. Pitsilis, G., Ramampiaro, H., and Langseth, H. (2018). Effective hate-speech detection in Twitter data using recurrent neural networks. Applied Intelligence, 48(12), pp. 4730–4742. https://doi.org/10.1007/s10489-018-1242-y ↗
  26. Wiedemann, G., Ruppert, E., Jindal, R., and Biemann, C. (2018). Transfer Learning from LDA to BiLSTM-CNN for Offensive Language Detection in Twitter. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/husain-2020-osact4-shared-task-offensive-language/