Elamur Research⌕
☰
home / papers / 2020 / Combining Character and Word Embeddings for the Detection of Offensive Language in Arabic

Combining Character and Word Embeddings for the Detection of Offensive Language in Arabic

Paper · 2020 · Abdullah I. Alharbi and Mark Lee

Summary

The paper tackles the detection of offensive language and hate speech in Arabic tweets for the OSACT4 shared task. Social media Arabic is noisy, ungrammatical, full of emojis and spread across many dialects with no fixed spelling. This produces many out-of-vocabulary (OOV) words that word-level embeddings cannot represent.

The task data is 10,000 annotated tweets used for two subtasks, and both are heavily imbalanced:

  • Subtask A labels tweets OFF vs NOT OFF.
  • Subtask B labels tweets HS vs NOT HS.

The authors combine two kinds of pre-trained word-level embeddings (AraVec and Mazajak) with their own character-level FastText model. They trained that model on 10M in-house tweets rich in dialectal sentiment and emotion vocabulary. Their reasoning is that the two levels capture different things:

  • Word vectors group words by meaning.
  • Character n-gram vectors group the morphological variants of a word, which can cover unseen spellings.

The systems placed 7th in Subtask A and 8th in Subtask B. The authors released the character-level model as a resource for Arabic dialect NLP.

Key points

  • Dataset size and split: 10,000 tweets divided 7000 / 1000 / 2000 into train / dev / test.
  • Class balance: about 19% of tweets are offensive (OFF, e.g. 1410 in training). Only about 5% are hate speech (HS, 361 in training), which makes Subtask B much harder.
  • Embedding resources:
    • AraVec (general Twitter, 77M tweets).
    • Mazajak (sentiment Twitter, 250M tweets).
    • The authors' FastText model (emotion Twitter, 10M tweets), built with Gensim using vector size 300, window 5, minimum count 3 and character n-grams of length 3–6.
  • Preprocessing:
    • Removal of unknown symbols, foreign letters, diacritics and punctuation, while emojis are kept as tokens with their own vectors.
    • Normalisation of hamza and ta marbuta variants.
    • A regex that splits the vocative "ya" when it is glued to the following word, a case MADAMIRA and Farasa miss.
  • Nearest-neighbour check: for an offensive query word, the character-level model returns mostly morphological variants (prefixed or suffixed forms). Mazajak returns mostly semantically related words.

Interesting ideas

  • I1. Feeding a classifier two complementary views of each word, a subword vector for form and a whole-word vector for meaning, lets unseen spellings borrow information from known relatives.
  • I2. Listing a query word's top nearest neighbours under each embedding is a cheap qualitative probe of what a representation actually encodes, morphology versus semantics.
  • I3. A small regex fixes a frequent glued-particle pattern that standard segmenters miss, and the authors openly note its false positives on names like Yasser.
  • I4. Keeping emojis as tokens instead of deleting them during cleaning preserves signals that carry tone.
  • I5. The pretraining corpus was chosen on a stated hypothesis, that emotion-heavy text correlates with offensive vocabulary, rather than simply on size.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Character n-gram similarity could give the tagger and lemmatizer a fallback for unknown or misspelled English words by mapping them to their nearest known forms, with the matched neighbour reported as the explanation.
  • I2 (learn): Neighbour lists could be added to evaluation reports for any learned vectors Mova uses, to show whether a component groups words by form or by sense before trusting it downstream.
  • I3 (tokens): Mova could use targeted split rules for informal fusions such as "gonna" or "wanna", paired with an exception list for names, keeping each split traceable to its rule.
  • I4 (frames): Emojis and similar markers kept in the token stream could serve as explicit evidence for irony and speech-act detection in prag.
  • I5 (world): When compiling lexical knowledge such as affect or evaluative values of words, sources could be selected for their domain fit, with the selection rationale recorded alongside the resource.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 works

Cite

Cite as Abdullah I. Alharbi and Mark Lee. 2020. Combining Character and Word Embeddings for the Detection of Offensive Language in Arabic. Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{alharbi2020combining,
  title = {{Combining Character and Word Embeddings for the Detection of Offensive Language in Arabic}},
  author = {Abdullah I. Alharbi and Mark Lee},
  booktitle = {Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection},
  year = {2020},
  url = {https://aclanthology.org/2020.osact-1.15/}
}

References

24 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdelali, A., Darwish, K., Durrani, N., and Mubarak, H. (2016). Farasa: A fast and furious segmenter for Arabic. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 11–16, San Diego, California, June. Association for Computational Linguistics.
  2. Abu Farha, I. and Magdy, W. (2019). Mazajak: An online Arabic sentiment analyser. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 192–198, Florence, Italy, August. Association for Computational Linguistics.
  3. Al-Hassan, A. and Al-Dossari, H. (2019). Detection of hate speech in social networks: a survey on multilingual corpus. In 6th International Conference on Computer Science and Information Technology. find
  4. Alakrot, A., Murray, L., and Nikolov, N. S. (2018). Towards accurate detection of offensive language in online communication in arabic. Procedia Computer Science, 142:315 – 320. Arabic Computational Linguistics. find
  5. Albadi, N., Kurdi, M., and Mishra, S. (2018). Are they our brothers? analysis and detection of religious hate speech in the arabic twittersphere. In 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pages 69–76, Aug. find
  6. Bojanowski, P., Grave, E., Joulin, A., and Mikolov, T. (2017). Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. ↗
  7. Bordes, A., Chopra, S., and Weston, J. (2014). Question Answering with Subgraph Embeddings. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 615–620. ↗
  8. Chen, T. and Guestrin, C. (2016). Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785–794. find
  9. Davidson, T., Warmsley, D., Macy, M., and Weber, I. (2017). Automated hate speech detection and the problem of offensive language. In Eleventh international aaai conference on web and social media. ↗
  10. Devlin, J., Zbib, R., Huang, Z., Lamar, T., Schwartz, R., and Makhoul, J. (2014). Fast and robust neural network joint models for statistical machine translation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1370–1380.
  11. Duwairi, R. and El-Orfali, M. (2014). A study of the effects of preprocessing strategies on sentiment analysis for arabic text. Journal of Information Science, 40(4):501–513. ↗
  12. Hochreiter, S. and Schmidhuber, J. (1997). Long short-term memory. Neural Comput., 9(8):1735–1780, November. ↗
  13. Kumar, R., Ojha, A. K., Malmasi, S., and Zampieri, M. (2018). Benchmarking aggression identification in social media. In Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC-2018), pages 1–11, Santa Fe, New Mexico, USA, August. Association for Computational Linguistics.
  14. Lin, C.-C., Ammar, W., Dyer, C., and Levin, L. (2015). Unsupervised POS Induction with Word Embeddings. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1311– 1316. ↗
  15. Malmasi, S. and Zampieri, M. (2017). Detecting hate speech in social media. In Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017, pages 467–472, Varna, Bulgaria, September. INCOMA Ltd. ↗
  16. Mubarak, H. and Darwish, K. (2019). Arabic offensive language classification on twitter. In International Conference on Social Informatics, pages 269–276. Springer. find
  17. Mubarak, H., Darwish, K., and Magdy, W. (2017). Abusive language detection on arabic social media. In Proceedings of the First Workshop on Abusive Language Online, pages 52–56.
  18. Mubarak, H., Darwish, K., Magdy, W., Elsayed, T., and Al-Khalifa, H. (2020). Overview of osact4 arabic offensive language detection shared task. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT), volume 4.
  19. Pasha, A., Al-Badrashiny, M., Diab, M., El Kholy, A., Eskander, R., Habash, N., Pooleery, M., Rambow, O., and Roth, R. (2014). MADAMIRA: A fast, comprehensive tool for morphological analysis and disambiguation of Arabic. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 1094–1101, Reykjavik, Iceland, May. European Language Resources Association (ELRA).
  20. Salameh, M., Bouamor, H., and Habash, N. (2018). Fine-grained Arabic dialect identification. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1332–1344, Santa Fe, New Mexico, USA, August. Association for Computational Linguistics.
  21. Soliman, A. B., Eissa, K., and El-Beltagy, S. R. (2017). Aravec: A set of arabic word embedding models for use in arabic nlp. Procedia Computer Science, 117:256–265. find
  22. Waseem, Z., Davidson, T., Warmsley, D., and Weber, I. (2017). Understanding abuse: A typology of abusive language detection subtasks. In Proceedings of the First Workshop on Abusive Language Online, pages 78–84, Vancouver, BC, Canada, August. Association for Computational Linguistics.
  23. Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., and Kumar, R. (2019). SemEval-2019 task 6: Identifying and categorizing offensive language in social media (OffensEval). In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 75–86, Minneapolis, Minnesota, USA, June. Association for Computational Linguistics.
  24. Zhang, J., Liu, S., Li, M., Zhou, M., and Zong, C. (2014). Bilingually-constrained phrase embeddings for machine translation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 111–121. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alharbi-2020-combining-character-word-embeddings-detection/