Elamur Research⌕
☰
home / papers / 2020 / Offensive Language and Hate Speech Detection for Danish

Offensive Language and Hate Speech Detection for Danish

Summary

Most work on automatically detecting abusive online content targets English, even though the problem exists in every language. This paper addresses the lack of Danish resources by building DK HATE, a collection of user comments from Reddit, Ekstra Bladet and Facebook (2018). The authors describe it as the first Danish dataset of its kind. Comments are labelled with the hierarchical scheme of Zampieri et al. (2019a), the framing behind OffensEval. That scheme asks three questions in turn: whether a post is offensive, whether the offence is aimed at someone, and whether the target is a person, a group or something else. The authors build four classification systems that work for both English and Danish and compare them across all three sub-tasks. The best Danish macro-F1 scores are 0.70 for offensiveness, 0.73 for targeting and 0.63 for target type. This shows that a compact, carefully annotated non-English resource can support usable detectors. Because the label scheme is shared, the results can also be compared directly with English.

Key points

  • New resource: DK HATE contains Danish social media comments from Reddit, Ekstra Bladet and Facebook, annotated with the OffensEval-style schema of Zampieri et al. (2019a, 2019b).

  • Three-level taxonomy:

    • Sub-task A labels posts NOT or OFF.
    • Sub-task B splits OFF posts into TIN (targeted) or UNT (untargeted).
    • Sub-task C assigns a target to TIN posts: IND, GRP or OTH.

  • Best macro-F1 per sub-task:

    Sub-taskEnglishDanish
    A0.740.70
    B0.620.73
    C0.560.63

  • Systems: four classifiers, each built to run on both English and Danish data.

  • Appendices:

    • A Danish hate speech lexicon collected from Reddit, mixing slurs, insults and politically loaded words.
    • A data statement giving language variety (da-DK), speaker information and annotator demographics: male, aged 25–40, native speakers of Icelandic or English.

Interesting ideas

  • I1. Borrow an existing schema instead of inventing one. Adopting an established annotation scheme makes results on a new language directly comparable to English benchmarks. This avoids the incompatible-definitions problem the authors describe in earlier abuse datasets, and it would carry over to a Ukrainian offensive-language corpus.
  • I2. Map labels onto legal and social categories. Coarse labels correspond to familiar concepts: targeted insults at individuals approximate cyberbullying, and attacks on groups approximate hate speech. One annotation pass therefore supports several downstream uses without separate datasets.
  • I3. Annotate hierarchically. Each finer label is applied only to posts that passed the previous level. This lowers annotation effort and gives naturally staged classifiers, at the cost of smaller data for the deeper sub-tasks.
  • I4. Publish a data statement. Documenting speakers, platforms and especially annotator background (here non-native, all-male) makes possible biases visible to future users of the dataset. It is a cheap practice worth adopting for any new corpus.
  • I5. Release a curated lexicon with the data. A platform-specific list of offensive and loaded terms can serve as a seed for locating rare offensive content in large comment streams. It is also a reusable resource on its own.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

83 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 80 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{sigurbergsson2020offensive,
  title = {{Offensive Language and Hate Speech Detection for Danish}},
  author = {Sigurbergsson, Gudbjartur Ingi and Derczynski, Leon},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  url = {https://aclanthology.org/2020.lrec-1.430/}
}

References

35 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Badjatiya, P., Gupta, S., Gupta, M., and Varma, V. (2017). Deep learning for hate speech detection in tweets. In Proceedings of the 26th International Conference on World Wide Web Companion, pages 759–760. find
  2. Banks, J. (2010). Regulating hate speech online. International Review of Law, Computers & Technology, 24(3):233–239. ↗
  3. Bender, E. M. and Friedman, B. (2018). Data statements for natural language processing: Toward mitigating system bias and enabling better science. Transactions of the Association for Computational Linguistics, 6:587–604. ↗
  4. Black, M. B. (2018). ’Hygge racism’:" noget som man nok bruger mere end man tænker over". A qualitative study of well-intentioned racism. Master’s thesis, Sociologiska Institutionen, Lund Universitet. find
  5. Chollet, F. et al. (2015). Keras. https://keras.io. ↗
  6. Davidson, T., Warmsley, D., Macy, M., and Weber, I. (2017). Automated hate speech detection and the problem of offensive language. In Eleventh International AAAI Conference on Web and Social Media. ↗ our card
  7. Derczynski, L., Maynard, D., Aswani, N., and Bontcheva, K. (2013). Microblog-genre noise and impact on semantic annotation accuracy. In Proceedings of the 24th ACM Conference on Hypertext, pages 21–30. find
  8. Derczynski, L., Albert-Lindqvist, T. O., Bendsen, M. V., Inie, N., Petersen, J. E., and Petersen, V. D. (2019). Kvinder nedgøres oftere end mænd i politiske debatter på sociale medier. TjekDet / Mandag Morgen. find
  9. EU. (2008). European Union Council Framework Decision 2008/913/JHA. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri= LEGISSUM%3Al33178. Accessed: 2019-05-29. ↗
  10. Gregorie, T. M. (2001). Cyberstalking: Dangers on the information superhighway. National Center for Victims of crime. find
  11. Hamers, L., Hemeryck, Y., Herweyers, G., Janssen, M., Keters, H., Rousseau, R., and Vanhoutte, A. (1989). Similarity measures in scientometric research: The Jaccard index versus Salton’s cosine formula. Information Processing and Management, 25(3):315–18. ↗
  12. Hutto, C. J. and Gilbert, E. (2014). Vader: A parsimonious rule-based model for sentiment analysis of social media text. In Proceedings of the International Conference on Weblogs and Social Media (ICWSM). AAAI. find
  13. Joseph, S. and Castan, M. (2013). The international covenant on civil and political rights: cases, materials, and commentary. Oxford University Press. find
  14. Kingma, D. P. and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. ↗
  15. Kirkedal, A., Plank, B., Derczynski, L., and Schluter, N. (2019). The Lacunae of Danish Natural Language Processing. In Proceedings of the Nordic Conference on Computational Linguistics (NODALIDA). find
  16. Le, Q. and Mikolov, T. (2014). Distributed representations of sentences and documents. In Proceedings of the International Conference on Machine Learning, pages 1188– 1196. ↗ our card
  17. Maynard, D. G. and Greenwood, M. A. (2014). Who cares about sarcastic tweets? Investigating the impact of sarcasm on sentiment analysis. In Proceedings of the International Conference on Language Resources and Evaluation (LREC). ELRA. find our card
  18. Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. ↗
  19. Mikolov, T., Grave, E., Bojanowski, P., Puhrsch, C., and Joulin, A. (2018). Advances in pre-training distributed word representations. In Proceedings of the International Conference on Language Resources and Evaluation (LREC). find our card
  20. Müller, K. and Schwarz, C. (2018). Fanning the flames of hate: Social media and hate crime. Available at SSRN 3082972. ↗
  21. Nielsen, F. Å. (2011). A new anew: Evaluation of a word list for sentiment analysis in microblogs. arXiv preprint arXiv:1103.2903. ↗
  22. Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., and Chang, Y. (2016). Abusive language detection in online user content. In Proceedings of the 25th international conference on the World Wide Web, pages 145–153. International World Wide Web Conferences Steering Committee. find
  23. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12(Oct):2825–2830. ↗
  24. Pennington, J., Socher, R., and Manning, C. (2014). GloVe: Global Vectors for word representation. In Proceedings of the 2014 conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532– 1543. find
  25. Schmidt, A. and Wiegand, M. (2017). A survey on hate speech detection using natural language processing. In Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media, pages 1–10. find our card
  26. Sennrich, R., Haddow, B., and Birch, A. (2016). Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1715–1725. find
  27. Sigurbergsson, G. I. (2019). Offensive and hate speech detection. Master’s thesis, IT University of Copenhagen. find
  28. Straffeloven. (2011). Straffeloven § 266 b. https://danskelove.dk/straffeloven/266b. Accessed: 2019-05-29. ↗
  29. Van Hee, C., Lefever, E., Verhoeven, B., Mennes, J., Desmet, B., De Pauw, G., Daelemans, W., and Hoste, V. (2015a). Detection and fine-grained classification of cyberbullying events. In Proceedings of the international conference Recent Advances in Natural Language Processing, pages 672–680. find
  30. Van Hee, C., Verhoeven, B., Lefever, E., De Pauw, G., Hoste, V., and Daelemans, W. (2015b). Guidelines for the fine-grained analysis of cyberbullying. Technical report, Language and Translation Technology Team, Ghent University. find
  31. Vidgen, B., Harris, A., Nguyen, D., Tromble, R., Hale, S., and Margetts, H. (2019). Challenges and frontiers in abusive content detection. In Proceedings of the Third Workshop on Abusive Language Online, pages 80–93. find
  32. Waseem, Z. and Hovy, D. (2016). Hateful symbols or hateful people? Predictive features for hate speech detection on twitter. In Proceedings of the NAACL student research workshop, pages 88–93. find
  33. Waseem, Z., Davidson, T., Warmsley, D., and Weber, I. (2017). Understanding abuse: A typology of abusive language detection subtasks. In Proceedings of the First Workshop on Abusive Language Online, pages 78–84, Vancouver, BC, Canada, August. Association for Computational Linguistics. find our card
  34. Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., and Kumar, R. (2019a). Predicting the type and target of offensive posts in social media. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1415–1420, Minneapolis, Minnesota, June. Association for Computational Linguistics. find our card
  35. Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., and Kumar, R. (2019b). SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval). In Proceedings of SemEval. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sigurbergsson-2020-offensive-language-hate-speech-detection/