Elamur Research⌕
☰
home / papers / 2020 / Toxic, Hateful, Offensive or Abusive? What Are We Really Classifying? An Empirical Analysis of Hate Speech Datasets

Toxic, Hateful, Offensive or Abusive? What Are We Really Classifying? An Empirical Analysis of Hate Speech Datasets

Summary

Research on detecting hate speech and related phenomena has produced many corpora, but each uses its own labels and definitions. As a result, it is unclear whether "toxic", "offensive", "abusive" or "aggressive" mean the same thing across resources, and models trained on one corpus often fail on another. The authors examine six public English datasets: Waseem, Davidson, Amievalita, Hateval, TRAC and Toxkaggle. First, they compare the annotation definitions and map equivalent categories onto a shared label set. Second, they represent each class with FastText word vectors and measure how similar classes are, both within one dataset and across datasets. Third, they run all datasets through the Perspective API toxicity classifier and compare F1 scores by category.

Hate-related categories turn out to be fairly coherent with each other, and so do aggression-related ones. The broad "catch-all" categories, such as toxicity and aggression, do not align across datasets, and classifier performance varies widely with them. The paper concludes that most corpora are not directly compatible. It proposes annotation and documentation guidelines to make future resources reusable.

Key points

  • Datasets and sources. Six datasets were analysed:
    • Waseem: 16,914 tweets
    • Davidson: 24,802 tweets
    • Amievalita: 4,000 tweets
    • Hateval: 9,000 tweets
    • TRAC: 12,000 Facebook posts
    • Toxkaggle: 159,571 Wikipedia comments
  • Class balance varies widely. The share of negative (clean) samples ranges from 16.79% in Davidson to 68.02% in Waseem. Only 10.16% of Toxkaggle messages carry any negative label.
  • Hierarchy violations in Toxkaggle. Some messages carry a subtype label but lack the parent "toxic" label: 317 obscene, 301 insult, 54 identity hate and 22 threat.
  • Perspective API results.
    • Toxicity is detected better on Davidson than on Toxkaggle, and TRAC "aggression" scores even lower. This suggests TRAC aggression should not be merged with toxicity.
    • Within Toxkaggle, large classes (toxic, obscene, insult) score higher than small ones (severe toxic, threat, hate speech).
    • Covert aggression is recognised worse than overt aggression.
  • Definition quality. Definitions in Waseem, Amievalita and Hateval are enumerative and precise. Those in Davidson and TRAC are vague, and Toxkaggle provides no explicit definitions at all.

Interesting ideas

  • I1. Explicit label-standardization table. Before comparing or merging corpora, build a mapping of equivalent categories, for example treating sexism and misogyny as one class, or creating a union "toxicity" label from Davidson's hate and offensive classes. This makes every cross-dataset decision visible and open to debate, which helps in any multi-source labelling project.
  • I2. Embedding-based category map. Represent each class by vectors of its messages and compare classes within and across datasets, for instance with a PCA projection. This shows empirically whether two labels with different names behave alike. The same check can position a newly proposed category against existing ones before committing to it.
  • I3. Off-the-shelf classifier as a probe. Run one fixed pretrained classifier over many datasets and compare per-category scores. Divergent results expose labels that look similar on paper but are annotated differently, which works as a cheap compatibility test for any shared taxonomy.
  • I4. Automatic consistency checks on hierarchical labels. Search for instances that have a subtype label without its declared parent. This quickly reveals annotation noise that would otherwise go unnoticed in a large corpus.
  • I5. Document sampling, balancing and authorship. Record the source, target groups, context, time and location, along with how classes were balanced. Author information also lets you check whether models are learning individual writing styles rather than the phenomenon itself. Such metadata is valuable for any dataset meant for reuse.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{fortuna2020toxic,
  title = {{Toxic, Hateful, Offensive or Abusive? What Are We Really Classifying? An Empirical Analysis of Hate Speech Datasets}},
  author = {Fortuna, Paula and Soler, Juan and Wanner, Leo},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  doi = {10.5281/zenodo.4529173},
  url = {https://doi.org/10.5281/zenodo.4529173}
}

References

21 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anzovino, M., Fersini, E., and Rosso, P. (2018). Automatic identification and classification of misogynistic language on twitter. In International Conference on Applications of Natural Language to Information Systems, pages 57–64. Springer. find
  2. Arango, A., Pérez, J., and Poblete, B. (2019). Hate speech detection is not as easy as you may think: A closer look at model validation. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 45–54. ACM. find
  3. Basile, V., Bosco, C., Fersini, E., Nozza, D., Patti, V., Pardo, F. M. R., Rosso, P., and Sanguinetti, M. (2019). Semeval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 54–63. find
  4. Bojanowski, P., Grave, E., Joulin, A., and Mikolov, T. (2016). Enriching word vectors with subword information. arXiv preprint arXiv:1607.04606. find
  5. Davidson, T., Warmsley, D., Macy, M., and Weber, I. (2017). Automated hate speech detection and the problem of offensive language. In Eleventh international aaai conference on web and social media. find our card
  6. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. find
  7. Fersini, E., Nozza, D., and Rosso, P. (2018). Overview of the evalita 2018 task on automatic misogyny identification (ami). In EVALITA@ CLiC-it. find
  8. Fortuna, P. Soler-Company, J. and Nunes, S. (2019). Stop propaghate at semeval-2019 tasks 5 and 6: Are abusive language classification results reproducible? In Proceedings of the 13th International Workshop on Semantic Evaluation. find
  9. Fortuna, P., da Silva, J. R., Wanner, L., Nunes, S., et al. (2019). A hierarchically-labeled portuguese hate speech dataset. In Proceedings of the Third Workshop on Abusive Language Online, pages 94–104. find
  10. Jigsaw. (2019a). Perspective api. Available in https://github.com/conversationai/perspectiveapi, accessed last time in November 2019. ↗
  11. Jigsaw. (2019b). Toxic comment classification challenge: Identify and classify toxic online comments. Available in https://www.kaggle.com/c/jigsaw -toxic-comment-classification-challenge, accessed last time in November 2019. ↗
  12. Kumar, R., Ojha, A. K., Malmasi, S., and Zampieri, M. (2018). Benchmarking Aggression Identification in Social Media. In Proceedings of the First Workshop on Trolling, Aggression and Cyberbulling (TRAC), Santa Fe, USA. find
  13. Mikolov, T., Grave, E., Bojanowski, P., Puhrsch, C., and Joulin, A. (2018). Advances in pre-training distributed word representations. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018). find our card
  14. Pearson, K. (1901). Liii. on lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 2(11):559–572. find
  15. Pennington, J., Socher, R., and Manning, C. D. (2014). Glove: Global vectors for word representation. In Proceedings of EMNLP. find
  16. Santucci, V., Spina, S., Milani, A., Biondi, G., and Di Bari, G. (2018). Detecting hate speech for italian language in social media. In EVALITA 2018, co-located with the Fifth Italian Conference on Computational Linguistics (CLiC-it 2018), volume 2263. find
  17. Swamy, S. D., Jamatia, A., and Gambäck, B. (2019). Studying generalisability across abusive language detection datasets. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 940–950, Hong Kong, China, November. Association for Computational Linguistics. find
  18. (2019). Challenges and frontiers in abusive content detection. In Proceedings of the Third Workshop on Abusive Language Online. Association for Computational Linguistics. find
  19. Waseem, Z. and Hovy, D. (2016). Hateful symbols or hateful people? predictive features for hate speech detection on twitter. In Proceedings of the NAACL Student Research Workshop, pages 88–93, San Diego, California, June. Association for Computational Linguistics. find
  20. Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., and Kumar, R. (2019a). Predicting the type and target of offensive posts in social media. arXiv preprint arXiv:1902.09666. find our card
  21. Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., and Kumar, R. (2019b). Semeval-2019 task 6: Identifying and categorizing offensive language in social media (offenseval). arXiv preprint arXiv:1903.08983. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/fortuna-2020-toxic-hateful-offensive-or-abusive/