Elamur Research
☰
home / papers / 2019 / Detection of Abusive Language: the Problem of Biased Datasets

Detection of Abusive Language: the Problem of Biased Datasets

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/N19-1060.pdf

Summary

The paper argues that published scores for abusive language detection are often inflated by how the benchmark datasets were collected. Abusive posts are rare on social media, so dataset creators boost them, either by enriching a random sample (boosted random sampling) or by selecting content through query words or chosen pages (biased sampling). The authors compare six widely used English datasets (Kaggle, Founta, Razavi, Warner, Waseem, Kumar). They binarize every dataset into abuse vs. no-abuse, estimate the share of explicit abuse with an abusive-word lexicon, and train the same FastText classifier on all of them.

Datasets built by biased sampling contain more implicit abuse. Waseem scores unexpectedly well despite this, and the analysis traces that result to topic bias and author bias rather than real abuse detection. Removing a handful of biased words lowers performance, and cross-domain tests reveal weak generalization. The authors recommend cross-domain evaluation, PMI-based checks for biased words, targeted resampling and per-author caps.

Key points

  • Explicit abuse and in-domain F1 (10-fold CV, macro-F1, FastText):
    • Kaggle: 76.9% explicit, F1 88.2
    • Founta: 75.9% explicit, F1 87.3
    • Razavi: 64.7% explicit, F1 83.3
    • Warner: 51.3% explicit, F1 71.8
    • Waseem: 44.4% explicit, F1 80.5
    • Kumar: 32.7% explicit, F1 70.4
  • Topic bias: By PMI, the words most associated with abuse in Waseem are topical rather than offensive (e.g., commentator, football, sport). In Founta, the top words are actual slurs.
  • Word removal on Waseem: F1 falls from 80.49 to 76.33 without the 17 query words and to 74.72 without 25 topic words. These words make up only about 0.2% of the vocabulary.
  • Author bias in Waseem:
    • About 99% of racist tweets come from a single author, Vile Islam.
    • Over 70% of sexist tweets come from two accounts.
    • As a result, author-feature gains reported in earlier work on this dataset are suspect.
  • Cross-domain results: Every model performs worse on other datasets. Kaggle, Founta and Razavi still transfer reasonably among each other, reaching up to about 85.8 F1. Warner, Waseem and Kumar mostly score in the 49–66 F1 range.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

118 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 112 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n19-1060

@inproceedings{wiegand2019detection,
  title = {{Detection of Abusive Language: the Problem of Biased Datasets}},
  author = {Wiegand, Michael and Ruppenhofer, Josef and Kleinbauer, Thomas},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  year = {2019},
  doi = {10.18653/v1/n19-1060},
  url = {https://doi.org/10.18653/v1/n19-1060}
}

References

25 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sweeta Agrawal and Amit Awekar. 2018. Deep Learning for Detecting Cyberbullying Across Multiple Social Media Platforms. In Proceedings of the European Conference in Information Retrieval (ECIR). Grenoble, France, pages 141–153. 5 Please note, however, that in the case of the Waseem-dataset, this for… find
  2. Pinkesh Badjatiya, Shashank Gupta, Manish Gupta, and Vasudeva Varma. 2017. Deep Learning for Hate Speech Detection in Tweets. In Proceedings of the International Conference on World Wide Web (WWW). Perth, Australia, pages 759–760. find
  3. Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. Reducing Gender Bias in Abusive Language Detection. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). Brussels, Belgium, pages 2799–2804. find our card
  4. Peter Bourgonje, Julian Moreno-Schneider, Ankit Srivastava, and Georg Rehm. 2017. Automatic Classification of Abusive Language and Personal Attacks in Various Forms of Online Communication. In Proceedings of the German Society for Computational Linguistics and Language Technology (GSCL). Springer, B… find
  5. Georgios K. Pitsilis, Heri Ramampiaro, and Helge Langseth. 2018. Detecting offensive language in tweets using deep learning. In arXiv preprint arXiv: 1801.04433. — [arXiv:1801.04433](https://arxiv.org/abs/1801.04433) find
  6. Jing Qian, Mai ElSherief, Elizabeth M. Belding, and William Yang Wang. 2018. Leveraging intra-user and inter-user representation learning for automated hate speech detection. In Proceedings of the Human Language Technology Conference of the North American Chapter of the ACL (HLT/NAACL). New Orleans,… find
  7. Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2017. Measuring and Mitigating Unintended Bias in Text Classification. In Proceedings of the National Conference on Artificial Intelligence (AAAI). San Francisco, CA, USA. find
  8. Amir Hossein Razavi, Diana Inkpen, Sasha Uritsky, and Stan Matwin. 2010. Offensive Language Detection Using Multi-level Classification. In Proceedings of the Canadian Conference on Artificial Intelligence. Ottawa, Canada, pages 16–27. find
  9. Antigoni-Maria Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018. Large Scale Crowdsourcing and Characterization of Twitter Abusive Behaviour. In Proceedings of the International… find
  10. Magnus Sahlgren, Tim Isbister, and Fredrik Olsson. 2018. Learning Representations for Detecting Abusive Language. In Proceedings of the Workshop on Abusive Language Online (ALW). Brussels, Belgium, pages 115–123. find
  11. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. Bag of Tricks for Efficient Text Classification. In Proceedings of the Conference on European Chapter of the Association for Computational Linguistics (EACL). Valencia, Spain, pages 427–431. find
  12. Sima Sharifirad, Borna Jafarpour, and Stan Matwin. 2018. Boosting Text Classification Performance on Sexist Tweets by Text Augmentation and Text Generation Using a Combination of Knowledge Graphs. In Proceedings of the Workshop on Abusive Language Online (ALW). Brussels, Belgium, pages 107– 101. find
  13. Mladen Karan and Jan Snajder. 2018. Cross-Domain Detection of Abusive Language Online. In Proceedings of the Workshop on Abusive Language Online (ALW). Brussels, Belgium, pages 132–137. find
  14. Elise Fehn Unsvåg and Björn Gambäck. 2018. The Effects of User Features on Twitter Hate Speech Detection. In Proceedings of the Workshop on Abusive Language Online (ALW). Brussels, Belgium, pages 75–85. find
  15. Rohan Kshirsagar, Tyrus Cukuvac, Kathleeen McKeown, and Susan McGregor. 2018. Predictive Embeddings for Hate Speech Detection on Twitter. In Proceedings of the Workshop on Abusive Language Online (ALW). Brussels, Belgium, pages 26–32. find
  16. William Warner and Julia Hirschberg. 2012. Detecting Hate Speech on the World Wide Web. In Proceedings of the Workshop on Language in Social Media (LSM). Montréal, Canada, pages 19–26. find
  17. Ritesh Kumar, Atul Kr. Ojha, Shervin Malmasi, and Marcos Zampieri. 2018. Benchmarking Aggression Identification in Social Media. In Proceedings of the Workshop on Trolling, Aggression and Cyberbullying (TRAC). Santa Fe, NM, USA, pages 1–11. find
  18. Zeerak Waseem, Thomas Davidson, Dana Warmsley, and Ingmar Weber. 2017. Understanding Abuse: A Typology of Abusive Language Detection Subtasks. In Proceedings of the ACL-Workshop on Abusive Language Online. Vancouver, BC, Canada, pages 78–84. find our card
  19. Pushkar Mishra, Marco Del Tredici, Helen Yannakoudakis, and Ekaterina Shutova. 2018a. Author Profiling for Abuse Detection. In Proceedings of the International Conference on Computational Linguistics (COLING). Santa Fe, NM, USA, pages 1088–1098. find
  20. Zeerak Waseem and Dirk Hovy. 2016. Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter. In Proceedings of the Human Language Technology Conference of the North American Chapter of the ACL – Student Research Workshop). San Diego, CA, USA, pages 88– 93. find
  21. Pushkar Mishra, Helen Yannakoudakis, and Ekaterina Shutova. 2018b. Neural Character-based Composition Models for Abuse Detectiion. In Proceedings of the Workshop on Abusive Language Online (ALW). Brussels, Belgium, pages 1–10. find
  22. Michael Wiegand, Josef Ruppenhofer, Anna Schmidt, and Clayton Greenberg. 2018a. Inducing a Lexicon of Abusive Words – A Feature-Based Approach. In Proceedings of the Human Language Technology Conference of the North American Chapter of the ACL (HLT/NAACL). New Orleans, LA, USA, pages 1046–1056. find
  23. Michael Wiegand, Melanie Siegel, and Josef Ruppenhofer. 2018b. Overview of the GermEval 2018 Shared Task on the Identification of Offensive Language. In Proceedings of the GermEval Workshop. Vienna, Austria, pages 1–10. find
  24. Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017. Ex Machina: Personal Attacks Seen at Scale. In Proceedings of the International Conference on World Wide Web (WWW). Perth, Australia, pages 1391–1399. find
  25. Haoti Zhong, Hao Li, Anna Cinzia Squicciarini, Sarah Michele Rajtmajer, Christopher Griffin, David J. Miller, and Cornelia Caragea. 2016. Content-Driven Detection of Cyberbullying on the Instagram Social Network. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)… find