Elamur Research
☰
home / papers / 2019 / Racial Bias in Hate Speech and Abusive Language Detection Datasets

Racial Bias in Hate Speech and Abusive Language Detection Datasets

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/W19-3504.pdf

Summary

Classifiers for detecting abusive language are increasingly used on social media, but few studies have checked whether they treat dialects fairly. This paper audits five English Twitter datasets labeled for hate speech and abuse for racial bias. For each dataset, the authors train a regularized logistic regression on TF-IDF bag-of-words n-grams. They then apply each model to the Blodgett et al. (2016) corpus of demographically aligned tweets and compare predictions for African-American English (AAE) with predictions for Standard American English (SAE). Bootstrap sampling estimates how often each group falls into each class. Every classifier assigns AAE tweets to negative categories, such as hate speech or harassment, significantly more often than SAE tweets. When the comparison is restricted to tweets containing particular keywords, the gap usually narrows but does not disappear. This matters because deploying such systems would disproportionately penalize African-American users, even though this group is frequently on the receiving end of the very abusive content these tools are designed to detect. The authors therefore call for more careful sampling and annotation, and for attention to realistic base rates and social context.

Key points

  • The audited datasets are Waseem and Hovy (2016; 16,849 tweets), Waseem (2016; 6,909), Davidson et al. (2017; 24,783), Golbeck et al. (2017; 20,360) and Founta et al. (2018; 91,951).
  • The model setup uses stemmed tokens, n-grams up to length 3, and at most 10,000 TF-IDF features. Training uses an 80/20 split, stratified 5-fold cross-validation and a grid search over regularization strength. The authors deliberately avoided pre-trained embeddings so they would not import outside bias.
  • Held-out F1 scores show that hate speech and harassment are the hardest classes. Davidson et al. hate scores 0.40 and Golbeck et al. harassment scores 0.26.
  • In Experiment 1, the share of tweets labeled with negative classes ranged from 1% to 18%. Founta et al. estimate that only 0.1–3% of real tweets are abusive.
  • Stated limitations include the lack of ground-truth author race, possible code-switching, possibly different underlying abuse rates across groups, and the use of a single classifier type.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

172 papers of our library cite it (of the 140673 whose bibliographies we read); 15 of them have a page here, 157 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-3504

@inproceedings{davidson2019racial,
  title = {{Racial Bias in Hate Speech and Abusive Language Detection Datasets}},
  author = {Davidson, Thomas and Bhattacharya, Debasmita and Weber, Ingmar},
  booktitle = {Proceedings of the Third Workshop on Abusive Language Online},
  year = {2019},
  doi = {10.18653/v1/w19-3504},
  url = {https://doi.org/10.18653/v1/w19-3504}
}

References

31 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2018. Measuring and Mitigating Unintended Bias in Text Classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society - AIES ’18, pages 67–73, New Orleans, LA, USA. ACM Press. find
  2. Wafa Alorainy, Pete Burnap, Han Liu, Amir Javed, and Matthew L. Williams. 2018. Suspended Accounts: A Source of Tweets with Disgust and Anger Emotions for Augmenting Hate Speech Data Sample. In 2018 International Conference on Machine Learning and Cybernetics (ICMLC), pages 581– 586, Chengdu. IEEE. find
  3. Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Grbovic, Vladan Radosavljevic, and Narayan Bhamidipati. 2015. Hate Speech Detection with Comment Embeddings. In WWW 2015 Companion, pages 29– 30. ACM Press. find
  4. Susan Athey. 2017. Beyond prediction: Using big data for policy problems. Science, 355(6324):483–485. find
  5. Su Lin Blodgett, Lisa Green, and Brendan O’Connor. 2016. Demographic Dialectal Variation in Social Media: A Case Study of African-American English. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1119–1130, Austin, Texas. Association for Computational Linguistics. find our card
  6. Bradley Efron and Robert Tibshirani. 1986. Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Statistical Science, 1(1):54–75. find
  7. Paula Fortuna and Srgio Nunes. 2018. A Survey on Automatic Detection of Hate Speech in Text. ACM Computing Surveys, 51(4):1–30. find
  8. Su Lin Blodgett and Brendan O’Connor. 2017. Racial disparity in natural language processing: A case study of social media african-american english. In Fairness, Accountability, and Transparency in Machine Learning (FAT/ML) workshop. find
  9. Antigoni-Maria Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018. Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior. In Proceedings of the Twelth International AAAI Conference on Web and Social Media, pages 491–500, Palo Alto, California, USA. find
  10. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. In 30th Conference on Neural Information Processing Systems, page 9, Barcelona, Spain. find our card
  11. Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. 2018. Word embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences, 115(16):E3635–E3644. find
  12. Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classication. In Proceedings of Machine Learning Research, volume 81, pages 1–15. find
  13. Jennifer Golbeck, Rajesh Kumar Gnanasekaran, Raja Rajan Gunasekaran, Kelly M. Hoffman, Jenny Hottle, Vichita Jienjitlert, Shivika Khare, Ryan Lau, Marianna J. Martindale, Shalmali Naik, Heather L. Nixon, Zahra Ashktorab, Piyush Ramachandran, Kristine M. Rogers, Lisa Rogers, Meghna Sardana Sarin, Gaurav Shahane, Jayanee Thanki, Priyanka Vengataraman, Zijian Wan, Derek Michael Wu, Rashad O. Banjo, Alexandra Berlinger, Siddharth Bhagwan, Cody Buntain, Paul Cheakalos, Alicia A. Geller, and Quint Gergory. 2017. A Large Labeled Corpus for Online Harassment Research. In Proceedings of the 2017 ACM on Web Science Conference - WebSci ’17, pages 229–233, Troy, New York, USA. ACM Press. find
  14. Pete Burnap and Matthew L. Williams. 2015. Cyber hate speech on twitter: An application of machine classification and statistical modeling for policy and decision making. Policy & Internet, 7(2):223–242. find
  15. Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. find
  16. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated Hate Speech Detection and the Problem of Offensive Language. In Proceedings of the 11th International Conference on Web and Social Media (ICWSM), pages 512–515, Montreal, Quebec, Canada. find our card
  17. Hila Gonen and Yoav Goldberg. 2019. Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them. In NAACL. ArXiv: 1903.03862. ↗ our card
  18. Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. 2019. Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting. In Proceedings of the Conference on Fairness, Accountability, and Transparency - FAT* ’19, pages 120–128, Atlanta, GA, USA. ACM Press. find
  19. Gary King, Patrick Lam, and Margaret E. Roberts. 2017. Computer-Assisted Keyword and Document Set Discovery from Unstructured Text. American Journal of Political Science, 61(4):971–988. find
  20. Svetlana Kiritchenko and Saif M. Mohammad. 2018. Examining gender and race bias in two hundred sentiment analysis systems. In Proceedings of the 7th Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 15– 20, New Orleans, Louisiana. Association for Computational Linguistics. Joint Conference on Lexical and Computational Semantics (*SEM), pages 42–53, New Orleans, LA, USA. Association for Computational Linguistics. find our card
  21. Irene Kwok and Yuzhou Wang. 2013. Locate the Hate: Detecting Tweets against Blacks. In Association for the Advancement of Artificial Intelligence. find
  22. Mark Anthony Neal. 2013. NIGGA: The 21st-Century Theoretical Superhero: NIGGA. Cultural Anthropology, 28(3):556–563. find
  23. Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. Reducing Gender Bias in Abusive Language Detection. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2799–2804, Brussels, Belgium. ArXiv: 1808.07231. ↗ our card
  24. Anna Schmidt and Michael Wiegand. 2017. A Survey on Hate Speech Detection using Natural Language Processing. In Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media, pages 1–10, Valencia, Spain. find our card
  25. Leandro Silva, Mainack Mondal, Denzil Correa, Fabr-cio Benevenuto, and Ingmar Weber. 2016. Analyzing the Targets of Hate in Online Social Media. In Association for the Advancement of Artificial Intelligence. find
  26. Rachael Tatman. 2017. Gender and Dialect Bias in YouTube’s Automatic Captions. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, pages 53–59, Valencia, Spain. Association for Computational Linguistics. find
  27. Zeerak Waseem. 2016. Are you a racist or am I seeing things? Annotator influence on hate speech detection on Twitter. In Proceedings of the 1st Workshop on Natural Language Processing and Computational Social Science, pages 138–142. find
  28. Zeerak Waseem, Thomas Davidson, Dana Warmsley, and Ingmar Weber. 2017. Understanding Abuse: A Typology of Abusive Language Detection Subtasks. In Proceedings of the 1st Workshop on Abusive Language Online (ACL 2017), pages 78–84, Vancouver, Canada. find our card
  29. Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on twitter. In Proceedings of NAACL-HLT, pages 88–93. find
  30. Zeerak Waseem, James Thorne, and Joachim Bingel. 2018. Bridging the Gaps: Multi Task Learning for Domain Transfer of Hate Speech Detection. In Jennifer Goldbeck, editor, Online Harassment, Human-Computer Interaction Series, page 270. Springer, New York, NY. find
  31. Michael Wiegand, Josef Ruppenhofer, and Thomas Kleinbauer. 2019. Detection of Abusive Language: the Problem of Biased Datasets. In Proceedings of NAACL-HLT, pages 602–608, Minneapolis, Minnesota. Association for Computational Linguistics. find our card