Elamur Research
☰
home / papers / 2018 / Reducing Gender Bias in Abusive Language Detection

Reducing Gender Bias in Abusive Language Detection

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/D18-1302.pdf

Summary

Abusive language classifiers trained on skewed data often learn to link neutral identity words with abuse. For example, a model trained on a sexism corpus labels a harmless compliment to a woman as sexist. Park, Shin and Fung measure this gender bias and test ways to reduce it. They train CNN, GRU and α-GRU (bidirectional GRU with self-attention) classifiers on two Twitter corpora, Sexist Tweets (st) and Abusive Tweets (abt). Each model uses random, FastText or word2vec embeddings. To measure bias, they build a template-based test set of sentence pairs that differ only in a male or female identity term. They score it with Gen. AUC and with FPED and FNED (false positive and false negative equality differences).

Bias is strong on st and always points toward female terms. It grows when pre-trained embeddings or attention-like architectures are used. The authors test three fixes: debiased word2vec, gender-swap augmentation, and fine-tuning from the less biased abt. Together these cut bias by 90–98% at a modest accuracy cost. The study argues that fairness should be evaluated alongside accuracy.

Key points

  • Datasets: st has about 18K tweets, 33% positive, keeping only the "sexist" label. abt has 60K crowdsourced tweets from Founta et al. (2018), 18.5% positive, made binary by merging Abusive/Hateful against None/Spam.
  • The unbiased test set has 1,152 samples (576 gender pairs). It was generated with the template code of Dixon et al. (2017) from neutral and offensive words.
  • abt-trained models were far less biased, with equality differences of at most 0.04. CNN max-pooling and α-GRU attention raised false positive bias on st.
  • The best setting was GRU with debiased embeddings plus gender swap. It reduced the equality differences by 98% and 89% while losing only 1.5% of original performance.
  • Fine-tuning from abt to st (pre-train, replace the softmax layer, retrain with a lower learning rate) strongly improved Gen. AUC. Its drop in original-test performance was the largest but stayed under 4%.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

149 papers of our library cite it (of the 140673 whose bibliographies we read); 16 of them have a page here, 133 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1302

@inproceedings{park2018reducing,
  title = {{Reducing Gender Bias in Abusive Language Detection}},
  author = {Park, Ji Ho and Shin, Jamin and Fung, Pascale},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1302},
  url = {https://doi.org/10.18653/v1/d18-1302}
}

References

19 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Pinkesh Badjatiya, Shashank Gupta, Manish Gupta, and Vasudeva Varma. 2017. Deep learning for hate speech detection in tweets. In Proceedings of the 26th International Conference on World Wide Web Companion, pages 759–760. International World Wide Web Conferences Steering Committee. find
  2. Alex Beutel, Jilin Chen, Zhe Zhao, and Ed H Chi. Data decisions and theoretical implications when adversarially learning fair representations. FAT/ML 2018: 5th Workshop on Fairness, Accountability, and Transparency in Machine Learning. find
  3. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics Volume 5, Issue 1. find
  4. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems, pages 4349–4357. find our card
  5. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. find
  6. Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. EMNLP2014. find
  7. Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2017. Measuring and mitigating unintended bias in text classification. In AAAI. find
  8. Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, and Sune Lehmann. 2017. Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. EMNLP2017. find our card
  9. Hossein Hosseini, Sreeram Kannan, Baosen Zhang, and Radha Poovendran. 2017. Deceiving google’s perspective api built for detecting toxic comments. arXiv preprint arXiv:1702.08138. — [arXiv:1702.08138](https://arxiv.org/abs/1702.08138) find
  10. Svetlana Kiritchenko and Saif M Mohammad. 2018. Examining gender and race bias in two hundred sentiment analysis systems. Proceedings of the 7th Joint Conference on Lexical and Computational Semantics(*SEM), New Orleans, USA. find our card
  11. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In NIPS, pages 3111–3119. find
  12. Ji Ho Park and Pascale Fung. 2017. One-step and two-step classification for abusive language detection on twitter. ALW1: 1st Workshop on Abusive Language Online to be held at the annual meeting of the Association of Computational Linguistics (ACL) 2017. find
  13. John Pavlopoulos, Prodromos Malakasiotis, and Ion Androutsopoulos. 2017. Deeper attention to abusive user content moderation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1125–1135. find
  14. Zeerak Waseem. 2016. Are you a racist or am i seeing things? annotator influence on hate speech detection on twitter. In Proceedings of the first workshop on NLP and computational social science, pages 138– 142. find
  15. Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on twitter. In Proceedings of the NAACL student research workshop, pages 88–93. find
  16. Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017. Ex machina: Personal attacks seen at scale. In Proceedings of the 26th International Conference on World Wide Web, pages 1391–1399. International World Wide Web Conferences Steering Committee. find
  17. Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018. Mitigating unwanted biases with adversarial learning. Proceedings of AAAI/ACM Conference on Ethics and Society(AIES) 2018. find
  18. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. Gender bias in coreference resolution: Evaluation and debiasing methods. find our card
  19. Antigoni-Maria Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018. Large scale crowdsourcing and characterization of twitter abusive behavior. AAAI. find