Elamur Research
☰
home / papers / 2019 / Mitigating Gender Bias in Natural Language Processing: Literature Review

Mitigating Gender Bias in Natural Language Processing: Literature Review

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/P19-1159.pdf

Summary

This survey, presented at ACL 2019, examines how gender bias enters NLP systems and what has been tried to reduce it. The authors note that bias can arise in training data, resources, pre-trained models such as word embeddings, and the algorithms themselves, and that models may even strengthen skews already present in their corpora. They adopt Crawford's (2017) split between allocation harms and representation harms. Representation harms are further divided into four types: denigration, stereotyping, recognition, and under-representation. Using this taxonomy, they classify typical failures in tasks such as machine translation, captioning, speech recognition, sentiment analysis, language modelling, and embeddings. They then group bias-detection methods into three families: tests adapted from psychology, geometric analysis of embedding spaces, and comparisons of performance on gender-swapped inputs. They also evaluate debiasing techniques and their weaknesses. The review finds the field immature. It lacks standard metrics and benchmarks, its fixes usually target one pipeline component, and most work is limited to English and binary gender. This makes the survey a useful map for anyone building or auditing NLP tools, including for gendered languages like Ukrainian.

Key points

  • Measuring bias in embeddings:
    • WEAT (Caliskan et al., 2017) adapts the Implicit Association Test to GloVe and Word2Vec embeddings.
    • SEAT (May et al., 2019) extends the same approach to sentence encoders like ELMo.
    • Bolukbasi et al. (2016) derive a gender direction by applying PCA to ten gendered word pairs.
    • Gonen and Goldberg (2019) show that biased words still cluster together after this direction is removed, a problem they call "cluster bias."
  • Gender Bias Evaluation Testsets (GBETs) reviewed in the paper:
    • Winogender Schemas: 720 sentences
    • WinoBias: 3,160 sentences
    • GAP: 4,454 contexts
    • EEC (sentiment): 8,640 sentences
    • The first three target coreference resolution.
  • Metrics on swapped inputs: FPED and FNED (Dixon et al., 2017) quantify gaps in false-positive and false-negative rates between original and gender-swapped inputs. They have been applied to abusive language detection.
  • Debiasing techniques covered:
    • RBA, which constrains predictions in visual semantic role labelling and is solved with Lagrangian relaxation.
    • Adversarial learning (Zhang et al., 2018), which keeps a discriminator from recovering the protected gender attribute.
  • Open problems:
    • Languages with grammatical gender, such as Spanish, need agreeing modifiers changed when swapping gender.
    • Non-binary and racial bias remain largely unaddressed.
    • Debiasing can introduce noise into models and lower their performance.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

237 papers of our library cite it (of the 140673 whose bibliographies we read); 9 of them have a page here, 228 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p19-1159

@inproceedings{sun2019mitigating,
  title = {{Mitigating Gender Bias in Natural Language Processing: Literature Review}},
  author = {Sun, Tony and Gaut, Andrew and Tang, Shirlyn and Huang, Yuxin and ElSherief, Mai and Zhao, Jieyu and Mirza, Diba and Belding, Elizabeth and Chang, Kai-Wei and Wang, William Yang},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1159},
  url = {https://doi.org/10.18653/v1/p19-1159}
}

References

61 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chen Avin, Barbara Keller, Zvi Lotker, Claire Mathieu, David Peleg, and Yvonne-Anne Pignolet. 2015. Homophily and the Glass Ceiling Effect in Social Networks. In Proceedings of the 2015 Conference on Innovations in Theoretical Computer Science (ITCS‘15), pages 41–50. ACM. find
  2. Solon Barocas, Moritz Hardt, and Arvind Narayanan. 2018. Fairness and Machine Learning. fairml-book.org. http://www.fairmlbook.org. ↗
  3. Emily M Bender and Batya Friedman. 2018. Data statements for natural language processing: Toward mitigating system bias and enabling better science. Transactions of the Association for Computational Linguistics, 6:587–604. find
  4. Alex Beutel, Jilin Chen, Zhe Zhao, and Ed Huai hsin Chi. 2017. Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations. In 2017 Workshop on Fairness, Accountability, and Transparency in Machine Learning. find
  5. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man Is to Computer Programmer As Woman Is to Homemaker? Debiasing Word Embeddings. In Neural Information Processing Systems (NIPS‘16). find our card
  6. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2018. Debiaswe. https://bit.ly/2ruopBZ. Accessed on 12.10.2018. ↗
  7. Kaylee Burns, Lisa Anne Hendricks, Trevor Darrell, Anna Rohrbach, and Kate Saenko. 2018. Women Also Snowboard: Overcoming Bias in Captioning Models. European Conference on Computer Vision (EECV’18). find
  8. Toon Calders and Sicco Verwer. 2010. Three Naive Bayes Approaches for Discrimination-Free Classification. Data Mining and Knowledge Discovery, 21(2):277–292. find
  9. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics Derived Automatically from Language Corpora Contain Human-Like Biases. Science, 356(6334):183–186. find
  10. Alexandra Chouldechova and Aaron Roth. 2018. The Frontiers of Fairness in Machine Learning. arXiv preprint arXiv:1810.08810. — [arXiv:1810.08810](https://arxiv.org/abs/1810.08810) find
  11. Kate Crawford. 2017. The Trouble With Bias. Keynote at Neural Information Processing Systems (NIPS‘17). find
  12. Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2017. Measuring and Mitigating Unintended Bias in Text Classification. In Association for the Advancement of Artificial Intelligence (AAAI’17). find
  13. Laura Douglas. 2017. AI is not Just Learning our Biases; It Is Amplifying Them. https://bit.ly/2zRvGhH. Accessed on 11.15.2018. ↗
  14. Yanai Elazar and Yoav Goldberg. 2018. Adversarial Removal of Demographic Attributes from Text Data. In Empirical Methods of Natural Language Processing (EMNLP‘18). find our card
  15. Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2015. Certifying and Removing Disparate Impact. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD‘15). find
  16. Antigoni-Maria Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018. Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior. In Association for the Advancement of… find
  17. Liye Fu, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2016. Tie-breaker: Using Language Models to Quantify Gender Bias in Sports Journalism. In Proceedings of the IJCAI workshop on NLP meets Journalism. find
  18. Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. 2018. Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes. Proceedings of the National Academy of Sciences, 115(16):E3635–E3644. find
  19. Hila Gonen and Yoav Goldberg. 2019. Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But Do Not Remove Them. In North American Chapter of the Association for Computational Linguistics (NAACL‘19). find our card
  20. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems (NIPS‘14). find
  21. Anthony G Greenwald, Debbie E McGhee, and Jordan LK Schwartz. 1998. Measuring Individual Differences in Implicit Cognition: The Implicit Association Test. Journal of Personality and Social Psychology, 74(6):1464. find
  22. Moritz Hardt, Eric Price, and Nati Srebro. 2016. Equality of Opportunity in Supervised Learning. In Advances in Neural Information Processing Systems (NIPS‘16). find our card
  23. Tatsunori B Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. 2018. Fairness Without Demographics in Repeated Loss Minimization. find
  24. Aurélie Herbelot, Eva Von Redecker, and Johanna Müller. 2012. Distributional Techniques for Philosophical Enquiry. In Proceedings of the 6th Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities, pages 45– 54. Association for Computational Linguistics. find
  25. Dirk Hovy and Shannon L Spruit. 2016. The Social Impact of Natural Language Processing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16), volume 2, pages 591–598. find our card
  26. Niki Kilbertus, Mateo Rojas-Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, and Bernhard Schölkopf. 2017. Avoiding Discrimination Through Causal Reasoning. In Neural Information Processing Systems (NIPS‘17). find
  27. Svetlana Kiritchenko and Saif M Mohammad. 2018. Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems. In 7th Joint Conference on Lexical and Computational Semantics (SEM‘18). find our card
  28. Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2016. Inherent Trade-offs in the Fair Determination of Risk Scores. In Computing Research Repository (CoRR ‘16). find
  29. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-End Neural Coreference Resolution. In Empirical Methods of Natural Language Processing (EMNLP‘17). find our card
  30. Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-Order Coreference Resolution with Coarse-to-Fine Inference. In Empirical Methods of Natural Language Processing (EMNLP’18). find our card
  31. Yitong Li, Timothy Baldwin, and Trevor Cohn. 2018. Towards Robust and Privacy-Preserving Text Representations. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL‘18), pages 1650–1659. find
  32. Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Amancharla, and Anupam Datta. 2018. Gender Bias in Neural Natural Language Processing. find our card
  33. Nishtha Madaan, Sameep Mehta, Taneea Agrawaal, Vrinda Malhotra, Aditi Aggarwal, Yatin Gupta, and Mayank Saxena. 2018. Analyze, Detect and Remove Gender Stereotyping from Bollywood Movies. In Conference on Fairness, Accountability and Transparency (FAT‘18), pages 92–105. find
  34. Thomas Manzini, Yao Chong Lim, Yulia Tsvetkov, and Alan W Black. 2019. Black is to Criminal as Caucasian is to Police: Detecting and Removing Multi-class Bias in Word Embeddings. In North American Chapter of the Association for Computational Linguistics (NAACL‘19). find
  35. Chandler May, Alex Wang, Shikha Bordia, Samuel R Bowman, and Rachel Rudinger. 2019. On Measuring Social Biases in Sentence Encoders. In North American Chapter of the Association for Computational Linguistics (NAACL‘19). find our card
  36. Ishan Misra, C. Lawrence Zitnick, Margaret Mitchell, and Ross Girshick. 2016. Seeing Through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labels. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE ’16), pages 2930–2939. find
  37. Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model cards for model reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 220–229. ACM. find
  38. Corinne A Moss-Racusin, John F Dovidio, Victoria L Brescoll, Mark J Graham, and Jo Handelsman. 2012. Science Faculty’s Subtle Gender Biases Favor Male Students. Proceedings of the National Academy of Sciences, 109(41):16474–16479. find
  39. Maximillian Nickel and Douwe Kiela. 2017. Poincarè Embeddings for Learning Hierarchical Representations. In Advances in Neural Information Processing Systems (NIPS‘17), pages 6338–6347. find our card
  40. Brian A Nosek, Frederick L Smyth, Natarajan Sriram, Nicole M Lindner, Thierry Devos, Alfonso Ayala, Yoav Bar-Anan, Robin Bergh, Huajian Cai, Karen Gonsalkorale, Selin Kesebir, Norbert Maliszewski, Felix Neto, Eero Olli, Jaihyun Park, Konrad Schnabel, Kimihiro Shiomura, Bogdan Tulbure, Reinout Wiers,… find
  41. Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. Reducing Gender Bias in Abusive Language Detection. In Empirical Methods of Natural Language Processing (EMNLP‘18). find our card
  42. Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep Contextualized Word Representations. In North American Chapter of the Association for Computational Linguistics (NAACL‘18). find
  43. Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. 2017. On Fairness and Calibration. In Advances in Neural Information Processing Systems (NIPS‘17), pages 5680–5689. find
  44. Marcelo O. R. Prates, Pedro H. Avelar, and Luı́s C. Lamb. 2018. Assessing gender bias in machine translation: a case study with Google Translate. Neural Computing and Applications. find
  45. Sravana Reddy and Kevin Knight. 2016. Obfuscating Gender in Social Media Writing. In Proceedings of the First Workshop on NLP and Computational Social Science, pages 17–26. find
  46. Christina Richards, Walter Pierre Bouman, Leighton Seal, Meg John Barker, Timo O Nieder, and Guy T‘Sjoen. 2016. Non-Binary or Genderqueer Genders. International Review of Psychiatry (IRP‘16‘). find
  47. Dan Roth and Wen-tau Yih. 2004. A linear programming formulation for global inference in natural language tasks. In Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004) at HLT-NAACL 2004. find our card
  48. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender Bias in Coreference Resolution. In North American Chapter of the Association for Computational Linguistics (NAACL‘18). find our card
  49. Alexander M Rush and Michael Collins. 2012. A Tutorial on Dual Decomposition and Lagrangian Relaxation for Inference in Natural Language Processing. Journal of Artificial Intelligence Research, 45:305– 362. find
  50. Natalie Schluter. 2018. The Glass Ceiling in NLP. In Empirical Methods of Natural Language Processing (EMNLP‘18). find
  51. Ben Schmidt. 2015. Rejecting the Gender Binary: A Vector-Space Operation. https://bit.ly/1OhXJM0. Accessed on 11.15.2018. ↗
  52. Rachel Tatman. 2017. Gender and Dialect Bias in YouTube‘s Automatic Captions. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing (ACL‘17), pages 53–59. find
  53. Eva Vanmassenhove, Christian Hardmeier, and Andy Way. 2018. Getting Gender Right in Neural Machine Translation. In Empirical Methods of Natural Language Processing (EMNLP‘18). find our card
  54. Zeerak Waseem and Dirk Hovy. 2016. Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter. In North American Chapter of the Association for Computational Linguistics (NAACL‘16). find
  55. Kellie Webster, Marta Recasens, Vera Axelrod, and Jason Baldridge. 2018. Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns. In Transactions of the ACL (TACL‘18). find our card
  56. Mark Yatskar, Luke Zettlemoyer, and Ali Farhadi. 2016. Situation Recognition: Visual Semantic Role Labeling for Image Understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE ‘16), pages 5534–5542. find
  57. Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018. Mitigating Unwanted Biases with Adversarial Learning. In AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES‘18). find
  58. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender Bias in Contextualized Word Embeddings. find our card
  59. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men Also Like Shopping: Reducing Gender Bias Amplification Using Corpus-Level Constraints. In Empirical Methods of Natural Language Processing (EMNLP‘17). find our card
  60. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018a. Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods. In North American Chapter of the Association for Computational Linguistics (NAACL‘18). find our card
  61. Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018b. Learning Gender-Neutral Word Embeddings. In Empirical Methods of Natural Language Processing (EMNLP‘18). find our card