Elamur Research
☰
home / papers / 2018 / Gender Bias in Neural Natural Language Processing

Gender Bias in Neural Natural Language Processing

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1807.11714

Summary

The paper asks whether neural NLP systems absorb gender stereotypes from their training text and how to measure and reduce them. The authors propose a general bias benchmark inspired by causal testing. Sentence pairs that differ only in a gendered word are fed to a model, and the gap in its internal scores is treated as bias. For coreference this means coreference scores, and for language models it means log-likelihoods. The benchmark is used to examine occupation words in two neural coreference systems trained on CoNLL-2012, the approaches of Lee et al. (2017) and of Clark and Manning (2016b), as well as in an LSTM language model (Zaremba et al. 2014) trained on Wikitext-2. All of these show measurable stereotypical bias.

As a remedy they introduce counterfactual data augmentation (CDA), which adds gender-swapped copies of training instances while keeping the original labels. They compare CDA with word embedding debiasing (WED) and with combinations of the two. CDA cuts bias with little accuracy loss and beats WED, especially when embeddings are trained jointly with the model. Bias also tends to rise as training loss falls, and CDA limits this growth.

Key points

  • Bias is measured as aggregate occupation bias (AOB) over matched gendered pairs, for example "he/she" alongside "doctor/nurse".
  • Language model results (2-layer LSTM, 1500 units, Wikitext-2):
    • The baseline has perplexity 83.39 and AOB 0.054.
    • Post-training WED pushes perplexity to 1128.15.
    • CDA with naive swapping lowers bias by 85% for only +0.24 perplexity.
    • Grammatical CDA lowers bias by 46%.
  • WED applied to pre-trained embeddings before training is largely undone, because the bias is relearned by the end of training.
  • Combining CDA with post-training WED in the Clark and Manning model (configurations 2.7 and 2.8) overcorrects, flipping the bias toward females and hurting accuracy.
  • CDA combined with pre-training WED gives complementary debiasing for models that use pre-trained embeddings.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Topics: Coreference resolution

Cited by

144 papers of our library cite it (of the 140673 whose bibliographies we read); 14 of them have a page here, 130 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1807.11714 · arXiv 1807.11714

@misc{lu2018gender,
  title = {{Gender Bias in Neural Natural Language Processing}},
  author = {Lu, Kaiji and Mardziel, Piotr and Wu, Fangjing and Amancharla, Preetam and Datta, Anupam},
  year = {2018},
  doi = {10.48550/arxiv.1807.11714},
  eprint = {1807.11714},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1807.11714}
}

References

19 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kevin Clark and Christopher D Manning. Improving coreference resolution by learning entity-level distributed representations. arXiv preprint arXiv:1606.01323, 2016a. find our card
  2. Kevin Clark and Christopher D Manning. Deep reinforcement learning for mention-ranking coreference models. arXiv preprint arXiv:1609.08667, 2016b. find our card
  3. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. End-to-end neural coreference resolution. arXiv preprint arXiv:1707.07045, 2017. find our card
  4. Rafal Jozefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, and Yonghui Wu. Exploring the limits of language modeling. arXiv preprint arXiv:1602.02410, 2016. find
  5. Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda B. Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean. Google’s multilingual neural machine translation system: Enabling zero-shot translation. TACL, 5: 339–351, 2017. UR… find
  6. Issie Lapowsky. Google autocomplete still has a hitler problem, Feb 2018. URL https://www. wired.com/story/google-autocomplete-vile-suggestions/. find
  7. Rachael Tatman. Gender and dialect bias in youtube’s automatic captions. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, pages 53–59, 2017. find
  8. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems, pages 4349–4357, 2016. find our card
  9. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186, 2017. find
  10. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013. find
  11. Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. Recurrent neural network regularization. arXiv preprint arXiv:1409.2329, 2014. find our card
  12. Jiaping Zheng, Wendy W Chapman, Rebecca S Crowley, and Guergana K Savova. Coreference resolution: A review of general methodologies and applications in the clinical domain. Journal of biomedical informatics, 44(6):1113–1122, 2011. find
  13. Alex Graves. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850, 2013. find
  14. Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton. Speech recognition with deep recurrent neural networks. In Acoustics, speech and signal processing (icassp), 2013 ieee international conference on, pages 6645–6649. find
  15. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014. find our card
  16. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. Conll-2012 shared task: Modeling multilingual unrestricted coreference in ontonotes. In Joint Conference on EMNLP and CoNLL-Shared Task, pages 1–40. Association for Computational Linguistics, 2012. find our card
  17. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843, 2016. find our card
  18. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Gender bias in coreference resolution: Evaluation and debiasing methods. arXiv preprint arXiv:1804.06876, 2018. find our card
  19. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. Gender bias in coreference resolution. arXiv preprint arXiv:1804.09301, 2018. find our card