Elamur Research
☰
home / papers / 2019 / Gender Bias in Contextualized Word Embeddings

Gender Bias in Contextualized Word Embeddings

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/N19-1064.pdf

Summary

The paper asks whether contextualized embeddings carry the same gender stereotypes already documented for static vectors such as GloVe, and whether that bias reaches downstream systems. The authors study ELMo in three ways. They count gendered pronouns in its training corpus. They use PCA to examine gender geometry in OntoNotes 5.0 sentences paired with gender-swapped versions. They also train an SVM probe that predicts an occupation mention's gender from its ELMo vector. They then test the ELMo-based coreference resolver of Lee et al. (2018) on the WinoBias diagnostic set and compare two remedies:

  • Data augmentation: retraining on OntoNotes plus a gender-swapped copy.
  • Neutralization: at test time, averaging each input's ELMo vectors with those of its gender-swapped counterpart.

ELMo's training data is heavily skewed toward male references, and the embeddings encode gender unevenly. The coreference system shows a stereotype gap at least as large as the GloVe baseline. Augmentation shrinks this gap to insignificant levels. Neutralization helps only partly. As contextual encoders spread, this shows that bias audits must extend beyond static vectors.

Key points

  • The One Billion Word Benchmark, ELMo's training corpus, has about 5.3M male pronoun occurrences against 1.6M female ones. Male pronouns also co-occur more with occupation words, even stereotypically female occupations.
  • ELMo's gender subspace has two main principal components, which the authors read as contextual gender and occupational gender. GloVe has a single component.
  • The RBF-kernel SVM probe was trained on 539 instances and tested on 62. It recovered gender with 95.1% accuracy for male contexts versus 80.6% for female ones, a gap of more than 14 points.
  • On the WinoBias Semantics Only subset, the ELMo system's pro/anti F1 gap was 29.6, compared with 26.6 for GloVe. ELMo still raised OntoNotes F1 from 67.7 to 72.7.
  • Data augmentation lowered the Semantics Only gap to 1.0 (OntoNotes F1 71.0). The neutralization configurations still showed significant gaps of 14.3 and 11.1 on that subset.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

184 papers of our library cite it (of the 140673 whose bibliographies we read); 19 of them have a page here, 165 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n19-1064 · arXiv 1904.03310

@inproceedings{zhao2019gender,
  title = {{Gender Bias in Contextualized Word Embeddings}},
  author = {Zhao, Jieyu and Wang, Tianlu and Yatskar, Mark and Cotterell, Ryan and Ordonez, Vicente and Chang, Kai-Wei},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  year = {2019},
  doi = {10.18653/v1/n19-1064},
  eprint = {1904.03310},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/n19-1064}
}

References

28 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In NeurIPS. find our card
  2. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. find
  3. Chih-Chung Chang and Chih-Jen Lin. 2011. Libsvm: a library for support vector machines. ACM transactions on intelligent systems and technology (TIST), 2(3):27. find
  4. Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillipp Koehn, and Tony Robinson. 2013. One billion word benchmark for measuring progress in statistical language modeling. arXiv preprint arXiv:1312.3005. find our card
  5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. find
  6. Joel Escudé Font and Marta R. Costa-jussà. 2019. Equalizing gender biases in neural machine translation with word embeddings techniques. arXiv preprint arXiv:1901.03116. — [arXiv:1901.03116](https://arxiv.org/abs/1901.03116) find
  7. Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. CoRR, abs/1903.03862. find our card
  8. Yvette Graham, Nitika Mathur, and Timothy Baldwin. 2014. Randomized significance tests in machine translation. In WMT@ ACL. find
  9. Matthew Kay, Cynthia Matuszek, and Sean A. Munson. 2015. Unequal representation and gender stereotypes in image search results for occupations. In Human Factors in Computing Systems. ACM. find
  10. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In EMNLP. find our card
  11. Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019. On measuring social biases in sentence encoders. arXiv preprint arXiv:1903.10561. find our card
  12. Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017. Learned in translation: Contextualized word vectors. In NeurIPS. find our card
  13. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In NeurIPS. find
  14. Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. Reducing gender bias in abusive language detection. In EMNLP. find our card
  15. Haoruo Peng, Daniel Khashabi, and Dan Roth. 2015. Solving hard coreference problems. In NAACL. find
  16. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In EMNLP. find
  17. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In NAACL. find
  18. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. find
  19. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. find
  20. Altaf Rahman and Vincent Ng. 2012. Resolving complex cases of definite pronouns: The winograd schema challenge. In EMNLP. find our card
  21. Karen Ross and Cynthia Carter. 2011. Women and news: A long and winding road. Media, Culture & Society, 33(8). find
  22. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. In NAACL. find our card
  23. Latanya Sweeney. 2013. Discrimination in online ad delivery. Queue, 11(3):10. find
  24. Eva Vanmassenhove, Christian Hardmeier, and Andy Way. 2018. Getting Gender Right in Neural Machine Translation. In EMNLP. find our card
  25. Ralph Weischedel, Sameer Pradhan, Lance Ramshaw, Jeff Kaufman, Michelle Franchini, Mohammed El-Bachouti, Nianwen Xue, Martha Palmer, Jena D Hwang, Claire Bonial, et al. 2012. Ontonotes release 5.0. find
  26. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In EMNLP. find our card
  27. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018a. Gender bias in coreference resolution: Evaluation and debiasing methods. In NAACL. find our card
  28. Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018b. Learning gender-neutral word embeddings. In EMNLP. find our card