Elamur Research
☰
home / papers / 2019 / Identifying and Reducing Gender Bias in Word-Level Language Models

Identifying and Reducing Gender Bias in Word-Level Language Models

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1904.03035

Summary

Text corpora used to train language models carry gender skew. For example, occupation words appear mostly near male pronouns, and models can reproduce or strengthen this pattern. Bordia and Bowman study the problem in word-level language models. They define a per-word bias score that compares how often a word co-occurs with female versus male terms. A score of zero means neutral, and the sign shows the direction. The score is computed either over a fixed context window or with exponentially weighted co-occurrences. They measure bias in three corpora (Penn Treebank, WikiText-2, CNN/Daily Mail) and in text sampled from an AWD-LSTM trained on each. To reduce bias, they add a regularizer adapted from the soft debiasing of Bolukbasi et al. (2016). It penalizes the projection of the learned input embeddings onto a gender subspace, scaled by a weight λ. Moderate λ lowers bias in generated text across all three corpora. Larger values make the model unstable and tend to raise perplexity, which reveals a bias–perplexity trade-off. The work offers a reproducible way to audit and partly reduce bias in models used in applications such as predictive keyboards.

Key points

  • Model: three-layer AWD-LSTM (1150 hidden units, embedding size 400, PyTorch). Baseline perplexities are 62.56 (PTB), 67.67 (WikiText-2) and 118.01 (CNN/Daily Mail, subsampled by a factor of 100).
  • Corpus-level mean absolute bias µ (fixed window) is 0.83 for Penn Treebank, 0.80 for WikiText-2 and 0.72 for CNN/Daily Mail. PTB is the most male-skewed corpus.
  • In generated text, µ falls to about 0.63 (PTB) and 0.61 (WikiText-2) at λ = 0.01, and to 0.34 (CNN/Daily Mail) at λ = 0.5. Bias rises again at higher λ.
  • Three evaluation measures are used: µ, standard deviation σ, and β. β is the slope from regressing word-level bias in generated text on bias in the training corpus. The authors argue that word-level β tracks debiasing better than corpus-level µ and σ.
  • Debiasing input and output embeddings together sharply raised perplexity, so the reported results debias only the input embeddings. Code is available at github.com/BordiaS/language-model-bias.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

152 papers of our library cite it (of the 140673 whose bibliographies we read); 17 of them have a page here, 135 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1904.03035 · arXiv 1904.03035

@misc{bordia2019identifying,
  title = {{Identifying and Reducing Gender Bias in Word-Level Language Models}},
  author = {Bordia, Shikha and Bowman, Samuel R.},
  year = {2019},
  doi = {10.48550/arxiv.1904.03035},
  eprint = {1904.03035},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1904.03035}
}

References

19 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems… find our card
  2. Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. find
  3. Joel Escudé Font and Marta R. Costa-Jussà. 2019. Equalizing gender biases in neural machine translation with word embeddings techniques. CoRR, abs/1901.03116. — [arXiv:1901.03116](https://arxiv.org/abs/1901.03116) find
  4. Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Ass… find our card
  5. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28,… find our card
  6. Dirk Hovy. 2015. Demographic factors improve classification performance. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), volume 1, pages 752–762. find
  7. Anja Lambrecht and Catherine E Tucker. 2018. Algorithmic bias? an empirical study into apparent gender-based discrimination in the display of stem career ads. Social Science Research Network (SSRN). find
  8. Omer Levy and Yoav Goldberg. 2014. Dependency-based word embeddings. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 302–308. find
  9. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of english: The penn treebank. Computational Linguistics, 19(2):313–330. find our card
  10. Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019. On measuring social bias in sentence encoders. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Compu… find
  11. Stephen Merity, Nitish Shirish Keskar, and Richard Socher. 2018. Regularizing and optimizing LSTM language models. In International Conference on Learning Representations. find our card
  12. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843. find our card
  13. Tomas Mikolov, Martin Karafiát, Lukás Burget, Jan Cernocký, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In IN-TERSPEECH, pages 1045–1048. ISCA. find
  14. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119. find
  15. Mei Ngan and Patrick Grother. 2015. Face recognition vendor test (FRVT) performance of automated gender classification algorithms. US Department of Commerce, National Institute of Standards and Technology. find
  16. Marta Recasens, Cristian Danescu-Niculescu-Mizil, and Dan Jurafsky. 2013. Linguistic models for analyzing and detecting biased language. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1650–1659. find
  17. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Louisiana. Associatio… find our card
  18. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In EMNLP, pages 2979– 2989. Association for Computational Linguistics. find our card
  19. Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018. Learning gender-neutral word embeddings. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4847–4853. Association for Computational Linguistics. find our card