Elamur Research
☰
home / papers / 2019 / Measuring Bias in Contextualized Word Representations

Measuring Bias in Contextualized Word Representations

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/W19-3823.pdf

Summary

Static embeddings like word2vec and GloVe are known to absorb social stereotypes. However, the usual ways to detect this, such as analogies and the cosine-based WEAT, do not transfer well to contextual models like BERT, where a word's vector changes with its sentence. The authors instead probe BERT's masked language model directly. A short template such as "[TARGET] is a [ATTRIBUTE]" is filled with a target like he or she. The target's probability when the attribute is visible is normalized by its probability when the attribute is also masked, giving an increased log probability score. The difference between two targets' scores is the log probability bias score. On the WEAT stimuli, using uncased BERT BASE, WEAT applied to final-layer embeddings found no significant bias, while the new score found significant bias in all five tested categories. A case study on Gendered Pronoun Resolution and an analysis of jobs, traits and skills suggest these biases affect downstream behavior. This raises concerns for uses such as automated resume filtering.

Key points

  • WEAT comparison (Table 3):
    • The proposed score gave effect sizes of 0.85–1.13, all significant at p < 0.01.
    • WEAT on BERT gave 0.50–1.00, with none significant.
    • WEAT on GloVe was significant in 3 of 5 categories. The EA/AA names test failed partly because only 5 African American names were in BERT's vocabulary.
  • GPR case study (Webster et al., 2018):
    • The training data is gender-balanced, yet the model predicted that a female pronoun refers to no entity more often than a male one (average probability 13.9% vs 11.5%, against priors of 9.8% vs 10.3%; p = 0.007).
    • The authors also scored sentence topics extracted with NMF (20 topics) and related them to this behavior.
  • Employment attributes: BERT associated most attributes more strongly with he than with she:
    • 88.5% of the top-1000 job titles from the Montgomery County salary data
    • 80.0% of positive traits
    • 78.9% of negative traits
    • 84.0% of O*NET 23.2 technology skills (template "TARGET can do ATTRIBUTE")
  • Code, data and extensions: Code and data are publicly released. The method is claimed to extend to multiclass biases such as race and religion.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

247 papers of our library cite it (of the 140673 whose bibliographies we read); 18 of them have a page here, 229 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-3823 · arXiv 1906.07337

@inproceedings{kurita2019measuring,
  title = {{Measuring Bias in Contextualized Word Representations}},
  author = {Kurita, Keita and Vyas, Nidhi and Pareek, Ayush and Black, Alan W and Tsvetkov, Yulia},
  booktitle = {Proceedings of the First Workshop on Gender Bias in Natural Language Processing},
  year = {2019},
  doi = {10.18653/v1/w19-3823},
  eprint = {1906.07337},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/w19-3823}
}

References

13 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Evelin Amorim, Marcia Cançado, and Adriano Veloso. 2018. Automated essay scoring in the presence of biased ratings. In Proc. of NAACL, pages 229–237. find
  2. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proc.of NIPS, pages 3111–3119. find
  3. Solon Barocas and Andrew D Selbst. 2016. Big data’s disparate impact. Calif. L. Rev., 104:671. find
  4. Christine Basta, Marta R Costa-jussà, and Noe Casas. 2019. Evaluating the underlying gender bias in contextualized word embeddings. arXiv preprint arXiv:1904.08783. find
  5. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proce. of EMNLP, pages 1532– 1543. find
  6. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Proc. of NIPS, pages 4349–4357. find our card
  7. Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proc. of NAACL. find
  8. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. In Proc. of NAACL. find our card
  9. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. find
  10. J Segal. 2014. Social media use in hiring: Assessing the risks. HR Magazine, 59(9). find
  11. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proc. of NAACL. find
  12. Nathaniel Swinger, Maria De-Arteaga, Neil Heffernan IV, Mark Leiserson, and Adam Kalai. 2019. What are the biases in my word embedding? In Proc. of the AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES). find
  13. Anjalie Field and Yulia Tsvetkov. 2019. Entity-centric contextual affective analysis. In Proc. of ACL. find