Elamur Research
☰
home / papers / 2019 / On Measuring Social Biases in Sentence Encoders

On Measuring Social Biases in Sentence Encoders

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/N19-1063.pdf

Summary

This paper asks whether sentence encoders carry the same social biases that Caliskan et al. (2017) found in GloVe and word2vec using the Word Embedding Association Test (WEAT). The authors propose the Sentence Encoder Association Test (SEAT). SEAT places each target or attribute word into semantically "bleached" templates, which are short deictic sentences that add almost no meaning of their own. WEAT is then run on the resulting sentence vectors, with pooling where a model outputs token sequences. SEAT also allows biases that single words cannot capture. The authors add tests for the angry black woman stereotype, an intersectional bias, and for a professional double bind on women, adapted from Heilman et al. (2004). They evaluate seven pretrained encoders. Older sentence-to-vector models show many significant associations. ELMo, GPT and BERT show fewer. However, SEAT behaves inconsistently across tests in ways that are hard to explain. The authors therefore warn that a null result does not show a model is unbiased, and that cosine similarity may be a poor way to measure similarity between sentence representations.

Key points

  • Models tested: CBoW (GloVe, 840B-token corpus), InferSent, GenSen, Universal Sentence Encoder (DAN), ELMo, GPT and BERT (large, cased). Embedding sizes range from 300 to 4096 dimensions.
  • Statistics: Significance comes from a permutation test at the 0.01 level, with and without Holm-Bonferroni correction. Bias strength is reported as an effect size, as in WEAT.
  • Caliskan tests 3–5: These tests link European American and African American names to pleasant and unpleasant words. For CBoW, p-values rise as expected as the tests get smaller, from 10⁻⁵ to 10⁻². For ELMo, the pattern is reversed: on the word versions p-values fall from 0.95 to 0.08, which points to incoherent concept sets.
  • Double bind tests: These include "unbleached" versions where the sentence context itself carries the bias. ELMo and GPT show no significant associations here, and USE and BERT show only one each.
  • Released resource: Code and test data are public at github.com/W4ngatang/sent-bias.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

323 papers of our library cite it (of the 140673 whose bibliographies we read); 22 of them have a page here, 301 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n19-1063 · arXiv 1903.10561

@inproceedings{may2019measuring,
  title = {{On Measuring Social Biases in Sentence Encoders}},
  author = {May, Chandler and Wang, Alex and Bordia, Shikha and Bowman, Samuel R. and Rudinger, Rachel},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  year = {2019},
  doi = {10.18653/v1/n19-1063},
  eprint = {1903.10561},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/n19-1063}
}

References

31 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Katherine Bailey and Oisin Deery. 2018. How (and how not!) to control for morally relevant bias in NLP. The Second ACL Workshop on Ethics in Nat-Andra Gillespie. 2016. Race, perceptions of femininity, and the power of the first lady: A comparative analysis. In Nadia E. Brown and Sarah Allen Ger-shon… find
  2. Sandra L. Bem. 1974. The measurement of psychological androgyny. Journal of Consulting and Clinical Psychology, 42(2):155–162. find
  3. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems… find our card
  4. Anthony G. Greenwald, Debbie E. McGhee, and Jordan L. K. Schwartz. 1998. Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74(6):1464–1480. find
  5. Anthony G. Greenwald, T. Andrew Poehlman, Eric Luis Uhlmann, and Mahzarin R. Banaji. 2009. Understanding and using the implicit association test: III. Meta-analysis of predictive validity. Journal of Personality and Social Psychology, 97(1):17– 41. find
  6. Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency, volume 81 of Proceedings of Machine Learning Research, pages 77–91, New York, NY, USA. PML… find
  7. Melissa V. Harris-Perry. 2011. Sister Citizen: Shame, Stereotypes, and Black Women in America. Yale University Press, New Haven, Connecticut. find
  8. Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. find
  9. Madeline E. Heilman, Aaron S. Wallen, Daniella Fuchs, and Melinda M. Tamkins. 2004. Penalties for success: Reactions to women who succeed at male gender-typed tasks. Journal of Applied Psychology, 89(3):416–427. find
  10. Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Brian Strope, and Ray Kurzweil. 2018. Universal sentence encoder for english. In Proceedings of the 2018 Conference on Empirical Methods in Natural Langua… find our card
  11. Aurélie Herbelot, Eva von Redecker, and Johanna Müller. 2012. Distributional techniques for philosophical enquiry. In Proceedings of the 6th Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities, pages 45– 54, Avignon, France. Association for Computational Linguist… find
  12. Patricia Hill Collins. 2004. Black Sexual Politics: African Americans, Gender, and the New Racism. Routledge, New York. find
  13. Sture Holm. 1979. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2):65–70. find
  14. Alexis Conneau, Douwe Kiela, Holger Schwenk, Loı̈c Barrault, and Antoine Bordes. 2017. Supervised learning of universal sentence representations from natural language inference data. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 670–680, Copenhagen,… find our card
  15. Svetlana Kiritchenko and Saif Mohammad. 2018. Examining gender and race bias in two hundred sentiment analysis systems. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pages 43–53, New Orleans, Louisiana. Association for Computational Linguistics. find our card
  16. Kimberle Crenshaw. 1989. Demarginalizing the intersection of race and sex: A black feminist critique of antidiscrimination doctrine, feminist theory, and antiracist politics. Feminist Legal Theory: Readings in Law and Gender, 1989(1):57–80. find
  17. D. Soyini Madison. 2009. Crazy patriotism and angry (post)black women. Communication and Critical/Cultural Studies, 6(3):321–326. find
  18. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. ArXiv e-prints. find
  19. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 26, pa… find
  20. Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. 2018. Word embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences. find
  21. Kaye Mitchell. 2012. Raunch versus prude: contemporary sex blogs and erotic memoirs by women. Psychology & Sexuality, 3(1):12–25. find
  22. Brian A. Nosek, Mahzarin R. Banaji, and Anthony G. Greenwald. 2002. Math = male, me = female, therefore math 6 = me. Journal of Personality and Social Psychology, 83(1):44–59. find
  23. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar. Association for Computational Linguistics. find
  24. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… find
  25. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. Online; accessed November 28, 2018. find
  26. Rachel Rudinger, Chandler May, and Benjamin Van Durme. 2017. Social bias in elicited natural language inferences. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, pages 74–79, Valencia, Spain. Association for Computational Linguistics. find our card
  27. Natalie Schluter. 2018. The glass ceiling in nlp. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2793–2798, Brussels, Belgium. Association for Computational Linguistics. find
  28. Pamela Stone and Meg Lovejoy. 2004. Fast-track women and the “choice” to stay home. The ANNALS of the American Academy of Political and Social Science, 596(1):62–83. find
  29. Sandeep Subramanian, Adam Trischler, Yoshua Bengio, and Christopher J Pal. 2018. Learning general purpose distributed sentence representations via large scale multi-task learning. In International Conference on Learning Representations, Vancouver, Canada. find
  30. Latanya Sweeney. 2013. Discrimination in online ad delivery. Communications of the ACM, 56(5):44– 54. find
  31. Rachael Tatman. 2017. Gender and dialect bias in youtube’s automatic captions. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, pages 53–59, Valencia, Spain. Association for Computational Linguistics. find