Elamur Research
☰
home / papers / 2016 / Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings

Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1607.06520

Summary

Word embeddings trained on large text collections absorb gender stereotypes. Many search, ranking and other NLP systems reuse them as input features, so these biases can be amplified downstream. The authors study w2vNEWS, the public word2vec model trained on Google News. Its analogy arithmetic produces stereotyped answers, for example pairing woman with homemaker where man is paired with computer programmer. They show that gender is mostly captured by a single direction, which can be estimated from definitional pairs such as she–he. They also find that gender-specific words can be separated linearly from gender-neutral ones. Using these properties, they define two measures: direct bias, where a neutral word leans toward one gender, and indirect bias, where similarity between neutral words is driven by gender. They then propose two algorithms. Hard debiasing ("neutralize and equalize") removes the gender component from neutral words. Soft debiasing trades some bias reduction for keeping the original distances. Ratings from Amazon Mechanical Turk workers confirm that the detected biases match common stereotypes. Hard debiasing cuts stereotyped analogies substantially without hurting standard benchmark performance.

Key points

  • Main embedding: w2vNEWS is a word2vec model trained on Google News, with about 3 million English words and phrases in 300 dimensions. Other public embeddings trained with different algorithms show similar biases.
  • Occupations on the she–he direction: the most "female" ones include homemaker, nurse and receptionist. The most "male" ones include maestro, skipper and protege.
  • Crowd evaluation: 10 crowd workers rated each generated analogy. Hard debiasing reduced stereotyped analogies from about 19% to about 6%. The number of gender-appropriate analogies (e.g. queen–king, sister–brother) stayed about the same. Soft debiasing removed less bias.
  • Indirect bias: the softball–football axis is used as an example. Before debiasing, receptionist and waitress sat near the softball end and businessman near the football end. Afterwards these were replaced by sport-related terms such as infielder and midfielder.
  • Limitations: the same embedding also contains racial stereotypes. The authors leave languages with grammatical gender, such as Ukrainian, as an open question. In those languages most nouns carry a gender marker, so this matters for Ukrainian NLP.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

792 papers of our library cite it (of the 140673 whose bibliographies we read); 59 of them have a page here, 733 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1607.06520 · arXiv 1607.06520

@misc{bolukbasi2016man,
  title = {{Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings}},
  author = {Bolukbasi, Tolga and Chang, Kai-Wei and Zou, James and Saligrama, Venkatesh and Kalai, Adam},
  year = {2016},
  doi = {10.48550/arxiv.1607.06520},
  eprint = {1607.06520},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1607.06520}
}

References

41 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. J. Angwin, J. Larson, S. Mattu, and L. Kirchner. Machine bias: There’s software used across the country to predict future criminals. and it’s biased against blacks., 2016. find
  2. C. Aschwanden. The finkbeiner test: What matters in stories about women scientists? DoubleXScience, 2013. find
  3. S. Barocas and A. D. Selbst. Big data’s disparate impact. Available at SSRN 2477899, 2014. find
  4. E. Beigman and B. B. Klebanov. Learning with annotation noise. In ACL, 2009. find
  5. A. J. Cuddy, E. B. Wolf, P. Glick, S. Crotty, J. Chong, and M. I. Norton. Men as cultural ideals: Cultural values moderate gender stereotype content. Journal of personality and social psychology, 109(4):622, 2015. find
  6. A. Datta, M. C. Tschantz, and A. Datta. Automated experiments on ad privacy settings. Proceedings on Privacy Enhancing Technologies, 2015. find
  7. C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel. Fairness through awareness. In Innovations in Theoretical Computer Science Conference, 2012. find
  8. J. Eisenstein, B. O’Connor, N. A. Smith, and E. P. Xing. Diffusion of lexical change in social media. PLoS ONE, pages 1–13, 2014. find
  9. M. Faruqui, J. Dodge, S. K. Jauhar, C. Dyer, E. Hovy, and N. A. Smith. Retrofitting word vectors to semantic lexicons. In NAACL, 2015. find
  10. M. Feldman, S. A. Friedler, J. Moeller, C. Scheidegger, and S. Venkatasubramanian. Certifying and removing disparate impact. In KDD, 2015. find
  11. C. Fellbaum, editor. Wordnet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, 1998. find
  12. L. Finkelstein, E. Gabrilovich, Y. Matias, E. Rivlin, Z. Solan, G. Wolfman, and E. Ruppin. Placing search in context: The concept revisited. In WWW. ACM, 2001. find
  13. P. Glick and S. T. Fiske. The ambivalent sexism inventory: Differentiating hostile and benevolent sexism. Journal of personality and social psychology, 70(3):491, 1996. find
  14. J. Gordon and B. Van Durme. Reporting bias and knowledge extraction. Automated Knowledge Base Construction (AKBC), 2013. find
  15. A. G. Greenwald, D. E. McGhee, and J. L. Schwartz. Measuring individual differences in implicit cognition: the implicit association test. Journal of personality and social psychology, 74(6):1464, 1998. find
  16. C. Hansen, M. Tosik, G. Goossen, C. Li, L. Bayeva, F. Berbain, and M. Rotaru. How to get the best word vectors for resume parsing. In SNN Adaptive Intelligence / Symposium: Machine Learning 2015, Nijmegen. find
  17. J. Holmes and M. Meyerhoff. The handbook of language and gender, volume 25. John Wiley & Sons, 2008. find
  18. O. İrsoy and C. Cardie. Deep recursive neural networks for compositionality in language. In NIPS. 2014. find
  19. R. Jakobson, L. R. Waugh, and M. Monville-Burston. On language. Harvard Univ Pr, 1990. find
  20. J. T. Jost and A. C. Kay. Exposure to benevolent sexism and complementary gender stereotypes: consequences for specific and diffuse forms of system justification. Journal of personality and social psychology, 88(3):498, 2005. find
  21. M. Kay, C. Matuszek, and S. A. Munson. Unequal representation and gender stereotypes in image search results for occupations. In Human Factors in Computing Systems. ACM, 2015. find
  22. T. Lei, H. Joshi, R. Barzilay, T. Jaakkola, A. M. Katerina Tymoshenko, and L. Marquez. Semi-supervised question retrieval with gated convolutions. In NAACL. 2016. find
  23. O. Levy and Y. Goldberg. Linguistic regularities in sparse and explicit word representations. In CoNLL, 2014. find
  24. T. Mikolov, K. Chen, G. Corrado, and J. Dean. Efficient estimation of word representations in vector space. In ICLR, 2013. find
  25. T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In NIPS. find
  26. T. Mikolov, W.-t. Yih, and G. Zweig. Linguistic regularities in continuous space word representations. In HLT-NAACL, pages 746–751, 2013. find
  27. E. Nalisnick, B. Mitra, N. Craswell, and R. Caruana. Improving document ranking with dual word embeddings. In www, April 2016. find
  28. B. A. Nosek, M. Banaji, and A. G. Greenwald. Harvesting implicit group attitudes and beliefs from a demonstration web site. Group Dynamics: Theory, Research, and Practice, 6(1):101, 2002. find
  29. D. Pedreshi, S. Ruggieri, and F. Turini. Discrimination-aware data mining. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 560–568. ACM, 2008. find
  30. J. Pennington, R. Socher, and C. D. Manning. Glove: Global vectors for word representation. In EMNLP, 2014. find
  31. K. Ross and C. Carter. Women and news: A long and winding road. Media, Culture & Society, 33(8):1148–1165, 2011. find
  32. H. Rubenstein and J. B. Goodenough. Contextual correlates of synonymy. Communications of the ACM, 8(10):627–633, 1965. find
  33. E. Sapir. Selected writings of Edward Sapir in language, culture and personality, volume 342. Univ of California Press, 1985. find
  34. B. Schmidt. Rejecting the gender binary: a vector-space operation. http://bookworm.benschmidt. org/posts/2015-10-30-rejecting-the-gender-binary.html, 2015. ↗
  35. J. P. Stanley. Paradigmatic woman: The prostitute. Papers in language variation, pages 303–321, 1977. find
  36. L. Sweeney. Discrimination in online ad delivery. Queue, 11(3):10, 2013. find
  37. A. Torralba and A. Efros. Unbiased look at dataset bias. In CVPR, 2012. find
  38. P. D. Turney. Domain and function: A dual-space model of semantic relations and compositions. Journal of Artificial Intelligence Research, pages 533–585, 2012. find
  39. C. Wagner, D. Garcia, M. Jadidi, and M. Strohmaier. It’s a man’s wikipedia? assessing gender inequality in an online encyclopedia. In Ninth International AAAI Conference on Web and Social Media, 2015. find
  40. D. Yogatama, M. Faruqui, C. Dyer, and N. A. Smith. Learning word representations with hierarchical sparse coding. In ICML, 2015. find
  41. I. Zliobaite. A survey on measuring indirect discrimination in machine learning. arXiv preprint arXiv:1511.00148, 2015. — [arXiv:1511.00148](https://arxiv.org/abs/1511.00148) find