Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them
Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/N19-1061.pdf
Summary
Gonen and Goldberg examine whether popular gender-debiasing techniques for word embeddings actually remove bias or only mask it. They test two methods. HARD-DEBIASED (Bolukbasi et al., 2016b) is post-processing applied to word2vec trained on Google News. GN-GLOVE (Zhao et al., 2018) modifies the GloVe training objective. Both methods target a single bias definition: a word's projection onto the he–she gender direction. The authors measure bias in other ways and compare each debiased model with its original counterpart. Their tests include k-means clustering of strongly biased words, a new bias score based on the gender makeup of a word's nearest neighbours, WEAT association tests, and an SVM classifier that predicts the original gender association. In every test, much of the stereotypical structure survives debiasing. Words that were socially gendered still sit near each other, so the bias remains recoverable from the geometry of the space. The authors conclude that projection-based debiasing gives a misleading sense of gender neutrality. They argue that bias definitions and removal methods need to go beyond the gender direction.
Key points
- Clustering: The 1,000 most biased words were split into two k-means clusters. These clusters still matched the original gender with 92.5% accuracy for HARD-DEBIASED (99.9% before debiasing) and 85.6% for GN-GLOVE (100% before).
- Neighbour-based bias: This score correlated with the original projection bias at Pearson 0.686 for HARD-DEBIASED and 0.736 for GN-GLOVE. For profession words, the correlations were 0.606 and 0.792.
- Classifier test: An RBF-kernel SVM was trained on 1,000 of the 5,000 most biased words. It recovered gender for the remaining words with 88.88% accuracy for HARD-DEBIASED (98.25% before) and 96.53% for GN-GLOVE (98.65% before).
- WEAT tests: All three Caliskan et al. (2017) WEAT tests (career/family, math/arts, science/arts, using names) remained significant after debiasing.
- Data and code: The reduced vocabularies contained 26,189 words for HARD-DEBIASED and 47,698 for GN-GLOVE. The code is published at github.com/gonenhila/gender_bias_lipstick.
Cited by
- JW300: A Wide-Coverage Parallel Corpus for Low-Resource Languages
- Language (Technology) is Power: A Critical Survey of ``Bias'' in NLP
- On the Opportunities and Risks of Foundation Models
- Identifying and Reducing Gender Bias in Word-Level Language Models
- Toward Gender-Inclusive Coreference Resolution
- Racial Bias in Hate Speech and Abusive Language Detection Datasets
- RobBERT: a Dutch RoBERTa-based Language Model
- Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation
- Bias and Fairness in Large Language Models: A Survey
- Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual
- The Importance of Modeling Social Factors of Language: Theory and Practice
- Towards Debiasing Sentence Representations
- A Survey on Bias and Fairness in Machine Learning
- StereoSet: Measuring stereotypical bias in pretrained language models
- CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
- GPT-4 Technical Report
- Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection
- Gender Bias in Machine Translation
- Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP
- On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning
- Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
- Evaluating Gender Bias in Machine Translation
- Mitigating Gender Bias in Natural Language Processing: Literature Review
- SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
- Measuring and Reducing Gendered Correlations in Pre-trained Models
- Gender Bias in Contextualized Word Embeddings
Cite
@inproceedings{gonen2019lipstick,
title = {{Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them}},
author = {Gonen, Hila and Goldberg, Yoav},
booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
year = {2019},
doi = {10.18653/v1/n19-1061},
eprint = {1903.03862},
archivePrefix = {arXiv},
url = {https://doi.org/10.18653/v1/n19-1061}
}References
- Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016a. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. arXiv:1607.06520. 3. The implicit gender of words with prevalent previous bias is easy to predict based on their vectors alone. find our card
- Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016b. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems. find our card