Elamur Research⌕
☰
home / papers / 2019 / Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings

Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings

Summary

Word embeddings trained on web text absorb social stereotypes and pass them on to downstream models. Earlier debiasing work (Bolukbasi et al., 2016) only handled binary attributes such as male/female. This paper generalizes that approach to attributes with more than two categories, namely race and religion. The authors build the bias subspace with PCA: each defining set holds one term per class, and the vectors used are the differences between each word and its set's mean. They then apply the existing hard ("Neutralize and Equalize") and soft debiasing procedures. They train word2vec on US-user posts from the L2-Reddit corpus and draw bias lexicons from social-science and NLP studies. They also propose MAC, a WEAT-inspired metric based on mean cosine distance, paired with a t-test. After debiasing, MAC scores move toward 1.0, meaning target terms become less associated with stereotyped attributes, and the changes are statistically significant. Performance on CoNLL 2003 tasks changes only slightly. The work matters because it gives a simple, reusable recipe for reducing non-binary demographic bias.

Key points

  • Data and setup: word2vec embeddings were trained on about 56 million sentences from L2-Reddit, restricted to US authors. Lexicons cover three attributes:

    • gender
    • race (Caucasian, African American, Asian American)
    • religion (Christian, Jewish, Muslim)

  • MAC results (biased → hard-debiased / soft-debiased with λ = 0.2):

    AttributeBiasedHardSoft
    Gender0.6231.0000.747
    Race0.8921.0090.985
    Religion0.8591.0040.894

  • Downstream evaluation: the authors tested NER, POS tagging, and POS chunking on CoNLL 2003, both by swapping in the debiased embedding matrix and by retraining the model. NER and chunking F1 rose slightly (about +0.004 to +0.015). POS tagging usually dropped, by up to about 0.035 F1.

  • Remaining "cluster bias": following Gonen and Goldberg (2019), the authors found that words with similar bias still cluster together after component removal. Using a larger bias subspace weakens this effect.

  • Limitations:

    • Per-class extremes cannot be measured in the shared linear subspace.
    • The lexicons are US-specific and partly subjective.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

77 papers of our library cite it (of the 140884 whose bibliographies we read); 11 of them have a page here, 66 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n19-1062

@inproceedings{manzini2019black,
  title = {{Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings}},
  author = {Manzini, Thomas and Chong, Lim Yao and Black, Alan W and Tsvetkov, Yulia},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  year = {2019},
  doi = {10.18653/v1/n19-1062},
  url = {https://doi.org/10.18653/v1/n19-1062}
}

References

26 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Evelyn Alsultany. 2012. Arabs and Muslims in the Media: Race and Representation after 9/11. nyu Press. find
  2. Solon Barocas and Andrew D Selbst. 2016. Big data’s disparate impact. Calif. L. Rev., 104:671. find
  3. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In Proc. of NIPS, pages 4349–4357. find our card
  4. Narissra M Punyanunt-Carter. 2008. The perceived realism of african american portrayals on television. The Howard Journal of Communications, 19(3):241–257. find
  5. Sonja M Brown Givens and Jennifer L Monahan. 2005. Priming mammies, jezebels, and other controlling images: An examination of the influence of mediated stereotypes on perceptions of an african american woman. Media Psychology, 7(1):87–106. find
  6. Ella Rabinovich, Yulia Tsvetkov, and Shuly Wintner. 2018. Native language cognate effects on second language lexical choice. In Proc. of the Transactions of Association for Computational Linguistics. find
  7. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. find
  8. Kimberly Rios, Zhen Hadassah Cheng, Rebecca R Tot-ton, and Azim F Shariff. 2015. Negative stereotypes cause christians to underperform in and disidentify with science. Social Psychological and Personality Science, 6(8):959–967. find
  9. Beth G Chung-Herrera and Melenie J Lankau. 2005. Are we there yet? an assessment of fit between stereotypes of minority managers and the successful-manager prototype. Journal of Applied Social Psychology, 35(10):2029–2056. find
  10. Jack G Shaheen. 1997. Arab and Muslim stereotyping in American popular culture. Center for Muslim-Christian Understanding, History and International Affairs . find
  11. Alan Dundes. 1971. A study of ethnic slurs: The jew and the polack in the united states. The Journal of American Folklore, 84(332):186–203. find
  12. Joel S Fetzer. 2000. Public attitudes toward immigration in the United States, France, and Germany. Cambridge University Press. find
  13. Andrew Shryock. 2010. Islamophobia/Islamophilia: Beyond the politics of enemy and friend. Indiana University Press. find
  14. Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. 2018. Word embeddings quantify 100 years of gender and ethnic stereotypes. Proc. of the National Academy of Sciences, 115(16). find
  15. Erik F Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the conll-2003 shared task: Language-independent named entity recognition. In Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003-Volume 4, pages 142–147. Association for Computational Linguistics. find
  16. Jim Goad. 1998. The Redneck Manifesto: How Hillbillies Hicks and White Trash Becames America’s Scapegoats. Simon and Schuster. find
  17. Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. find our card
  18. James D Unnever, Francis T Cullen, and Brandon K Applegate. 2005. Turning the other cheek: Reassessing the impact of religion on punitive ideology. Justice Quarterly, 22(3):304–339. find
  19. Ernest A Hakanen. 1995. Emotional use of music by african american adolescents. Howard Journal of Communications, 5(3):214–222. find
  20. Kelly Welch. 2007. Black criminal stereotypes and racial profiling. Journal of Contemporary Criminal Justice, 23(3):276–288. find
  21. Yuko Kawai. 2005. Stereotyping asian americans: The dialectic of the model minority and the yellow peril. The Howard Journal of Communications, 16(2):109–130. find
  22. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Proc. of EMNLP, pages 2979–2989. find our card
  23. Frederick TL Leong and Thomas J Hayes. 1990. Occupational stereotyping of asian americans. The Career Development Quarterly, 39(2):143–154. find
  24. Phil Zuckerman. 2009. Atheism, secularity, and well-being: How the findings of social science counter negative stereotypes and assumptions. Sociology Compass, 3(6):949–971. find
  25. Monica H Lin, Virginia SY Kwan, Anna Cheung, and Susan T Fiske. 2005. Stereotype content model explains prejudice for an envied outgroup: Scale of anti-asian american stereotypes. Personality and Social Psychology Bulletin, 31(1):34–47. find
  26. A Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/manzini-2019-black-is-criminal-as-caucasian/