Elamur Research⌕
☰
home / papers / 2021 / Intrinsic Bias Metrics Do Not Correlate with Application Bias

Intrinsic Bias Metrics Do Not Correlate with Application Bias

Summary

Research on debiasing word embeddings usually assumes that lowering bias inside the vector space will also lower unfair behaviour in downstream systems. This assumption had not been tested. The authors check whether WEAT, the most popular intrinsic bias measure, tracks extrinsic bias, meaning performance gaps between demographic groups in real tasks.

They train hundreds of models across English and Spanish, word2vec and fastText, and two tasks: coreference resolution (Ontonotes/WinoBias) and Twitter hate speech detection. To get many data points, they push embedding bias both down and up, using one method applied before training and one applied after. They then compare WEAT effect sizes with Predictive Parity and Equality of Opportunity using Pearson correlation.

A moderate positive link appears in only a few configurations. Most settings show no correlation or even a negative one, and fastText behaves less predictably than word2vec.

The authors conclude that WEAT is not a reliable indicator of whether debiasing efforts are succeeding. They call for application-level bias tests, more challenge sets and more demographically annotated data.

Key points

  • Experimental grid: 2 languages (English, Spanish) × 2 embedding algorithms (word2vec, fastText) × tasks (coreference on Ontonotes/WinoBias, hate speech on Twitter) × 2 extrinsic metrics (Predictive Parity, Equality of Opportunity).
  • Intrinsic tests:
    • English uses WEAT 6, 7 and 8 for gender.
    • Spanish uses newly built XWEAT tests for gender (7+8) and for bias against migrants. These were constructed to balance grammatical gender, fixing problems in earlier translations.
  • Data points come from bidirectional bias modification ("debiasing" and "overbiasing"). One preprocessing method changes the training data, and one post-processing method changes the trained embedding space.
  • WEAT is heavily used in the field: the authors count 34 CL and FAT papers since January 2020 relying on WEAT or SEAT. Yet no consistent correlation with application bias holds across tasks, languages, or even within most single tasks.
  • Released resources:
    • code;
    • a new intrinsic metric (Spanish WEAT tests);
    • an English hate speech test set annotated for gender.
  • Stated limitation: only discriminative classification tasks were studied, so representational harms in text generation remain untested.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

110 papers of our library cite it (of the 140884 whose bibliographies we read); 12 of them have a page here, 98 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.acl-long.150

@inproceedings{goldfarbtarrant2021intrinsic,
  title = {{Intrinsic Bias Metrics Do Not Correlate with Application Bias}},
  author = {Goldfarb-Tarrant, Seraphina and Marchant, Rebecca and Sánchez, Ricardo Muñoz and Pandya, Mugdha and Lopez, Adam},
  booktitle = {Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  year = {2021},
  doi = {10.18653/v1/2021.acl-long.150},
  url = {https://doi.org/10.18653/v1/2021.acl-long.150}
}

References

43 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Valerio Basile, C. Bosco, E. Fersini, Debora Nozza, V. Patti, F. Pardo, P. Rosso, and M. Sanguinetti. 2019. Semeval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter. In SemEval@NAACL-HLT. find our card
  2. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of “bias” in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454– 5476, Online. Association for Computational Linguistics… find our card
  3. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. find
  4. Tolga Bolukbasi, Kai-Wei Chang, James Y. Zou, Venkatesh Saligrama, and Adam Tauman Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In NIPS. find our card
  5. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356:183–186. find
  6. Kate Crawford. 2017. The trouble with bias. (keynote at neurips). find
  7. Thomas Davidson, Debasmita Bhattacharya, and Ingmar Weber. 2019. Racial bias in hate speech and abusive language detection datasets. In Proceedings of the Third Workshop on Abusive Language Online, pages 25–35, Florence, Italy. Association for Computational Linguistics. find our card
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  9. Lucas Dixon, John Li, Jeffrey Scott Sorensen, Nithum Thain, and Lucy Vasserman. 2018. Measuring and mitigating unintended bias in text classification. In AIES ’18. find
  10. Kawin Ethayarajh, David Duvenaud, and Graeme Hirst. 2019. Understanding undesirable word embedding associations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1696–1705, Florence, Italy. Association for Computational Linguistics. find
  11. Manaal Faruqui, Yulia Tsvetkov, Pushpendre Rastogi, and Chris Dyer. 2016. Problems with evaluation of word embeddings using word similarity tasks. In Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP, pages 30– 35, Berlin, Germany. Association for Computational Lingu… find our card
  12. Antigoni Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018. Large scale crowdsourcing and characterization of twitter abusive behavior. In ICWSM. find
  13. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. Realtoxicityprompts: Evaluating neural toxic degeneration in language models. In EMNLP. find our card
  14. Goran Glavas, Robert Litschko, Sebastian Ruder, and Ivan Vulic. 2019. How to (properly) evaluate cross-lingual word embeddings: On strong baselines, comparative analyses, and some misconceptions. In ACL. find
  15. Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. In NAACL-HLT. find our card
  16. Hila Gonen, Yova Kementchedjhieva, and Yoav Goldberg. 2019. How does grammatical gender affect noun representations in gender-marking languages? In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 463–471, Hong Kong, China. Association for Computational Li… find
  17. A. Greenwald, D. McGhee, and J. L. Schwartz. 1998. Measuring individual differences in implicit cognition: the implicit association test. Journal of personality and social psychology, 74 6:1464–80. find
  18. Moritz Hardt, Eric Price, and Nathan Srebro. 2016. Equality of opportunity in supervised learning. In NIPS. find our card
  19. Felix Hill, Roi Reichart, and A. Korhonen. 2015. Simlex-999: Evaluating semantic models with (genuine) similarity estimation. Computational Linguistics, 41:665–695. find our card
  20. Ben Hutchinson and Margaret Mitchell. 2019. 50 years of test (un)fairness: Lessons for machine learning. In FAT* ’19. find
  21. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In EMNLP. find our card
  22. Keita Kurita, N. Vyas, Ayush Pareek, A. Black, and Yulia Tsvetkov. 2019. Quantifying social biases in contextual word representations. In 1st ACL Workshop on Gender Bias for Natural Language Processing. find
  23. Anne Lauscher and Goran Glavas. 2019. Are we consistently biased? multidimensional analysis of biases in distributional word vectors. In *SEM@NAACL-HLT. find
  24. Anne Lauscher, Goran Glavas, Simone Paolo Ponzetto, and Ivan Vulic. 2020. A general framework for implicit and explicit debiasing of distributional word vector spaces. In AAAI. find
  25. Kenton Lee, Luheng He, M. Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In EMNLP. find our card
  26. Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019. On measuring social biases in sentence encoders. In NAACL-HLT. find our card
  27. K. McCurdy and Oguz Serbetci. 2017. Grammatical gender associations outweigh topical gender bias in crosslinguistic word embeddings. WiNLP. find
  28. Tomas Mikolov, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. CoRR, abs/1301.3781. find
  29. Nikola Mrksic, Ivan Vulic, Diarmuid Ó Séaghdha, Ira Leviant, Roi Reichart, Milica Gasic, Anna Korhonen, and Steve J. Young. 2017. Semantic specialization of distributional word vector spaces using monolingual and cross-lingual constraints. Transactions of the Association for Computational Linguist… find
  30. Gastã Salamanca and Lidia Pereira. 2013. PRESTI-GIO Y ESTIGMATIZACIÓN DE 60 NOMBRES PROPIOS EN 40 SUJETOS DE NIVEL EDUCA-CIONAL SUPERIOR. Universum (Talca), 28:35 – 57. find
  31. Maarten Sap, D. Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019. The risk of racial bias in hate speech detection. In ACL. find our card
  32. João Sedoc and Lyle Ungar. 2019. The role of protected class word lists in bias identification of contextualized word representations. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, pages 55–61, Florence, Italy. Association for Computational Linguistics. find
  33. Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020. Investigating gender bias in language models using causal mediation analysis. In Advances in Neural Information Processing Systems, volume 33, pages 12388–12401. Curran Associates, Inc. find
  34. R. Weischedel, E. Hovy, M. Marcus, and Martha Palmer. 2017. Ontonotes : A large training corpus for enhanced processing. find
  35. Jieyu Zhao and Kai-Wei Chang. 2020. LOGAN: Local group bias detection by clustering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1968–1977, Online. Association for Computational Linguistics. find
  36. Jieyu Zhao, Subhabrata Mukherjee, Saghar Hosseini, Kai-Wei Chang, and Ahmed Hassan Awadallah. 2020. Gender bias in multilingual embeddings and cross-lingual transfer. In ACL. find
  37. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender bias in contextualized word embeddings. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1… find our card
  38. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018a. Gender bias in coreference resolution: Evaluation and debiasing methods. In NAACL-HLT. find our card
  39. Jieyu Zhao, Yichao Zhou, Z. Li, W. Wang, and Kai-Wei Chang. 2018b. Learning gender-neutral word embeddings. In EMNLP. find our card
  40. Pei Zhou, Weijia Shi, Jieyu Zhao, Kuan-Hao Huang, Muhao Chen, Ryan Cotterell, and Kai-Wei Chang. 2019. Examining gender bias in languages with grammatical gender. In EMNLP/IJCNLP. find
  41. Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2019. The woman worked as a babysitter: On biases in language generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Process… find our card
  42. Gabriel Stanovsky, Noah A. Smith, and Luke Zettlemoyer. 2019. Evaluating gender bias in machine translation. In ACL. find our card
  43. Rachael Tatman. 2017. Gender and dialect bias in youtube’s automatic captions. In EthNLP@EACL. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tarrant-2021-intrinsic-bias-metrics-do-not/