Elamur Research⌕
☰
home / papers / 2020 / Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Bias

Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Bias

Paper · 2020 · Marion Bartl, Malvina Nissim and Albert Gatt

Summary

The paper examines whether BERT ties gendered person words to occupations, whether that tie reflects real employment patterns or stereotypes, and whether it can be weakened. The authors built BEC-Pro, a template corpus in English and German. Each sentence pairs a gendered noun phrase with a profession. Professions were chosen from 2019 U.S. Bureau of Labor Statistics figures and grouped as female-dominated, male-dominated or gender-balanced. Following Kurita et al. (2019), they query BERT's masked language model and measure how a profession shifts the probability of the masked person word. To reduce bias, they gender-swap the GAP corpus with Counterfactual Data Substitution (CDS) and fine-tune uncased BERT BASE on it.

In English, BERT mirrors labour-market skew. It also links women weakly to balanced professions, which points to stereotype rather than statistics. Fine-tuning moves associations in the intended direction. In German, agreement between grammatical gender and profession forms dominates the scores, so the method does not transfer. The work matters because bias probes for multilingual models need checking language by language.

Key points

  • BEC-Pro uses three groups of 20 professions each, defined by female workforce share:
    • female-typical: 88.3–98.7%
    • male-typical: 0.7–3.3%
    • balanced: 48.5–53.3%
  • Long statistical job labels were shortened to forms likely found in BERT's vocabulary (e.g., a long clerk category became "bookkeeper"). German masculine and feminine equivalents came from dict.cc and were checked by a native speaker; most feminine forms use the suffix -in.
  • The resources involved are:
    • the EEC (8,640 sentences from 11 templates), used for preliminary tests and as a source of person words;
    • GAP (8,908 pronoun–name pairs in 4,454 Wikipedia contexts), used for CDS-based fine-tuning.
  • After fine-tuning, pro-typical associations dropped for female terms with female-typical jobs, and anti-typical associations rose. Female person words changed more than male ones. The authors suggest two possible causes: male terms hold a more stable place in BERT, and GAP contained slightly more female words and names after CDS.
  • Dataset and code are released at github.com/marionbartl/gender-bias-BERT. The authors note two main limitations: only one model was tested, and the curated templates and word lists shape the outcome.

Interesting ideas

  • I1. Anchor attribute lists in external statistics. Choosing professions by real workforce shares lets you tell whether a model's association is "accurate" or stereotypical. The same trick suits any probe where ground-truth distributions exist, such as regional or trade data.
  • I2. Use a balanced group as a control. Professions with roughly equal gender shares should show no preference. Any association there is a clean signal of stereotype, a useful design pattern for bias benchmarks generally.
  • I3. Debias by fine-tuning on a small swapped corpus. Applying CDS to a modest dataset like GAP and fine-tuning avoids costly retraining. This makes it a practical first step for adapting existing pretrained models.
  • I4. Validate probes per language, not per family. German fails despite being related to English, because grammatical gender agreement swamps the semantic signal. For gender-marking languages such as Ukrainian, a probe should separate morphological agreement from the social association before its scores are trusted.
  • I5. Watch for markedness and frequency effects. Rare derived feminine forms can shift model probabilities on their own. Controlling for word frequency or form type helps avoid misreading such effects as bias.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of GroningenGroningen · Netherlands49 works · 4,843 citations🎓University of MaltaMsida · Malta16 works · 775 citations

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 9,085 worksArtificial Intelligence · 8,888 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksSocial Sciences · 744 worksSafety Research · 221 works

🧭 Topics 1Research subjects and languages the work is about.Bias, fairness & safety · 209 works

Cite

Cite as Marion Bartl, Malvina Nissim and Albert Gatt. 2020. Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Bias. Proceedings of the Second Workshop on Gender Bias in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{bartl2020unmasking,
  title = {{Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Bias}},
  author = {Marion Bartl, Malvina Nissim and Albert Gatt},
  booktitle = {Proceedings of the Second Workshop on Gender Bias in Natural Language Processing},
  year = {2020},
  url = {https://aclanthology.org/2020.gebnlp-1.1/}
}

References

37 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Douglas Adams. 2017. The Ultimate Hitchhiker’s Guide to the Galaxy, volume 6. Pan Macmillan. find
  2. Christine Basta, Marta R Costa-jussà, and Noe Casas. 2019. Evaluating the underlying gender bias in contextualized word embeddings. arXiv preprint arXiv:1904.08783. ↗
  3. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of “bias” in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454–5476, Online, July. Association for Computational Linguistics.
  4. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in neural information processing systems, pages 4349–4357. ↗
  5. Bureau of Labor Statistics. 2020. Labor force statistics from the current population survey, January. [Online; accessed 16-March-2020]. find
  6. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. ↗
  7. Greville G. Corbett. 2013. Number of genders. In Matthew S. Dryer and Martin Haspelmath, editors, The World Atlas of Language Structures Online. Max Planck Institute for Evolutionary Anthropology, Leipzig. find
  8. Marta R. Costa-jussà, Christian Hardmeier, Will Radford, and Kellie Webster, editors. 2019. Proceedings of the First Workshop on Gender Bias in Natural Language Processing. Association for Computational Linguistics, Florence, Italy, August. find
  9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805. ↗
  10. Matthew S. Dryer and Martin Haspelmath, editors. 2013. WALS Online. Max Planck Institute for Evolutionary Anthropology, Leipzig. find
  11. Andy Field, Jeremy Miles, and Zoë Field. 2012. Discovering statistics using R. Sage publications. find
  12. Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. arXiv preprint arXiv:1903.03862. ↗
  13. Hila Gonen, Yova Kementchedjhieva, and Yoav Goldberg. 2019. How does grammatical gender affect noun representations in gender-marking languages? arXiv preprint arXiv:1910.14161. ↗
  14. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. In Proceedings of ACL. find
  15. Dirk Hovy and Shannon L Spruit. 2016. The social impact of natural language processing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 591–598. ↗
  16. Dirk Hovy, Shannon Spruit, Margaret Mitchell, Emily M. Bender, Michael Strube, and Hanna Wallach, editors. 2017. Proceedings of the First ACL Workshop on Ethics in Natural Language Processing. Association for Computational Linguistics, Valencia, Spain, April. find
  17. Svetlana Kiritchenko and Saif M Mohammad. 2018. Examining gender and race bias in two hundred sentiment analysis systems. arXiv preprint arXiv:1805.04508. ↗
  18. Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019. Measuring bias in contextualized word representations. arXiv preprint arXiv:1906.07337. ↗
  19. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942. ↗
  20. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. ↗
  21. Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Amancharla, and Anupam Datta. 2018. Gender bias in neural natural language processing. arXiv preprint arXiv:1807.11714. ↗
  22. Rowan Hall Maudslay, Hila Gonen, Ryan Cotterell, and Simone Teufel. 2019. It’s all in the name: Mitigating gender bias with name-based counterfactual data substitution. arXiv preprint arXiv:1909.00871. ↗
  23. Chandler May, Alex Wang, Shikha Bordia, Samuel R Bowman, and Rachel Rudinger. 2019. On measuring social biases in sentence encoders. arXiv preprint arXiv:1903.10561. ↗
  24. Corinne A Moss-Racusin, John F Dovidio, Victoria L Brescoll, Mark J Graham, and Jo Handelsman. 2012. Science faculty’s subtle gender biases favor male students. Proceedings of the national academy of sciences, 109(41):16474–16479. ↗
  25. Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365. ↗
  26. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, 1(8):9. find
  27. Robert Rosenthal. 1991. Applied social research methods series, Vol. 6. Meta-analytic procedures for social research (Rev. ed.). Sage Publications, Inc. https://doi.org/10.4135/9781412984997. ↗
  28. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. arXiv preprint arXiv:1804.09301. ↗
  29. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. ↗
  30. Deven Shah, H Andrew Schwartz, and Dirk Hovy. 2019. Predictive biases in natural language processing models: A conceptual framework and overview. arXiv preprint arXiv:1912.11078. ↗
  31. Paul Stocker. 2012. Nouns. In A Student Grammar of German, page 11–29. Cambridge University Press. find
  32. Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019. Mitigating gender bias in natural language processing: Literature review. arXiv preprint arXiv:1906.08976. ↗
  33. Kellie Webster, Marta Recasens, Vera Axelrod, and Jason Baldridge. 2018. Mind the gap: A balanced corpus of gendered ambiguous pronouns. In Transactions of the ACL, page to appear. ↗
  34. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface’s transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771. ↗
  35. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender bias in contextualized word embeddings. arXiv preprint arXiv:1904.03310. ↗
  36. Pei Zhou, Weijia Shi, Jieyu Zhao, Kuan-Hao Huang, Muhao Chen, Ryan Cotterell, and Kai-Wei Chang. 2019. Examining gender bias in languages with grammatical gender. arXiv preprint arXiv:1909.02224. ↗
  37. Ran Zmigrod, Sebastian J Mielke, Hanna Wallach, and Ryan Cotterell. 2019. Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology. arXiv preprint arXiv:1906.04571. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bartl-2020-unmasking-contextual-stereotypes-measuring-mitigating/