📄 Evaluating the Underlying Gender Bias in Contextualized Word Embeddings

Summary

This workshop paper asks whether contextualized word embeddings carry gender bias, and how much compared with standard and debiased static embeddings. The authors use ELMo, chosen over BERT because ELMo gives word-level vectors rather than subwords. They extract representations from the English side of the WMT18 English-German news corpus. They adapt existing bias tests to the setting where a word has a different vector in each sentence: the gender-subspace and direct-bias measures of Bolukbasi et al. (2016), and the clustering, classification and nearest-neighbour probes of Gonen and Goldberg (2019). They use word lists from those works: definitional pairs, professions, and biased and extended biased lists.

ELMo vectors show a weaker gender subspace, lower direct bias and weaker male/female clustering than static embeddings, even debiased ones. However, a classifier still recovers gender fairly well, and professions' neighbourhoods remain strongly stereotyped. The work shows that bias in contextual models is reduced on some axes but persists, and that no single metric captures it.

Key points

• Data and lists
  • Corpus: English side of the WMT18 news commentary data (464,947 lines).
  • Lists: 'Definitional List' (pairs like she–he), 'Professional List', 'Biased List' (500 female- and 500 male-biased tokens), and 'Extended Biased List' (5000 tokens).
• Clustering: k-means with 2 clusters on Biased List tokens.
  • ELMo averaged 70.1% accuracy over 10 runs (range 69.1–71.3%).
  • Reported static-embedding results were 99.9% (original) and 92.5% (debiased).
• Classification: an RBF-kernel SVM was trained on 1000 Extended Biased List words and tested on the remaining 4000.
  • ELMo averaged 85.56% (range 83.33–88.43%).
  • Static embeddings reached 98.25% (biased) and 88.88% (debiased).
• K-nearest neighbours on professions: the share of female/male-stereotyped neighbours was correlated with each profession's original bias.
  • ELMo's average Pearson correlation was 0.89 (range 0.801–0.961, p < 1e-40).
  • This is higher than for static embeddings: 0.774 (biased) and 0.606 (debiased).
  • On this axis, contextual vectors look more stereotyped.
• Overall conclusion: the paper does not claim that contextualization clearly removes bias. Instead, it lists the aspects that improve: gender space, direct bias and clustering.

Interesting ideas

• I1. Counterfactual swapping for the gender direction: instead of comparing two words in different contexts, take one sentence, replace the gendered word with its opposite-gender partner, and subtract the two contextual vectors. This keeps the context fixed and isolates the attribute. The same trick could probe other attributes in any contextual model.
• I2. Filtering confounding contexts: when measuring direct bias, sentences containing both a profession and an explicit gender word were dropped. This keeps the gender word's influence from leaking into the target vector, a useful precaution for any context-dependent probe.
• I3. Repeated sampling of one context per word: static-embedding tests expect one vector per word. Running the test 10 times, each with a randomly chosen sentence per word, adapts them to contextual models and gives a range rather than a single number.
• I4. Matching model granularity to the analysis: picking a word-level model (ELMo) avoids having to merge subword vectors, which could distort word-level measurements. This is worth considering for morphologically rich languages, where subword splitting is heavy.
• I5. Using several complementary metrics: the measures disagreed. Clustering suggested little bias, while nearest-neighbour correlation suggested strong bias. Any bias audit should therefore report a battery of tests rather than relying on one score.