Elamur Research⌕
☰
home / papers / 2021 / Marked Attribute Bias in Natural Language Inference

Marked Attribute Bias in Natural Language Inference

Paper · 2021 · Hillary Dawkins

Summary

The paper reports a previously undocumented gender bias in natural language inference. When a premise mentions a generic "person", NLI models tend to treat a male hypothesis as entailed but a female one as merely neutral. In other words, masculinity acts as the unmarked default and femininity as a marked trait. The author builds test sets that probe this effect on three kinds of words: explicit gender words, names (implicit carriers) and stereotyped occupations (latent carriers). The model is a Decomposable Attention Model trained on SNLI, run on top of various static word embeddings.

Popular debiased embeddings, such as Hard Debias, GN-GloVe and Gender-Preserving, fail to remove the error. Existing intrinsic bias metrics also do not predict it. A new intrinsic measure, Multi-dimensional Information-weighted Direct Bias (MIDB), does correlate with the effect. It motivates MISP, a post-processing method that partially removes a multi-dimensional gender subspace, scaling each direction by its explained variance. Applied to GN-GloVe, MISP gives the lowest MAB error without harming similarity or analogy scores. The work ties embedding geometry to a concrete downstream inference error and releases reusable tests.

Key points

  • Setup: Decomposable Attention Model with intra-attention, trained on SNLI for 100 epochs; base embeddings are GloVe Common Crawl 840B.
  • Example of the bias: for "A person prepared a meal", the "He…" hypothesis gets Entail 0.929, while the "She…" hypothesis gets Neutral 0.687. The correct output for both is (N, E, C) = (1, 0, 0).
  • Difference from prior work: the MAB test differs from the occupation-based NLI set of Dev et al. (2020a), which has 1,936,512 pairs. MAB measures what gender words themselves carry, not their association with a chosen concept.
  • Main result: MISP applied to GN-GloVe(w_a) cuts MAB error by 26% versus the previous best and by 45% versus undebiased embeddings.
  • Hard variant fails: a multi-dimensional hard projection (all weights equal to 1) does not fix the bias (E = 0.1501).
  • Quality checks: embedding quality is verified on RG, MTurk, MEN, SimLex999, Google and MSR. SemBias shows no harmful analogies were reintroduced.
  • Availability: code and data are at github.com/hillary-dawkins/MAB.

Interesting ideas

  • I1. Build symmetric hypothesis pairs that differ only in a default versus a marked attribute, and require the same neutral verdict for both; any asymmetry exposes a hidden default assumption.
  • I2. Validate an internal diagnostic by checking whether it actually correlates with downstream error, rather than trusting its intuitive appeal.
  • I3. Use a permutation control: reassign the weights to the "wrong" components (23 alternatives) and confirm that the principled ordering beats all of them.
  • I4. Attenuate unwanted directions softly in proportion to how much signal they hold, instead of zeroing them; the paper shows hard removal failed where soft weighting worked.
  • I5. Weight learned directions by how far a probe beats chance, using a = 1 − 2·acc, so a dimension at 50% accuracy gets exactly zero weight.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): test coreference with "a person … he" versus "a person … she" pairs, where linking confidence and resulting frames should be identical, flagging gender defaults in the resolver.
  • I2 (learn): verify that internal signals like witness-parser agreement or tree confidence really predict wrong answers on school tests before using them to filter silver data.
  • I3 (learn): when weighting witness parsers or repair rules, shuffle the weights as a negative control to prove the weighting scheme itself carries information.
  • I4 (tree): if gparse representations need nuisance features suppressed, graded down-weighting may preserve parsing accuracy better than projecting features away entirely.
  • I5 (learn): score self-induced repair rules or probe features by their margin over chance, so rules that only perform at baseline are automatically switched off.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Vector InstituteToronto · Canada19 works · 4,281 citations🎓University of GuelphGuelph · Canada2 works · 245 citations

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 9,085 worksArtificial Intelligence · 8,888 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksSocial Sciences · 744 worksSafety Research · 221 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 510 works

Cite

Cite as Hillary Dawkins. 2021. Marked Attribute Bias in Natural Language Inference. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.findings-acl.369

@inproceedings{dawkins2021marked,
  title = {{Marked Attribute Bias in Natural Language Inference}},
  author = {Hillary Dawkins},
  booktitle = {Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021},
  year = {2021},
  doi = {10.18653/v1/2021.findings-acl.369},
  url = {https://doi.org/10.18653/v1/2021.findings-acl.369}
}

References

27 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. E. Bruni N. K. Tran M. Baroni. 2014. Multimodal distributional semantics. Journal of Artificial Intelligence Research, 49:1–47. ↗
  2. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems, volume 29, pages 4349–4357. Curran Associates, Inc. ↗
  3. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. ↗
  4. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal. Association for Computational Linguistics. find
  5. Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. ↗
  6. Brent D. Davis, Ethan Jackson, and Daniel J. Lizotte. 2020. Decision-directed data decomposition. ↗
  7. Sunipa Dev, Tao Li, Jeff M. Phillips, and Vivek Srikumar. 2020a. On measuring and mitigating biased inferences of word embeddings. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):7659–7666. ↗
  8. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013b. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, volume 26, pages 3111–3119. Curran Associates, Inc. ↗
  9. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013c. Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 746–751, Atlanta, Georgia. Association for Computational Linguistics.
  10. Sunipa Dev, Tao Li, Jeff M Phillips, and Vivek Srikumar. 2020b. Oscar: Orthogonal subspace correction and rectification of biases in word embeddings. find
  11. Ankur Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2249–2255, Austin, Texas. Association for Computational Linguistics. ↗
  12. Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. In Proceedings of NAACL-HLT. ↗
  13. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. ↗
  14. Felix Hill, Roi Reichart, and Anna Korhonen. 2015. Simlex-999: Evaluating semantic models with (genuine) similarity estimation. Computational Linguistics, 41(4):665–695. ↗
  15. Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. 2020. Null it out: Guarding protected attributes by iterative nullspace projection. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 7237– 7256. Association for Computational Linguistics.
  16. Stanislaw Jastrzebski, Damian Lesniak, and Wojciech Marian Czarnecki. 2017. How to evaluate word embeddings? on importance of data efficiency and simple supervised tasks. CoRR, abs/1702.02170. ↗
  17. Masahiro Kaneko and Danushka Bollegala. 2019. Gender-preserving debiasing for pre-trained word embeddings. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1641–1650, Florence, Italy. Association for Computational Linguistics. find
  18. Vaibhav Kumar, Tenzin Singhay Bhotia, Vaibhav Kumar, and Tanmoy Chakraborty. 2020. Nurse is closer to woman than surgeon? mitigating gender-biased proximities in word embeddings. Transactions of the Association for Computational Linguistics, 8:486–503. ↗
  19. Anne Lauscher, Goran Glavaš, Simone Paolo Ponzetto, and Ivan Vulić. 2019. A general framework for implicit and explicit debiasing of distributional word vector spaces. In Accepted for publication at the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2020). ↗
  20. Tomas Mikolov, Kai Chen, Greg S. Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. ↗
  21. Andrew Rosenberg and Julia Hirschberg. 2007. Vmeasure: A conditional entropy-based external cluster evaluation measure. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 410– 420, Prague, Czech Republic. Association for Computational Linguistics. find
  22. Herbert Rubenstein and John B. Goodenough. 1965. Contextual correlates of synonymy. Communications of the ACM, 8(10):627–633. ↗
  23. Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019. Mitigating gender bias in natural language processing: Literature review. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1630–1640, Florence, Italy. Association for Computational Linguistics. ↗
  24. Tianlu Wang, Xi Victoria Lin, Nazneen Fatema Rajani, Bryan McCann, Vicente Ordonez, and Caiming Xiong. 2020. Double-hard debias: Tailoring word embeddings for gender bias mitigation. In Association for Computational Linguistics (ACL). find
  25. Wen-tau Yih and Vahed Qazvinian. 2012. Measuring word relatedness using heterogeneous vector space models. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 616–620, Montréal, Canada. Association for Computational Linguistics. find
  26. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018a. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 15–20, New Orleans, Louisiana. Association for Computational Linguistics. ↗
  27. Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018b. Learning gender-neutral word embeddings. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4847–4853, Brussels, Belgium. Association for Computational Linguistics. A

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/dawkins-2021-marked-attribute-bias-natural-language/