Elamur Research
☰
home / papers / 2018 / Gender Bias in Coreference Resolution

Gender Bias in Coreference Resolution

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/N18-2002.pdf

Summary

Coreference systems can absorb stereotypes from their training data and lexical resources, but in 2018 there were few targeted tests for this. The authors built Winogender schemas, a diagnostic set modeled on Winograd schemas. It has 120 handwritten templates covering 60 occupations. In each sentence, a pronoun points either to a person named by their occupation or to a second participant, and only the pronoun's gender (female, male or neutral) varies. Fully expanded, the set contains 720 sentences, which Mechanical Turk workers validated. The authors tested three off-the-shelf English systems: a rule-based sieve, a feature-based statistical model and a neural reinforcement-learning model. All three changed their answers depending on pronoun gender, even though the correct reading never depends on it. Each system's occupation-level preferences tracked U.S. Bureau of Labor Statistics figures and the gender counts in the Bergsma and Lin (B&L) noun resource. That resource is itself skewed toward men compared with real employment data. The work gives the field a public, reusable probe and shows how bias can grow at each stage of an NLP pipeline. The authors note that such a test can expose bias but cannot prove a system is free of it.

Key points

  • The dataset crosses 60 occupations × 2 templates × 2 participants (a specific one or "someone") × 3 pronouns, giving 720 sentences. Each sentence got 10 MTurk judgments: 94.9% of individual answers matched the intended answer, and the majority vote matched for 718 of 720 sentences.
  • Male–female minimal pairs got different resolutions in 68% of cases for RULE (Stanford sieve, Lee et al. 2011), 28% for STAT (Durrett & Klein 2013) and 13% for NEURAL (Clark & Manning 2016a).
  • Male pronouns were linked to the occupation more often than female ones in every system: 72% vs 29% for RULE, 71% vs 63% for STAT and 87% vs 80% for NEURAL.
  • The correlation (r) between system bias and B&L / BLS statistics was 0.87 / 0.55 for RULE, 0.46 / 0.31 for STAT and 0.35 / 0.31 for NEURAL. B&L and BLS correlate with each other at r = 0.67. Example: "manager" is 38.5% female in BLS data and 5.18% female in B&L, and none of the three systems ever resolved a female pronoun to it.
  • Accuracy dropped for every system on "gotcha" sentences, where the pronoun's gender goes against the occupation's majority gender. The data is released at github.com/rudinger/winogender-schemas. WinoBias (Zhao et al. 2018) is a closely related benchmark published at the same time.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

423 papers of our library cite it (of the 140673 whose bibliographies we read); 53 of them have a page here, 370 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n18-2002 · arXiv 1804.09301

@inproceedings{rudinger2018gender,
  title = {{Gender Bias in Coreference Resolution}},
  author = {Rudinger, Rachel and Naradowsky, Jason and Leonard, Brian and Durme, Benjamin Van},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-2002},
  eprint = {1804.09301},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/n18-2002}
}

References

25 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Solon Barocas and Andrew D Selbst. 2016. Big data’s disparate impact. California Law Review, pages 104(3):671–732. find
  2. Corina Koolen and Andreas van Cranenburgh. 2017. These are not the stereotypes you are looking for: Bias and fairness in authorial gender attribution. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, pages 12–22, Valencia, Spain. Association for Computational Linguistics. find
  3. Shane Bergsma and Dekang Lin. 2006. Bootstrapping path-based pronoun resolution. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 33–40, Sydney, Australia. Association for Computational Linguistics. find
  4. Brian Larson. 2017. Gender as a variable in natural-language processing: Ethical considerations. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, pages 1–11, Valencia, Spain. Association for Computational Linguistics. find
  5. Alex Beutel, Jilin Chen, Zhe Zhao, and Ed H. Chi. 2017. Data decisions and theoretical implications when adversarially learning fair representations. CoRR, abs/1707.00075. ↗
  6. Anders Björkelund and Jonas Kuhn. 2014. Learning structured perceptrons for coreference resolution with latent antecedents and non-local features. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 47–57, Baltimore, Maryland. Association for Computational Linguistics. find
  7. Heeyoung Lee, Yves Peirsman, Angel Chang, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. 2011. Stanford’s multi-pass sieve coreference resolution system at the conll-2011 shared task. In Conference on Natural Language Learning (CoNLL) Shared Task. find our card
  8. Hector J. Levesque, Ernest Davis, and Leora Morgenstern. 2011. The winograd schema challenge. In KR. find
  9. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 4349–4357. Curran Associates, Inc. find our card
  10. Emiel van Miltenburg. 2016. Stereotyping and bias in the flickr30k dataset. MMC. find
  11. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In Proceedings of the Sixteenth Conference on Computational Natural Language Learning (CoNLL 2012), Jeju, Korea. find our card
  12. Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. find
  13. Sameer Pradhan, Lance Ramshaw, Mitchell Marcus, Martha Palmer, Ralph Weischedel, and Nianwen Xue. 2011. Conll-2011 shared task: Modeling unrestricted coreference in ontonotes. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning (CoNLL 2011), Portland, Oregon. find our card
  14. Kevin Clark and Christopher D. Manning. 2016a. Deep reinforcement learning for mention-ranking coreference models. In Empirical Methods on Natural Language Processing (EMNLP). find our card
  15. Kevin Clark and Christopher D. Manning. 2016b. Improving coreference resolution by learning entity-level distributed representations. In Association for Computational Linguistics (ACL). find our card
  16. Rachel Rudinger, Chandler May, and Benjamin Van Durme. 2017. Social bias in elicited natural language inferences. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, pages 74–79, Valencia, Spain. Association for Computational Linguistics. find our card
  17. Greg Durrett and Dan Klein. 2013. Easy victories and uphill battles in coreference resolution. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Seattle, Washington. Association for Computational Linguistics. find our card
  18. Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. 2018. Word embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences. find
  19. Rachael Tatman. 2017. Gender and dialect bias in youtube’s automatic captions. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, pages 53–59, Valencia, Spain. Association for Computational Linguistics. find
  20. Dirk Hovy and Shannon L. Spruit. 2016. The social impact of natural language processing. In Proceedings of the 54th Annual Meeting of the Association Mikaela Wapman and Deborah Belle. 2014. Undergraduate thesis. https://mikaelawapman.com/category/gender-schemas/. ↗ our card
  21. Sam Wiseman, Alexander M. Rush, Stuart Shieber, and Jason Weston. 2015. Learning anaphoricity and antecedent ranking features for coreference resolution. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1416–1426. Association for Computational Linguistics. find our card
  22. Sam Wiseman, Alexander M. Rush, and Stuart M. Shieber. 2016. Learning global features for coreference resolution. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 994–1004, San Diego, California. Association for Computational Linguistics. find our card
  23. Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018. Mitigating unwanted biases with adversarial learning. In Proceedings of the First AAAI/ACM Conference on AI, Ethics, and Society. find
  24. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2979–2989, Copenhagen, Denmark. Association for Computational Linguistics. find our card
  25. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Louisiana. Association for Computational Linguistics. find our card