Elamur Research
☰
home / papers / 2017 / Social Bias in Elicited Natural Language Inferences

Social Bias in Elicited Natural Language Inferences

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/W17-1609.pdf

Summary

The paper investigates whether the Stanford Natural Language Inference (SNLI) corpus contains social stereotypes, and in particular whether its data-collection protocol introduces them. In SNLI, crowdworkers read a Flickr30k caption without seeing the photo and wrote a new sentence that was certainly true, possibly true, or certainly false relative to it. The authors separate bias already present in the captions (caption bias) from bias added when workers wrote hypotheses (elicitation bias), and they target the latter. Their analysis does not depend on any particular model. They compute pointwise mutual information (PMI) between words in premises and words in the paired hypotheses, check significance with likelihood ratio tests, and add qualitative examples. Query terms cover identity categories based on US EEOC discrimination types, such as gender, age, race, and religion. The hypotheses show strong gender stereotypes, especially under the NEUTRAL label, along with weaker racial, religious, and age-related ones. Because models trained on SNLI may learn these associations and overgeneralize them, crowdsourced inference datasets warrant this kind of audit.

Key points

  • SNLI has about 570K premise–hypothesis pairs labeled ENTAILMENT, NEUTRAL, or CONTRADICTION. Its premises are Flickr30k captions.
  • The method counts co-occurrences after lowercasing the Stanford tokenization and drops pairs seen fewer than five times. Candidate pairs are ranked by PMI and tested with Dunning-style likelihood ratio tests. Holm-Bonferroni correction is applied separately to the full corpus and to each label subset.
  • Gendered associations dominate the results. Female terms co-occur with words like gossip, hairdresser, and husbands, while male terms co-occur with words like workers and engineers. Girls and boys also receive stereotyped activities.
  • Terms referring to Asian people cluster around food and the pejorative word "oriental." Many race and nationality queries return other ethnic terms, reflecting a habit of simply swapping ethnicity to write contradictions.
  • Code and the query list are released at github.com/cjmay/snli-ethics.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

71 papers of our library cite it (of the 140673 whose bibliographies we read); 16 of them have a page here, 55 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w17-1609

@inproceedings{rudinger2017social,
  title = {{Social Bias in Elicited Natural Language Inferences}},
  author = {Rudinger, Rachel and May, Chandler and Durme, Benjamin Van},
  booktitle = {Proceedings of the First ACL Workshop on Ethics in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/w17-1609},
  url = {https://doi.org/10.18653/v1/w17-1609}
}

References

17 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Solon Barocas and Andrew D. Selbst. 2016. Big data’s disparate impact. California Law Review, 104(3):671–732. find
  2. Tolga Bolukbasi, Kai-Wei Chang, James Y. Zou, Venkatesh Saligrama, and Adam T. Kalai. 2016. find
  3. Dirk Hovy and Shannon L. Spruit. 2016. The social impact of natural language processing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 591–598, Berlin, Germany, August. Association for Computational Linguistics. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 4349–4357. find our card
  4. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal, September. Association for Computational Linguistics. find
  5. Dekang Lin and Patrick Pantel. 2001. Dirt – discovery of inference rules from text. In Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, pages 323– 328. ACM. find
  6. Emiel van Miltenburg. 2016. Stereotyping and bias in the Flickr30K dataset. In Jens Edlund, Dirk Heylen, and Patrizia Paggio, editors, Proceedings of the Workshop on Multimodal Corpora (MMC-2016), pages 1–4, May. find
  7. Aylin Caliskan-Islam, Joanna J. Bryson, and Arvind Narayanan. 2016. Semantics derived automatically from language corpora necessarily contain human biases. Preprint, arXiv:1608.07187. ↗
  8. Jordan Carpenter, Daniel Preotiuc-Pietro, Lucie Flekova, Salvatore Giorgi, Courtney Hagan, Margaret L. Kern, Anneke E. K. Buffone, Lyle Ungar, and Martin E. P. Seligman. 2017. Real men dont say “cute”: Using automatic language analysis to isolate inaccurate aspects of stereotypes. to appear. find
  9. Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014. From image descriptions to visual denotations. Transactions of the Association of Computational Linguistics, 2:67–78. find
  10. Nathanael Chambers and Dan Jurafsky. 2009. Unsupervised learning of narrative schemas and their participants. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 602–610, Suntec, Singapore, August. Association for Computational Linguistics. find
  11. Kenneth Ward Church and Patrick Hanks. 1990. Word association norms mutual information, and lexicography. Computational Linguistics, Volume 16, Number 1, March 1990, 16(1). find our card
  12. Kate Crawford, Kate Miltner, and Mary L. Gray. 2014. Critiquing big data: Politics, ethics, epistemology. International Journal of Communication, 8:1663– 1672. find
  13. Kate Crawford. 2013. Think again: Big data. http://atfp.co/2k9jaBT. Accessed 2017-01-26. ↗
  14. Kate Crawford. 2016. Artificial intelligence’s white guy problem. http://nyti.ms/2jVLJUh. Accessed 2017-01-22. ↗
  15. Ted Dunning. 1993. Accurate methods for the statistics of surprise and coincidence. Computational Linguistics, Special Issue on Using Large Corpora: I, 19(1). find our card
  16. Liye Fu, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2016. Tie-breaker: Using language models to quantify gender bias in sports journalism. In Proceedings of the IJCAI workshop on NLP meets Journalism. find
  17. Sture Holm. 1979. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2):65–70. find