Elamur Research
☰
home / papers / 2018 / Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems

Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/S18-2005.pdf

Summary

Automatic NLP systems can pick up and amplify unfair human biases, but earlier studies usually checked only one or two systems and had no shared benchmark for doing so. Kiritchenko and Mohammad address this by building the Equity Evaluation Corpus (EEC), a set of 8,640 short, simple English sentences. These come from eleven templates. Seven include emotion words drawn from Roget's Thesaurus categories for anger, fear, joy and sadness, and four are neutral. Each sentence is filled with a gender-marked noun phrase or a first name typical of African American or European American people. The EEC was given as an extra test set in SemEval-2018 Task 1 'Affect in Tweets'. The authors then compared the scores that 219 submitted systems assigned to sentence pairs differing in just one gender or race word. Over three quarters of the systems showed statistically significant bias, and race bias was more widespread than gender bias. The average gaps were small, but some were large. The work matters because it offers a reusable, freely available tool for auditing fairness in sentiment analysis.

Key points

  • The EEC has 8,640 sentences from 11 templates. Names come from Caliskan et al. (2017), with ten per race/gender group, plus paired female/male noun phrases such as "my daughter" and "my son".
  • The study evaluated 219 systems on emotion intensity (anger, fear, joy, sadness) and on valence.
  • More than 75% of the systems were biased. For race, only about 11–24% of submissions on the emotion tasks and about 8% on valence showed no significant difference.
  • Mean score gaps stayed below 0.03 on the 0–1 scale, but some systems reached 0.34. The direction of bias depended on the affect dimension: sentences with African American names were scored higher for anger, fear and sadness, while joy and valence mostly leaned the other way.
  • Even the resource-free SVM-Unigrams baseline was biased (gender spreads of 0.09–0.2, race below 0.002). This suggests the distantly collected training tweets are themselves a source of bias.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

195 papers of our library cite it (of the 140673 whose bibliographies we read); 17 of them have a page here, 178 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/s18-2005

@inproceedings{kiritchenko2018examining,
  title = {{Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems}},
  author = {Kiritchenko, Svetlana and Mohammad, Saif},
  booktitle = {Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics},
  year = {2018},
  doi = {10.18653/v1/s18-2005},
  url = {https://doi.org/10.18653/v1/s18-2005}
}

References

31 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. of the 54th Annual Meeting of the Association for Computational Linguistics, volume 2, pages 591– 598. find
  2. Su Lin Blodgett, Lisa Green, and Brendan O’Connor. 2016. Demographic dialectal variation in social media: A case study of African-American English. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1119–1130, Austin, Texas. find our card
  3. Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 168–177, Seattle, WA, USA. find
  4. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS), pages 4349– 4357. find our card
  5. David Jurgens, Yulia Tsvetkov, and Dan Jurafsky. 2017. Incorporating dialectal variability for socially equitable language identification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 51–57. find
  6. Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proceedings of the Conference on Fairness, Accountability and Transparency, pages 77–91. find
  7. Niki Kilbertus, Mateo Rojas Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, and Bernhard Schölkopf. 2017. Avoiding discrimination through causal reasoning. In Advances in Neural Information Processing Systems, pages 656–666. find
  8. Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. find
  9. Svetlana Kiritchenko, Xiaodan Zhu, and Saif M. Mohammad. 2014. Sentiment analysis of short informal texts. Journal of Artificial Intelligence Research, 50:723–762. find
  10. Alexandra Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data, 5(2):153–163. find
  11. Ryan Kiros, Yukun Zhu, Ruslan R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Skip-thought vectors. In Advances in Neural Information Processing Systems, pages 3294–3302. find
  12. Amit Datta, Michael Carl Tschantz, and Anupam Datta. 2015. Automated experiments on ad privacy settings. Proceedings on Privacy Enhancing Technologies, 2015(1):92–112. find
  13. Saif M. Mohammad. 2018. Word affect intensities. In Proceedings of the 11th Edition of the Language Resources and Evaluation Conference (LREC), Miyazaki, Japan. find
  14. Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, and Sune Lehmann. 2017. Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1… find our card
  15. Saif M. Mohammad and Felipe Bravo-Marquez. 2017. WASSA-2017 shared task on emotion intensity. In Proceedings of the Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA), Copenhagen, Denmark. find
  16. Gabriel Goh, Andrew Cotter, Maya Gupta, and Michael P Friedlander. 2016. Satisfying real-world goals with dataset constraints. In Advances in Neural Information Processing Systems, pages 2415– 2423. find
  17. Saif M. Mohammad, Felipe Bravo-Marquez, Mohammad Salameh, and Svetlana Kiritchenko. 2018. Semeval-2018 Task 1: Affect in tweets. In Proceedings of International Workshop on Semantic Evaluation (SemEval-2018), New Orleans, LA, USA. find our card
  18. Sara Hajian and Josep Domingo-Ferrer. 2013. A methodology for direct and indirect discrimination prevention in data mining. IEEE Transactions on Knowledge and Data Engineering, 25(7):1445– 1459. find
  19. Saif M. Mohammad and Peter D. Turney. 2013. Crowdsourcing a word–emotion association lexicon. Computational Intelligence, 29(3):436–465. find
  20. Finn Årup Nielsen. 2011. A new ANEW: Evaluation of a word list for sentiment analysis in microblogs. In Proceedings of the ESWC Workshop on ’Making Sense of Microposts’: Big things come in small packages, pages 93–98, Heraklion, Crete. find
  21. Dirk Hovy. 2015. Demographic factors improve classification performance. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, volume 1, pages 752–762. find
  22. Dino Pedreshi, Salvatore Ruggieri, and Franco Turini. 2008. Discrimination-aware data mining. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 560–568. find
  23. Dirk Hovy and Shannon L Spruit. 2016. The social impact of natural language processing. In Proceedings Danielle Popp, Roxanne Angela Donovan, Mary Crawford, Kerry L. Marsh, and Melanie Peele. 2003. Gender, race, and speech style stereotypes. Sex Roles, 48(7):317–325. find our card
  24. Hee Jung Ryu, Margaret Mitchell, and Hartwig Adam. 2017. Improving smiling detection with race and gender diversity. arXiv preprint arXiv:1712.00193. — [arXiv:1712.00193](https://arxiv.org/abs/1712.00193) find
  25. Ben Schmidt. 2015. Rejecting the gender binary: a vector-space operation. http://bookworm.benschmidt.org/posts/2015-10-30-rejecting-the-gender-binary.html. ↗
  26. Stephanie A. Shields. 2002. Speaking from the heart: Gender and the social meaning of emotion. Cambridge, U.K.: Cambridge University Press. find
  27. Rob Speer. 2017. ConceptNet Numberbatch 17.04: better, less-stereotyped word vectors. https://blog.conceptnet.io/2017/04/24/conceptnet-numberbatch-17-04-better-less-stereotyped-word-vectors/. ↗
  28. Latanya Sweeney. 2013. Discrimination in online ad delivery. Queue, 11(3):10. find
  29. Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018. Mitigating unwanted biases with adversarial learning. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. find
  30. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the Annual Conference of the North American Chapter of the ACL (NAACL). find our card
  31. Indre Zliobaite. 2015. A survey on measuring indirect discrimination in machine learning. arXiv preprint arXiv:1511.00148. — [arXiv:1511.00148](https://arxiv.org/abs/1511.00148) find