Elamur Research⌕
☰
home / papers / 2019 / Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview

Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview

Summary

Research on bias in NLP has grown as a set of isolated studies. Each one defines and fixes its own kind of "bias" without shared terminology, so work often treats symptoms rather than causes. Shah, Schwartz and Hovy propose a unifying conceptual framework built from a survey of NLP literature and selected social-science work. They adapt Swinton's (1981) notion of predictive bias from standardized testing and formalize two measurable consequences:

  • Outcome disparity: predictions conditioned on a human attribute deviate from an application-specific ideal distribution.
  • Error disparity: errors are unevenly distributed across attribute values, whether discrete or continuous.

They then locate four sources of bias within a standard supervised pipeline: bias in the annotated labels, bias in how the data sample is chosen, the model itself amplifying bias beyond what the data contain, and semantic bias carried by pre-trained embeddings. Existing countermeasures are grouped under these origins, and the paper ends with a practical checklist. A central claim is that bias belongs to a model in a specific application, not to the model alone. Naming where a bias comes from makes biases and mitigations easier to identify and compare across papers.

Key points

  • Outcome disparity, formally Q(Ŷ|A) ≁ P(Y|A). Example: image captioning models over-predict women in images with ovens and men in images with snowboards (Zhao et al., 2017; Hendricks et al., 2018).
  • Error disparity, formally Q(ε|A_i) ≁ Q(ε|A_j). Example: the "Wall Street Journal Effect" (Hovy and Søgaard, 2015), where POS tagging errors increase as authors differ demographically from the 1980s–90s WSJ writers whose texts trained the taggers. This is a selection-bias case.
  • Origins mapped to pipeline stages: embedding corpus (semantic bias), source sample (selection bias), annotator labels (label bias), and model fitting (over-amplification, e.g., MT outputs sounding older and more male).
  • Suggested remedies by origin:
    • Label bias: post-stratification or annotator retraining.
    • Selection bias: stratified sampling or re-weighting.
    • Over-amplification: synthetic distribution matching or adding disparity to the cost function.
    • Semantic bias: hard/soft embedding de-biasing (Bolukbasi et al., 2016).
  • Community-level documentation: Data Statements (Bender and Friedman, 2018), Datasheets (Gebru et al., 2018) and model reporting across user groups (Mitchell et al., 2019) are presented as tools for surfacing label and selection bias.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

102 papers of our library cite it (of the 140884 whose bibliographies we read); 13 of them have a page here, 89 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.acl-main.468 · arXiv 1912.11078

@misc{shah2019predictive,
  title = {{Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview}},
  author = {Shah, Deven and Schwartz, H. Andrew and Hovy, Dirk},
  year = {2019},
  doi = {10.18653/v1/2020.acl-main.468},
  eprint = {1912.11078},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2020.acl-main.468}
}

References

94 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jussara M Almeida, Gisele L Pappa, et al. 2015. Twitter population sample bias and its impact on predictive outcomes: A case study on elections. In Proceedings of the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2015, pages 1254–1261. ACM. find
  2. Fatemeh Almodaresi, Lyle Ungar, Vivek Kulkarni, Mohsen Zakeri, Salvatore Giorgi, and H. Andrew Schwartz. 2017. On the distribution of lexical features at multiple levels of analysis. In The 55th Annual Meeting of the Association for Computational Linguistics. find
  3. Ron Artstein and Massimo Poesio. 2008. Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4):555–596. find
  4. Reg Baker, J Michael Brick, Nancy A Bates, Mike Battaglia, Mick P Couper, Jill A Dever, Krista J Gile, and Roger Tourangeau. 2013. Summary report of the aapor task force on non-probability sampling. Journal of Survey Statistics and Methodology, 1(2):90–143. find
  5. Roy F Baumeister, Kathleen D Vohs, and David C Funder. 2007. Psychology as the science of self-reports and finger movements: Whatever happened to actual behavior? Perspectives on Psychological Science, 2(4):396–403. find
  6. Emily M Bender and Batya Friedman. 2018. Data statements for natural language processing: Toward mitigating system bias and enabling better science. Transactions of the Association for Computational Linguistics, 6:587–604. find
  7. Adrian Benton, Margaret Mitchell, and Dirk Hovy. 2017. Multitask learning for mental health conditions with limited social media data. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, volume 1, pages 152–162. find
  8. Richard A Berk. 1983. An introduction to sample selection bias in sociological data. American Sociological Review, pages 386–398. find
  9. Sudeep Bhatia. 2017. Associative judgment and vector space semantics. Psychological review, 124(1):1. find
  10. Su Lin Blodgett and Brendan O’Connor. 2017. Racial disparity in natural language processing: A case study of social media African-American English. arXiv preprint arXiv:1707.00061. find
  11. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in neural information processing systems, pages 4349–4357. find our card
  12. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. find
  13. Murphy Choy, Michelle Cheong, Ma Nang Laik, and Koo Ping Shung. 2012. Us presidential election 2012 prediction using census corrected twitter model. arXiv preprint arXiv:1211.0938. — [arXiv:1211.0938](https://arxiv.org/abs/1211.0938) find
  14. Murphy Choy, Michelle LF Cheong, Ma Nang Laik, and Koo Ping Shung. 2011. A sentiment analysis of singapore presidential election 2011 using twitter data with census correction. arXiv preprint arXiv:1108.5520. — [arXiv:1108.5520](https://arxiv.org/abs/1108.5520) find
  15. Christopher Clark, Mark Yatskar, and Luke Zettlemoyer. 2019. DonâĂŹt take the easy way out: Ensemble based methods for avoiding known dataset biases. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lang… find
  16. Maximin Coavoux, Shashi Narayan, and Shay B Cohen. 2018. Privacy-preserving neural representations of text. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1–10. find
  17. Glen Coppersmith, Mark Dredze, Craig Harman, Kristy Hollingshead, and Margaret Mitchell. 2015. find
  18. Clpsych 2015 shared task: Depression and ptsd on twitter. In CLPsych@ HLT-NAACL, pages 31–39. find our card
  19. Sam Corbett-Davies and Sharad Goel. 2018. The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint arXiv:1808.00023. — [arXiv:1808.00023](https://arxiv.org/abs/1808.00023) find
  20. Fintan Costello and Paul Watts. 2014. Surprisingly rational: Probability theory plus noise explains biases in judgment. Psychological review, 121(3):463. find
  21. Mick P Couper. 2013. Is the sky falling? new technology, changing media, and the future of surveys. In Survey Research Methods, volume 7, pages 145– 156. find
  22. Aron Culotta. 2014. Reducing sampling bias in social media data for county health inference. In Joint Statistical Meetings Proceedings, pages 1–12. find
  23. Cristian Danescu-Niculescu-Mizil, Robert West, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. No country for old members: User lifecycle and linguistic change in online communities. In Proceedings of the 22nd international conference on World Wide Web, pages 307–318. find
  24. Hal Daume III. 2007. Frustratingly easy domain adaptation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 256–263. find our card
  25. Jacob Eisenstein, Brendan O’Connor, Noah A Smith, and Eric P Xing. 2010. A latent variable model for geographic lexical variation. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1277–1287. Association for Computational Linguistics. find
  26. Yanai Elazar and Yoav Goldberg. 2018. Adversarial removal of demographic attributes from text data. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 11–21. find our card
  27. Sorelle A Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. 2016. On the (im) possibility of fairness. corr abs/1609.07236 (2016). — [arXiv:1609.07236](https://arxiv.org/abs/1609.07236) find
  28. Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. 2018. Word embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences, 115(16):E3635–E3644. find
  29. Aparna Garimella, Carmen Banea, Dirk Hovy, and Rada Mihalcea. 2019. Women’s syntactic resilience and men’s grammatical luck: Gender-bias in part-of-speech tagging and dependency parsing. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3493– 3498, Flo… find
  30. Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, and Kate Crawford. 2018. Datasheets for datasets. In Proceedings of the 5 th Workshop on Fairness, Accountability, and Transparency in Machine Learning, Stockholm, Sweden. find
  31. Salvatore Giorgi, Veronica Lynn, Keshav Gupta, Sandra Matz, Lyle Ungar, and H.A. Schwartz. 2019. Correcting sociodemographic selection biases for population prediction. ArXiv. find
  32. Bruce Glymour and Jonathan Herington. 2019. Measuring the biases that matter: The ethical and casual foundations for measures of fairness in algorithms. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* ’19, pages 269–278, New York, NY, USA. ACM. find
  33. Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol… find our card
  34. Ron D Hays, Honghu Liu, and Arie Kapteyn. 2015. Use of internet panels to conduct surveys. Behavior research methods, 47(3):685–690. find
  35. Lisa Anne Hendricks, Kaylee Burns, Kate Saenko, Trevor Darrell, and Anna Rohrbach. 2018. Women also snowboard: Overcoming bias in captioning models. In European Conference on Computer Vision, pages 793–811. Springer. find
  36. Joseph Henrich, Steven J Heine, and Ara Norenzayan. 2010. The weirdest people in the world? Behavioral and brain sciences, 33(2-3):61–83. find
  37. Miguel A Hernán, Sonia Hernández-Díaz, and James M Robins. 2004. A structural approach to selection bias. Epidemiology, pages 615–625. find
  38. Yasmeen Hitti, Eunbee Jang, Ines Moreno, and Carolyne Pelletier. 2019. Proposed taxonomy for gender bias in text; a filtering methodology for the gender generalization subtype. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, pages 8–17, Florence, Italy. Associatio… find
  39. Dirk Hovy. 2015. Demographic factors improve classification performance. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), volume 1, pages 752–762. find our card
  40. Dirk Hovy. 2018. The social and the neural network: How to make natural language processing about people again. In Proceedings of the Second Workshop on Computational Modeling of Peo-pleâĂŹs Opinions, Personality, and Emotions in Social Media, pages 42–49. find
  41. Dirk Hovy, Taylor Berg-Kirkpatrick, Ashish Vaswani, and Eduard Hovy. 2013. Learning whom to trust with mace. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1120–1130. find our card
  42. Dirk Hovy, Federico Bianchi, and Tommaso Fornaciari. 2020. Can you translate that into man? commercial machine translation systems include stylistic biases. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Seattle, WA, USA. Association for Computational Lin… find
  43. Dirk Hovy and Anders Søgaard. 2015. Tagging performance correlates with author age. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), volume 2, pages 483–488. find
  44. Dirk Hovy and Shannon L Spruit. 2016. The social impact of natural language processing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 591–598. find our card
  45. Jing Jiang and ChengXiang Zhai. 2007. Instance weighting for domain adaptation in nlp. In Proceedings of the 45th annual meeting of the association of computational linguistics, pages 264–271. find our card
  46. Anna Jørgensen, Dirk Hovy, and Anders Søgaard. 2015. Challenges of studying and processing dialects in social media. In Proceedings of the Workshop on Noisy User-generated Text, pages 9–18. find
  47. Kenneth Joseph, Lisa Friedland, William Hobbs, Oren Tsur, and David Lazer. 2017. Constance: Modeling annotation contexts to improve stance classification. arXiv preprint arXiv:1708.06309. find
  48. David Jurgens. 2013. That’s what friends are for: Inferring location in online social media platforms based on social relationships. In Seventh International AAAI Conference on Weblogs and Social Media. find
  49. David Jurgens, Yulia Tsvetkov, and Dan Jurafsky. 2017. Incorporating dialectal variability for socially equitable language identification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 51–57. find
  50. ML Kern, G Park, JC Eichstaedt, HA Schwartz, M Sap, LK Smith, and LH Ungar. 2016. Gaining insights from social media language: Methodologies and challenges. Psychological methods, 21(4):507–525. find
  51. Svetlana Kiritchenko and Saif Mohammad. 2018. Examining gender and race bias in two hundred sentiment analysis systems. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pages 43–53. find our card
  52. Jon Kleinberg, Jens Ludwig, Sendhil Mullainathan, and Cass R Sunstein. 2018. Discrimination in the age of algorithms. Journal of Legal Analysis, 10. find
  53. Austin C Kozlowski, Matt Taddy, and James A Evans. 2018. The geometry of culture: Analyzing meaning through word embeddings. arXiv preprint arXiv:1803.09288. — [arXiv:1803.09288](https://arxiv.org/abs/1803.09288) find
  54. Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019. Measuring bias in contextualized word representations. arXiv preprint arXiv:1906.07337. find our card
  55. Anne Lauscher, Goran Glavaš, Simone Paolo Ponzetto, and Ivan Vulić. 2020. A general framework for implicit and explicit debiasing of distributional word vector spaces. find
  56. Yitong Li, Timothy Baldwin, and Trevor Cohn. 2018. Towards robust and privacy-preserving text representations. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 25–30. find our card
  57. Veronica E. Lynn, Youngseo Son, Vivek Kulkarni, Niranjan Balasubramanian, and H. Andrew Schwartz. 2017. Human centered nlp with user-factor adaptation. In Empirical Methods in Natural Language Processing. find
  58. Thomas Manzini, Yao Chong Lim, Yulia Tsvetkov, and Alan W Black. 2019. Black is to criminal as caucasian is to police: Detecting and removing multiclass bias in word embeddings. arXiv preprint arXiv:1904.04047. find our card
  59. Robert R. McCrae and Paul T. Costa Jr. 1989. Reinterpreting the Myers-Briggs type indicator from the perspective of the five-factor model of personality. Journal of Personality, 57(1):17–40. find
  60. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In Proceedings of ICLR. find
  61. Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model cards for model reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 220–229. ACM. find
  62. Ehsan Mohammady and Aron Culotta. 2014. Using county demographics to infer attributes of twitter users. In Proceedings of the joint workshop on social dynamics and personal attributes in social media, pages 7–16. find
  63. Malvina Nissim, Rik van Noord, and Rob van der Goot. 2019. Fair is better than sensational: Man is to doctor as woman is to doctor. arXiv preprint arXiv:1905.09866. find
  64. David K Park, Andrew Gelman, and Joseph Bafumi. 2006. State-level opinions from national surveys: Poststratification using multilevel logistic regression. Public opinion in state politics, pages 209–28. find
  65. Rebecca J Passonneau and Bob Carpenter. 2014. The benefits of a model of annotation. Transactions of the Association for Computational Linguistics, 2:311–326. find our card
  66. Silviu Paun, Bob Carpenter, Jon Chamberlain, Dirk Hovy, Udo Kruschwitz, and Massimo Poesio. 2018. Comparing bayesian models of annotation. Transactions of the Association for Computational Linguistics, 6:571–585. find
  67. Ellie Pavlick, Matt Post, Ann Irvine, Dmitry Kachaev, and Chris Callison-Burch. 2014. The language demographics of amazon mechanical turk. Transactions of the Association for Computational Linguistics, 2:79–92. find
  68. James W Pennebaker. 2011. The secret life of pronouns. New Scientist, 211(2828):42–45. find
  69. James W. Pennebaker and Lori D. Stone. 2003. Words of wisdom: Language use over the life span. Journal of Personality and Social Psychology, 85(2):291. find
  70. Barbara Plank, Dirk Hovy, and Anders Søgaard. 2014. Learning part-of-speech taggers with inter-annotator agreement loss. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 742–751. find our card
  71. Marcelo OR Prates, Pedro H Avelar, and Luís C Lamb. 2018. Assessing gender bias in machine translation: a case study with google translate. Neural Computing and Applications, pages 1–19. find
  72. Daniel Preoţiuc-Pietro, Johannes Eichstaedt, Gregory Park, Maarten Sap, Laura Smith, Victoria Tobolsky, H Andrew Schwartz, and Lyle Ungar. 2015. The role of personality, age, and gender in tweeting about mental illness. In Proceedings of the 2nd workshop on computational linguistics and clinical ps… find
  73. Daniel Preotiuc-Pietro, Maarten Sap, H Andrew Schwartz, and Lyle Ungar. 2015. Mental illness detection at the world well-being project for the clpsych 2015 shared task. In Proceedings of the Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, NA… find
  74. Iyad Rahwan, Manuel Cebrian, Nick Obradovich, Josh Bongard, Jean-François Bonnefon, Cynthia Breazeal, Jacob W Crandall, Nicholas A Christakis, Iain D Couzin, Matthew O Jackson, et al. 2019. Machine behaviour. Nature, 568(7753):477. find
  75. Alexey Romanov, Maria De-Arteaga, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, Anna Rumshisky, and Adam Tauman Kalai. 2019. What’s in a name? reducing bias in bios without access to protected attributes. arXiv preprint arXiv:1904.05233. find
  76. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), volume 2,… find our card
  77. Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019. The risk of racial bias in hate speech detection. In Proceedings of the 57th Conference of the Association for Computational Linguistics, pages 1668–1678, Florence, Italy. Association for Computational Linguistics. find our card
  78. H Andrew Schwartz, Johannes C Eichstaedt, Margaret L Kern, Lukasz Dziurzynski, Megha Agrawal, Gregory J Park, Shrinidhi K Lakshmikanth, Sneha Jha, Martin EP Seligman, Lyle Ungar, et al. 2013. Characterizing geographic variation in well-being using tweets. In Seventh International AAAI Conference on… find
  79. Gabriel Stanovsky, Noah A. Smith, and Luke Zettlemoyer. 2019. Evaluating gender bias in machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1679–1684, Florence, Italy. Association for Computational Linguistics. find our card
  80. Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019. Mitigating gender bias in natural language processing: Literature review. In Proceedings of the 57th Conference of the Association for Computational… find our card
  81. Harini Suresh and John V Guttag. 2019. A framework for understanding unintended consequences of machine learning. arXiv preprint arXiv:1901.10002. — [arXiv:1901.10002](https://arxiv.org/abs/1901.10002) find
  82. Chris Sweeney and Maryam Najafian. 2019. A transparent framework for evaluating unintended demographic bias in word embeddings. In Proceedings of the 57th Conference of the Association for Computational Linguistics, pages 1662–1667, Florence, Italy. Association for Computational Linguistics. find
  83. Spencer S Swinton. 1981. Predictive bias in graduate admissions tests. ETS Research Report Series, 1981(1):i–53. find
  84. Peter Trudgill. 2000. Sociolinguistics: An introduction to language and society. Penguin UK. find
  85. Amos Tversky and Daniel Kahneman. 1973. Availability: A heuristic for judging frequency and probability. Cognitive psychology, 5(2):207–232. find
  86. Svitlana Volkova, Theresa Wilson, and David Yarowsky. 2013. Exploring demographic language variations to improve multilingual sentiment analysis in social media. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1815–1827. find
  87. Kellie Webster, Marta Recasens, Vera Axelrod, and Jason Baldridge. 2018. Mind the gap: A balanced corpus of gendered ambiguous pronouns. Transactions of the Association for Computational Linguistics, 6:605–617. find our card
  88. Thomas A. Widiger and Douglas B. Samuel. 2005. Diagnostic categories or dimensions? A question for the Diagnostic and Statistical Manual of Mental Disorders—Fifth Edition. Journal of Abnormal Psychology, 114(4):494. find
  89. Diyi Yang, Robert E Kraut, Tenbroeck Smith, Elijah Mayfield, and Dan Jurafsky. 2019. Seekers, providers, welcomers, and storytellers: Modeling social roles in online health communities. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pages 1–14. find
  90. Mark Yatskar, Luke Zettlemoyer, and Ali Farhadi. 2016. Situation recognition: Visual semantic role labeling for image understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5534–5542. find
  91. Emilio Zagheni and Ingmar Weber. 2015. Demographic research with non-representative internet data. International Journal of Manpower, 36(1):13– 25. find
  92. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2979–2989. find our card
  93. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. Gender bias in coreference resolution: Evaluation and debiasing methods. arXiv preprint arXiv:1804.06876. find our card
  94. Ran Zmigrod, Sebastian J. Mielke, Hanna Wallach, and Ryan Cotterell. 2019. Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology. In Proceedings of the 57th Conference of the Association for Computational Linguistics, pages 1651–1661. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shah-2019-predictive-biases-natural-language-processing/