Elamur Research
☰
home / papers / 2020 / Language (Technology) is Power: A Critical Survey of ``Bias'' in NLP

Language (Technology) is Power: A Critical Survey of ``Bias'' in NLP

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2020.acl-main.485.pdf

Summary

This critical review looks at how NLP research handles social "bias." The authors gathered 146 papers on written-text systems published before May 2020. They started with an ACL Anthology search for "bias" and "fairness," then followed citation links and checked major ML, HCI, and web venues. Each paper's motivation and its proposed measurement or mitigation technique were coded with an extended version of the allocational versus representational harms taxonomy.

The review finds that motivations are frequently vague, inconsistent, or lack any normative argument about who is harmed and why. Techniques often fail to match the stated motivations, rarely draw on scholarship outside NLP, and cannot be compared with one another.

The authors offer three recommendations:

  • ground the work in how language relates to social hierarchies;
  • state clearly what "bias" means and the reasoning behind it;
  • center affected communities while rethinking their power relations with technologists.

A case study on African-American English (AAE) and toxicity detection shows what this looks like in practice. The paper matters because it reframes fairness work in NLP as a normative, socially situated project.

Key points

  • Corpus of 146 papers. By task, embeddings dominate (54), followed by coreference resolution (20), language modeling/dialogue (17), hate-speech detection (17), sentiment analysis (15), machine translation (8), and tagging/parsing (5).
  • Papers were sorted into six coding categories: harms of allocation, stereotypes, representational harms beyond stereotyping, correlations of doubtful validity, motivations that are vague or unstated, and surveys/frameworks. Allocational harms appear as a motivation in 30 papers but are measured by only 4 techniques.
  • 33% of papers give several motivations, 16% give vague or no motivation, and 32% show no apparent normative concern, focusing mainly on system performance.
  • Speech papers were excluded. Non-social senses of "bias," such as inductive or hypothesis-only bias, were filtered out.
  • The AAE case study links toxicity-detection disparities to raciolinguistic ideologies and the deficit perspective on AAE speakers. It argues these systems re-stigmatize those speakers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

679 papers of our library cite it (of the 140673 whose bibliographies we read); 34 of them have a page here, 645 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.acl-main.485 · arXiv 2005.14050

@inproceedings{blodgett2020language,
  title = {{Language (Technology) is Power: A Critical Survey of ``Bias'' in NLP}},
  author = {Blodgett, Su Lin and Barocas, Solon and III, Hal Daumé and Wallach, Hanna},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.485},
  eprint = {2005.14050},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2020.acl-main.485}
}

References

224 references, 34 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Artem Abzaliev. 2019. On GAP coreference resolution shared task: insights from the 3rd place solution. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 107–112, Florence, Italy. find
  2. ADA. 2018. Guidelines for Writing About People With Disabilities. ADA National Network. https://bit.ly/2KREbkB. ↗
  3. Shaowen Bardzell and Jeffrey Bardzell. 2011. Towards a Feminist HCI Methodology: Social Science, Feminism, and HCI. In Proceedings of the Conference on Human Factors in Computing Systems (CHI), pages 675–684, Vancouver, Canada. find
  4. Solon Barocas, Asia J. Biega, Benjamin Fish, J˛edrzej Niklas, and Luke Stark. 2020. When Not to Design, Build, or Deploy. In Proceedings of the Conference on Fairness, Accountability, and Transparency, Barcelona, Spain. find
  5. Oshin Agarwal, Funda Durupinar, Norman I. Badler, and Ani Nenkova. 2019. Word embeddings (also) encode human personality stereotypes. In Proceedings of the Joint Conference on Lexical and Computational Semantics, pages 205–211, Minneapolis, MN. find
  6. Solon Barocas, Kate Crawford, Aaron Shapiro, and Hanna Wallach. 2017. The Problem With Bias: Allocative Versus Representational Harms in Machine Learning. In Proceedings of SIGCIS, Philadelphia, PA. find
  7. H. Samy Alim. 2004. You Know My Steez: An Ethnographic and Sociolinguistic Study of Styleshifting in a Black American Speech Community. American Dialect Society. find
  8. Christine Basta, Marta R. Costa-jussà, and Noe Casas. 2019. Evaluating the underlying gender bias in contextualized word embeddings. In Proceedings of the Workshop on Gender Bias for Natural Language Processing, pages 33–39, Florence, Italy. find
  9. H. Samy Alim, John R. Rickford, and Arnetha F. Ball, editors. 2016. Raciolinguistics: How Language Shapes Our Ideas About Race. Oxford University Press. find
  10. Sandeep Attree. 2019. Gendered ambiguous pronouns shared task: Boosting model confdence by evidence pooling. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, Florence, Italy. find
  11. Pinkesh Badjatiya, Manish Gupta, and Vasudeva Varma. 2019. Stereotypical bias removal for hate speech detection task using knowledge-based generalizations. In Proceedings of the International World Wide Web Conference, pages 49–59, San Francisco, CA. find
  12. John Baugh. 2018. Linguistics in Pursuit of Justice. Cambridge University Press. find
  13. Emily M. Bender. 2019. A typology of ethical risks in language technology with an eye towards where transparent documentation can help. Presented at The Future of Artifcial Intelligence: Language, Ethics, Technology Workshop. https://bit.ly/ 2P9t9M6 . ↗
  14. Ruha Benjamin. 2019. Race After Technology: Aboli-tionist Tools for the New Jim Code. John Wiley & Sons. find
  15. Ruha Benjamin. 2020. 2020 Vision: Reimagining the Default Settings of Technology & Society. Keynote at ICLR. find
  16. Eugene Bagdasaryan, Omid Poursaeed, and Vitaly Shmatikov. 2019. Differential Privacy Has Disparate Impact on Model Accuracy. In Proceedings of the Conference on Neural Information Processing Systems, Vancouver, Canada. find
  17. Cynthia L. Bennett and Os Keyes. 2019. What is the Point of Fairness? Disability, AI, and The Complexity of Justice. In Proceedings of the ASSETS Workshop on AI Fairness for People with Disabilities, Pittsburgh, PA. find
  18. April Baker-Bell. 2019. Dismantling anti-black linguistic racism in English language arts classrooms: Toward an anti-racist black language pedagogy. Theory Into Practice. find
  19. Camiel J. Beukeboom and Christian Burgers. 2019. How Stereotypes Are Shared Through Language: A Review and Introduction of the Social Categories and Stereotypes Communication (SCSC) Framework. Review of Communication Research, 7:1–37. find
  20. David Bamman, Sejal Popat, and Sheng Shen. 2019. An annotated dataset of literary entities. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 2138–2144, Minneapolis, MN. find
  21. Shruti Bhargava and David Forsyth. 2019. Exposing and Correcting the Gender Bias in Image Captioning Datasets and Models. arXiv preprint arXiv:1912.00578. — [arXiv:1912.00578](https://arxiv.org/abs/1912.00578) find
  22. Xingce Bao and Qianqian Qiao. 2019. Transfer Learning from Pre-trained BERT for Pronoun Resolution. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 82–88, Florence, Italy. find
  23. Jayadev Bhaskaran and Isha Bhallamudi. 2019. Good Secretaries, Bad Truck Drivers? Occupational Gender Stereotypes in Sentiment Analysis. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 62–68, Florence, Italy. find
  24. Su Lin Blodgett, Lisa Green, and Brendan O’Connor. 2016. Demographic Dialectal Variation in Social Media: A Case Study of African-American English. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 1119–1130, Austin, TX. find our card
  25. Su Lin Blodgett and Brendan O’Connor. 2017. Racial Disparity in Natural Language Processing: A Case Study of Social Media African-American English. In Proceedings of the Workshop on Fairness, Accountability, and Transparency in Machine Learning (FAT/ML), Halifax, Canada. find
  26. Su Lin Blodgett, Johnny Wei, and Brendan O’Connor. 2018. Twitter Universal Dependency Parsing for African-American and Mainstream American English. In Proceedings of the Association for Computational Linguistics (ACL), pages 1415–1425, Melbourne, Australia. find our card
  27. Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016a. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. In Proceedings of the Conference on Neural Information Processing Systems, pages 4349–4357, Barcelona, Spain. find our card
  28. Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016b. Quantifying and reducing stereotypes in word embeddings. In Proceedings of the ICML Workshop on \#Data4Good: Machine Learning in Social Good Applications, pages 41–45, New York, NY. find
  29. Shikha Bordia and Samuel R. Bowman. 2019. Identifying and reducing gender bias in word-level language models. In Proceedings of the NAACL Student Research Workshop, pages 7–15, Minneapolis, MN. find our card
  30. Student Researchers as Linguistic Experts. guage and Linguistics Compass, 8:144–157. Lan-Kaylee Burns, Lisa Anne Hendricks, Kate Saenko, Trevor Darrell, and Anna Rohrbach. 2018. Women also Snowboard: Overcoming Bias in Captioning Models. In Procedings of the European Conference on Computer Vision (E… find
  31. Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334). find
  32. Kathryn Campbell-Kibler. 2009. The nature of sociolinguistic perception. Language Variation and Change, 21(1):135–156. find
  33. Yang Trista Cao and Hal Daumé, III. 2019. Toward gender-inclusive coreference resolution. arXiv preprint arXiv:1910.13913. find our card
  34. Rakesh Chada. 2019. Gendered pronoun resolution using bert and an extractive question answering formulation. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 126– 133, Florence, Italy. find
  35. Kaytlin Chaloner and Alfredo Maldonado. 2019. Measuring Gender Bias in Word Embedding across Domains and Discovering New Gender Bias Word Categories. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 25–32, Florence, Italy. find
  36. Anne H. Charity Hudley. 2017. Language and Racial-ization. In Ofelia García, Nelson Flores, and Massimiliano Spotti, editors, The Oxford Handbook of Language and Society. Oxford University Press. find
  37. Marc-Etienne Brunet, Colleen Alkalay-Houlihan, Ashton Anderson, and Richard Zemel. 2019. Understanding the Origins of Bias in Word Embeddings. In Proceedings of the International Conference on Machine Learning, pages 803–811, Long Beach, CA. find
  38. Won Ik Cho, Ji Won Kim, Seok Min Kim, and Nam Soo Kim. 2019. On measuring gender bias in translation of gender-neutral pronouns. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 173–181, Florence, Italy. find
  39. Mary Bucholtz, Dolores Inés Casillas, and Jin Sook Lee. 2016. Beyond Empowerment: Accompaniment and Sociolinguistic Justice in a Youth Research Program. In Robert Lawson and Dave Sayers, editors, Sociolinguistic Research: Application and Impact, pages 25–44. Routledge. find
  40. Shivang Chopra, Ramit Sawhney, Puneet Mathur, and Rajiv Ratn Shah. 2020. Hindi-English Hate Speech Detection: Author Profling, Debiasing, and Practical Perspectives. In Proceedings of the AAAI Conference on Artifcial Intelligence (AAAI), New York, NY. find
  41. Mary Bucholtz, Dolores Inés Casillas, and Jin Sook Lee. 2019. California Latinx Youth as Agents of Sociolinguistic Justice. In Netta Avineri, Laura R. Graham, Eric J. Johnson, Robin Conley Riner, and Jonathan Rosa, editors, Language and Social Justice in Practice, pages 166–175. Routledge. find
  42. Marika Cifor, Patricia Garcia, T.L. Cowan, Jasmine Rault, Tonia Sutherland, Anita Say Chan, Jennifer Rode, Anna Lauren Hoffmann, Niloufar Salehi, and Lisa Nakamura. 2019. Feminist Data Manifest-No. Retrieved from https://www.manifestno. com/ . ↗
  43. Mary Bucholtz, Audrey Lopez, Allina Mojarro, Elena Skapoulli, Chris VanderStouwe, and Shawn Warner-Garcia. 2014. Sociolinguistic Justice in the Schools: Patricia Hill Collins. 2000. Black Feminist Thought: Knowledge, Consciousness, and the Politics of Empowerment. Routledge. find
  44. Justin T. Craft, Kelly E. Wright, Rachel Elizabeth Weissler, and Robin M. Queen. 2020. Language and Discrimination: Generating Meaning, Perceiving Identities, and Discriminating Outcomes. Annual Review of Linguistics, 6(1). find
  45. Kate Crawford. 2017. The Trouble with Bias. Keynote at NeurIPS. find
  46. Kimberle Crenshaw. 1989. Demarginalizing the Intersection of Race and Sex: A Black Feminist Critique of Antidiscrmination Doctrine, Feminist Theory and Antiracist Politics. University of Chicago Legal Forum. find
  47. Amanda Cercas Curry and Verena Rieser. 2018. \#MeToo: How Conversational Systems Respond to Sexual Harassment. In Proceedings of the Workshop on Ethics in Natural Language Processing, pages 7– 14, New Orleans, LA. find
  48. Karan Dabas, Nishtha Madaan, Gautam Singh, Vijay Arya, Sameep Mehta, and Tanmoy Chakraborty. 2020. Fair Transfer of Multiple Style Attributes in Text. arXiv preprint arXiv:2001.06693. — [arXiv:2001.06693](https://arxiv.org/abs/2001.06693) find
  49. Thomas Davidson, Debasmita Bhattacharya, and Ingmar Weber. 2019. Racial bias in hate speech and abusive language detection datasets. In Proceedings of the Workshop on Abusive Language Online, pages 25–35, Florence, Italy. find our card
  50. Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. 2019. Bias in bios: A case study of semantic representation bias in a high-stakes setting. In Proceedings of the Conference on Fairnes… find
  51. Sunipa Dev, Tao Li, Jeff Phillips, and Vivek Srikumar. 2019. On Measuring and Mitigating Biased Inferences of Word Embeddings. arXiv preprint arXiv:1908.09369. — [arXiv:1908.09369](https://arxiv.org/abs/1908.09369) find
  52. Sunipa Dev and Jeff Phillips. 2019. Attenuating Bias in Word Vectors. In Proceedings of the International Conference on Artifcial Intelligence and Statistics, pages 879–887, Naha, Japan. find
  53. Mark Díaz, Isaac Johnson, Amanda Lazar, Anne Marie Piper, and Darren Gergle. 2018. Addressing age-related bias in sentiment analysis. In Proceedings of the Conference on Human Factors in Computing Systems (CHI), Montréal, Canada. find
  54. Emily Dinan, Angela Fan, Adina Williams, Jack Urbanek, Douwe Kiela, and Jason Weston. 2019. Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation. arXiv preprint arXiv:1911.03842. find our card
  55. Carl DiSalvo, Andrew Clement, and Volkmar Pipek. 2013. Communities: Participatory Design for, with and by communities. In Jesper Simonsen and Toni Robertson, editors, Routledge International Handbook of Participatory Design, pages 182–209. Routledge. find
  56. Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2018. Measuring and mitigating unintended bias in text classifcation. In Proceedings of the Conference on Artifcial Intelligence, Ethics, and Society (AIES), New Orleans, LA. find
  57. Jacob Eisenstein. 2013. What to do about bad language on the Internet. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 359–369. find
  58. Kawin Ethayarajh. 2020. Is Your Classifer Actually Biased? Measuring Fairness under Uncertainty with Bernstein Bounds. In Proceedings of the Association for Computational Linguistics (ACL). find
  59. Kawin Ethayarajh, David Duvenaud, and Graeme Hirst. 2019. Understanding Undesirable Word Embedding Assocations. In Proceedings of the Association for Computational Linguistics (ACL), pages 1696–1705, Florence, Italy. find
  60. Joseph Fisher. 2019. Measuring social bias in knowledge graph embeddings. arXiv preprint arXiv:1912.02761. — [arXiv:1912.02761](https://arxiv.org/abs/1912.02761) find
  61. Nelson Flores and Sofa Chaparro. 2018. What counts as language education policy? Developing a materi-alist Anti-racist approach to language activism. Language Policy, 17(3):365–384. find
  62. Omar U. Florez. 2019. On the Unintended Social Bias of Training Language Generation Models with Data from Local Media. In Proceedings of the NeurIPS Workshop on Human-Centric Machine Learning, Vancouver, Canada. find
  63. Joel Escudé Font and Marta R. Costa-jussà. 2019. Equalizing gender biases in neural machine translation with word embeddings techniques. In Proceedings of the Workshop on Gender Bias for Natural Language Processing, pages 147–154, Florence, Italy. find
  64. Batya Friedman and David G. Hendry. 2019. Value Sensitive Design: Shaping Technology with Moral Imagination. MIT Press. find
  65. Batya Friedman, Peter H. Kahn Jr., and Alan Borning. 2006. Value Sensitive Design and Information Systems. In Dennis Galletta and Ping Zhang, editors, Human-Computer Interaction in Management Information Systems: Foundations, pages 348–372. M.E. Sharpe. find
  66. Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. 2018. Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes. Proceedings of the National Academy of Sciences, 115(16). find
  67. Sahaj Garg, Vincent Perot, Nicole Limtiaco, Ankur Taly, Ed H. Chi, and Alex Beutel. 2019. Counterfactual fairness in text classifcation through robustness. In Proceedings of the Conference on Artifcial Intelligence, Ethics, and Society (AIES), Honolulu, HI. find
  68. Aparna Garimella, Carmen Banea, Dirk Hovy, and Rada Mihalcea. 2019. Women’s syntactic resilience and men’s grammatical luck: Gender bias in part-of-speech tagging and dependency parsing data. In Proceedings of the Association for Computational Linguistics (ACL), pages 3493–3498, Florence, Italy. find
  69. Andrew Gaut, Tony Sun, Shirlyn Tang, Yuxin Huang, Jing Qian, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2020. Towards Understanding Gender Bias in Relation Extraction. In Proceedings of the Association for Computational Linguistics (ACL). find
  70. R. Stuart Geiger, Kevin Yu, Yanlai Yang, Mindy Dai, Jie Qiu, Rebekah Tang, and Jenny Huang. 2020. Garbage In, Garbage Out? Do Machine Learning Application Papers in Social Computing Report Where Human-Labeled Training Data Comes From? In Proceedings of the Conference on Fairness, Accountability, and… find
  71. Oguzhan Gencoglu. 2020. Cyberbullying Detection with Fairness Constraints. arXiv preprint arXiv:2005.06625. — [arXiv:2005.06625](https://arxiv.org/abs/2005.06625) find
  72. Alexandra Reeve Givens and Meredith Ringel Morris. 2020. Centering Disability Perspecives in Algorithmic Fairness, Accountability, and Transparency. In Proceedings of the Conference on Fairness, Accountability, and Transparency, Barcelona, Spain. find
  73. Hila Gonen and Yoav Goldberg. 2019. Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 609–614, Minneapolis, MN. find our card
  74. Hila Gonen and Kellie Webster. 2020. Automatically Identifying Gender Issues in Machine Translation using Perturbations. arXiv preprint arXiv:2004.14065. find
  75. Ben Green. 2019. “Good” isn’t good enough. In Proceedings of the AI for Social Good Workshop, Vancouver, Canada. find
  76. Enoch Opanin Gyamf, Yunbo Rao, Miao Gou, and Yanhua Shao. 2020. deb2viz: Debiasing gender in word embedding data using subspace visualization. In Proceedings of the International Conference on Graphics and Image Processing. find
  77. Foad Hamidi, Morgan Klaus Scheuerman, and Stacy M. Branham. 2018. Gender Recognition or Gender Reductionism? The Social Implications of Automatic Gender Recognition Systems. In Proceedings of the Conference on Human Factors in Computing Systems (CHI), Montréal, Canada. find
  78. Alex Hanna, Emily Denton, Andrew Smart, and Jamila Smith-Loud. 2020. Towards a Critical Race Methodology in Algorithmic Fairness. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 501–512, Barcelona, Spain. find
  79. Madeline E. Heilman, Aaaron S. Wallen, Daniella Fuchs, and Melinda M. Tamkins. 2004. Penalties for Success: Reactions to Women Who Succeed at Male Gender-Typed Tasks. Journal of Applied Psychology, 89(3):416–427. find
  80. Jane H. Hill. 2008. The Everyday Language of White Racism. Wiley-Blackwell. find
  81. Dirk Hovy, Federico Bianchi, and Tommaso Fornaciari. 2020. Can You Translate that into Man? Commercial Machine Translation Systems Include Stylistic Biases. In Proceedings of the Association for Computational Linguistics (ACL). find
  82. Dirk Hovy and Anders Søgaard. 2015. Tagging Performance Correlates with Author Age. In Proceedings of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, pages 483–488, Beijing, China. find
  83. Dirk Hovy and Shannon L. Spruit. 2016. The social impact of natural language processing. In Proceedings of the Association for Computational Linguistics (ACL), pages 591–598, Berlin, Germany. find our card
  84. Po-Sen Huang, Huan Zhang, Ray Jiang, Robert Stanforth, Johannes Welbl, Jack W. Rae, Vishal Maini, Dani Yogatama, and Pushmeet Kohli. 2019. Reducing Sentiment Bias in Language Models via Counterfactual Evaluation. arXiv preprint arXiv:1911.03064. find
  85. Lisa J. Green. 2002. African American English: A Linguistic Introduction. Cambridge University Press. find
  86. Xiaolei Huang, Linzi Xing, Franck Dernoncourt, and Michael J. Paul. 2020. Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition. In Proceedings of the Language Resources and Evaluation Conference (LREC), Marseille, France. find
  87. Anthony G. Greenwald, Debbie E. McGhee, and Jordan L.K. Schwartz. 1998. Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74(6):1464–1480. find
  88. Christoph Hube, Maximilian Idahl, and Besnik Fetahu. 2020. Debiasing Word Embeddings from Sentiment Associations in Names. In Proceedings of the International Conference on Web Search and Data Mining, pages 259–267, Houston, TX. find
  89. Ben Hutchinson, Vinodkumar Prabhakaran, Emily Denton, Kellie Webster, Yu Zhong, and Stephen Denuyl. 2020. Social Biases in NLP Models as Barriers for Persons with Disabilities. In Proceedings of the Association for Computational Linguistics (ACL). find our card
  90. Masahiro Kaneko and Danushka Bollegala. 2019. Gender-preserving debiasing for pre-trained word embeddings. In Proceedings of the Association for Computational Linguistics (ACL), pages 1641–1650, Florence, Italy. find
  91. Matei Ionita, Yury Kashnitsky, Ken Krige, Vladimir Larin, Dennis Logvinenko, and Atanas Atanasov. 2019. Resolving gendered ambiguous pronouns with BERT. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 113–119, Florence, Italy. find
  92. Saket Karve, Lyle Ungar, and João Sedoc. 2019. Conceptor debiasing of word representations evaluated on WEAT. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 40– 48, Florence, Italy. find
  93. Hailey James-Sorenson and David Alvarez-Melis. 2019. Probabilistic Bias Mitigation in Word Embeddings. In Proceedings of the Workshop on Human-Centric Machine Learning, Vancouver, Canada. find
  94. Shengyu Jia, Tao Meng, Jieyu Zhao, and Kai-Wei Chang. 2020. Mitigating Gender Bias Amplifcation in Distribution by Posterior Regularization. In Proceedings of the Association for Computational Linguistics (ACL). find
  95. Michael Katell, Meg Young, Dharma Dailey, Bernease Herman, Vivian Guetler, Aaron Tam, Corinne Bintz, Danielle Raz, and P.M. Krafft. 2020. Toward situated interventions for algorithmic equity: lessons from the feld. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages… find
  96. Stephen Kemmis. 2006. Participatory action research and the public sphere. Educational Action Research, 14(4):459–476. find
  97. Taylor Jones, Jessica Rose Kalbfeld, Ryan Hancock, and Robin Clark. 2019. Testifying while black: An experimental study of court reporter accuracy in transcription of African American English. Language, 95(2). find
  98. Os Keyes. 2018. The Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW). find
  99. Anna Jørgensen, Dirk Hovy, and Anders Søgaard. 2015. Challenges of studying and processing dialects in social media. In Proceedings of the Workshop on Noisy User-Generated Text, pages 9–18, Beijing, China. find
  100. Os Keyes, Josephine Hoy, and Margaret Drouhard. 2019. Human-Computer Insurrection: Notes on an Anarchist HCI. In Proceedings of the Conference on Human Factors in Computing Systems (CHI), Glasgow, Scotland, UK. find
  101. Anna Jørgensen, Dirk Hovy, and Anders Søgaard. 2016. Learning a POS tagger for AAVE-like language. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 1115– 1120, San Diego, CA. find our card
  102. Jae Yeon Kim, Carlos Ortiz, Sarah Nam, Sarah Santiago, and Vivek Datta. 2020. Intersectional Bias in Hate Speech and Abusive Language Datasets. In Proceedings of the Association for Computational Linguistics (ACL). find
  103. Pratik Joshi, Sebastian Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The State and Fate of Linguistic Diversity and Inclusion in the NLP World. In Proceedings of the Association for Computational Linguistics (ACL). find our card
  104. Svetlana Kiritchenko and Saif M. Mohammad. 2018. Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems. In Proceedings of the Joint Conference on Lexical and Computational Semantics, pages 43–53, New Orleans, LA. find our card
  105. Jaap Jumelet, Willem Zuidema, and Dieuwke Hupkes. 2019. Analysing Neural Language Models: Contextual Decomposition Reveals Default Reasoning in Number and Gender Assignment. In Proceedings of the Conference on Natural Language Learning, Hong Kong, China. find
  106. Moshe Koppel, Shlomo Argamon, and Anat Rachel Shimoni. 2002. Automatically Categorizing Written Texts by Author Gender. Literary and Linguistic Computing, 17(4):401–412. find
  107. Marie-Odile Junker. 2018. Participatory action research for Indigenous linguistics in the digital age. In Shannon T. Bischoff and Carmen Jany, editors, Insights from Practices in Community-Based Research, pages 164–175. De Gruyter Mouton. find
  108. Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W. Black, and Yulia Tsvetkov. 2019. Measuring bias in contextualized word representations. In Proceedings of the Workshop on Gender Bias for Natural Language Processing, pages 166–172, Florence, Italy. find our card
  109. David Jurgens, Yulia Tsvetkov, and Dan Jurafsky. 2017. Incorporating Dialectal Variability for Socially Equitable Language Identifcation. In Proceedings of the Association for Computational Linguistics (ACL), pages 51–57, Vancouver, Canada. find
  110. Sonja L. Lanehart and Ayesha M. Malik. 2018. Black Is, Black Isn’t: Perceptions of Language and Black-ness. In Jeffrey Reaser, Eric Wilbanks, Karissa Wojcik, and Walt Wolfram, editors, Language Variety in the New South. University of North Carolina Press. find
  111. Brian N. Larson. 2017. Gender as a variable in natural-language processing: Ethical considerations. In Proceedings of the Workshop on Ethics in Natural Language Processing, pages 30–40, Valencia, Spain. find
  112. Anne Lauscher and Goran Glavaš. 2019. Are We Consistently Biased? Multidimensional Analysis of Biases in Distributional Word Vectors. In Proceedings of the Joint Conference on Lexical and Computational Semantics, pages 85–91, Minneapolis, MN. find
  113. Anastassia Loukina, Nitin Madnani, and Klaus Zechner. 2019. The many dimensions of algorithmic fairness in educational applications. In Proceedings of the Workshop on Innovative Use of NLP for Building Educational Applications, pages 1–10, Florence, Italy. find
  114. Kaiji Lu, Peter Mardziel, Fangjing Wu, Preetam Aman-charla, and Anupam Datta. 2018. Gender bias in neural natural language processing. arXiv preprint arXiv:1807.11714. find our card
  115. Anne Lauscher, Goran Glavaš, Simone Paolo Ponzetto, and Ivan Vulić. 2019. A General Framework for Implicit and Explicit Debiasing of Distributional Word Vector Spaces. arXiv preprint arXiv:1909.06092. — [arXiv:1909.06092](https://arxiv.org/abs/1909.06092) find
  116. Anne Maass. 1999. Linguistic intergroup bias: Stereotype perpetuation through language. Advances in Experimental Social Psychology, 31:79–121. find
  117. Christopher A. Le Dantec, Erika Shehan Poole, and Susan P. Wyche. 2009. Values as Lived Experience: Evolving Value Sensitive Design in Support of Value Discovery. In Proceedings of the Conference on Human Factors in Computing Systems (CHI), Boston, MA. find
  118. Nitin Madnani, Anastassia Loukina, Alina von Davier, Jill Burstein, and Aoife Cahill. 2017. Building Better Open-Source Tools to Support Fairness in Automated Scoring. In Proceedings of the Workshop on Ethics in Natural Language Processing, pages 41– 52, Valencia, Spain. find
  119. Nayeon Lee, Andrea Madotto, and Pascale Fung. 2019. Exploring Social Bias in Chatbots using Stereotype Knowledge. In Proceedings of the Workshop on Widening NLP, pages 177–180, Florence, Italy. find
  120. Thomas Manzini, Yao Chong Lim, Yulia Tsvetkov, and Alan W. Black. 2019. Black is to Criminal as Caucasian is to Police: Detecting and Removing Multi-class Bias in Word Embeddings. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 801–809, Minneapolis, MN. find
  121. Wesley Y. Leonard. 2012. Reframing language reclamation programmes for everybody’s empowerment. Gender and Language, 6(2):339–367. find
  122. Paul Pu Liang, Irene Li, Emily Zheng, Yao Chong Lim, Ruslan Salakhutdinov, and Louis-Philippe Morency. 2019. Towards Debiasing Sentence Representations. In Proceedings of the NeurIPS Workshop on Human-Centric Machine Learning, Vancouver, Canada. find our card
  123. Rosina Lippi-Green. 2012. English with an Accent: Language, Ideology, and Discrimination in the United States. Routledge. find
  124. Bo Liu. 2019. Anonymized BERT: An Augmentation Approach to the Gendered Pronoun Resolution Challenge. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 120– 125, Florence, Italy. find
  125. Haochen Liu, Jamell Dacon, Wenqi Fan, Hui Liu, Zitao Liu, and Jiliang Tang. 2019. Does Gender Matter? Towards Fairness in Dialogue Systems. arXiv preprint arXiv:1910.10486. find
  126. Felipe Alfaro Lois, José A.R. Fonollosa, and Costa-jà. 2019. BERT Masked Language Modeling for Coreference Resolution. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 76–81, Florence, Italy. find
  127. Brandon C. Loudermilk. 2015. Implicit attitudes and the perception of sociolinguistic variation. In Alexei Prikhodkine and Dennis R. Preston, editors, Responses to Language Varieties: Variability, processes and outcomes, pages 137–156. find
  128. Ramón Antonio Martínez and Alexander Feliciano Mejía. 2019. Looking closely and listening carefully: A sociocultural approach to understanding the complexity of Latina/o/x students’ everyday language. Theory Into Practice. find
  129. Rowan Hall Maudslay, Hila Gonen, Ryan Cotterell, and Simone Teufel. 2019. It’s All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitution. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 5270–5278, Hong Kong, China. find
  130. Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019. On Measuring Social Biases in Sentence Encoders. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 629–634, Minneapolis, MN. find our card
  131. Elijah Mayfeld, Michael Madaio, Shrimai Prabhumoye, David Gerritsen, Brittany McLaughlin, Ezekiel Dixon-Roman, and Alan W. Black. 2019. Equity Beyond Bias in Language Technologies for Education. In Proceedings of the Workshop on Innovative Use of NLP for Building Educational Applications, Florence,… find
  132. Katherine McCurdy and Oğuz Serbetçi. 2017. Grammatical gender associations outweigh topical gender bias in crosslinguistic word embeddings. In Proceedings of the Workshop for Women & Underrepresented Minorities in Natural Language Processing, Vancouver, Canada. find
  133. Ninareh Mehrabi, Thamme Gowda, Fred Morstatter, Nanyun Peng, and Aram Galstyan. 2019. Man is to Person as Woman is to Location: Measuring Gender Bias in Named Entity Recognition. arXiv preprint arXiv:1910.10872. — [arXiv:1910.10872](https://arxiv.org/abs/1910.10872) find
  134. Michela Menegatti and Monica Rubini. 2017. Gender bias and sexism in language. In Oxford Research Encyclopedia of Communication. Oxford University Press. find
  135. Inom Mirzaev, Anthony Schulte, Michael Conover, and Sam Shah. 2019. Considerations for the interpretation of bias measures of word embeddings. arXiv preprint arXiv:1906.08379. — [arXiv:1906.08379](https://arxiv.org/abs/1906.08379) find
  136. Salikoko S. Mufwene, Guy Bailey, and John R. Rickford, editors. 1998. African-American English: Structure, History, and Use. Routledge. find
  137. Michael J. Muller. 2007. Participatory Design: The Third Space in HCI. In The Human-Computer Interaction Handbook, pages 1087–1108. CRC Press. find
  138. Moin Nadeem, Anna Bethke, and Siva Reddy. 2020. StereoSet: Measuring stereotypical bias in pretrained language models. arXiv preprint arXiv:2004.09456. find our card
  139. Dong Nguyen, Rilana Gravel, Dolf Trieschnigg, and Theo Meder. 2013. “How Old Do You Think I Am?”: A Study of Language and Age in Twitter. In Proceedings of the Conference on Web and Social Media (ICWSM), pages 439–448, Boston, MA. find
  140. Ellie Pavlick and Tom Kwiatkowski. 2019. Inherent Disagreements in Human Textual Inferences. Transactions of the Association for Computational Linguistics, 7:677–694. find our card
  141. Xiangyu Peng, Siyan Li, Spencer Frazier, and Mark Riedl. 2020. Fine-Tuning a Transformer-Based Language Model to Avoid Generating Non-Normative Text. arXiv preprint arXiv:2001.08764. — [arXiv:2001.08764](https://arxiv.org/abs/2001.08764) find
  142. Florian Lemmerich, and Markus Radomir Popovic, Strohmaier. 2020. Joint Multiclass Debiasing of Word Embeddings. In Proceedings of the International Symposium on Intelligent Systems, Graz, Austria. find
  143. Vinodkumar Prabhakaran, Ben Hutchinson, and Margaret Mitchell. 2019. Perturbation Sensitivity Analysis to Detect Unintended Model Biases. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 5744–5749, Hong Kong, China. find
  144. Shrimai Prabhumoye, Elijah Mayfeld, and Alan W. Black. 2019. Principled Frameworks for Evaluating Ethics in NLP Systems. In Proceedings of the Workshop on Innovative Use of NLP for Building Educational Applications, Florence, Italy. find
  145. Marcelo Prates, Pedro Avelar, and Luis C. Lamb. 2019. Assessing gender bias in machine translation: A case study with google translate. Neural Computing and Applications. find
  146. Malvina Nissim, Rik van Noord, and Rob van der Goot. 2020. Fair is better than sensational: Man is to doctor as woman is to doctor. Computational Linguistics. find
  147. Rasmus Précenth. 2019. Word embeddings and gender stereotypes in Swedish and English. Master’s thesis, Uppsala University. find
  148. Debora Nozza, Claudia Volpetti, and Elisabetta Fersini. 2019. Unintended Bias in Misogyny Detection. In Proceedings of the Conference on Web Intelligence, pages 149–155. find
  149. Dennis R. Preston. 2009. Are you really smart (or stupid, or cute, or ugly, or cool)? Or do you just talk that way? Language attitudes, standardization and language change. Oslo: Novus forlag, pages 105– 129. find
  150. Alexandra Olteanu, Carlos Castillo, Fernando Diaz, and Emre Kıcıman. 2019. Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries. Frontiers in Big Data, 2. find
  151. Alexandra Olteanu, Kartik Talamadupula, and Kush R. Varshney. 2017. The Limits of Abstract Evaluation Metrics: The Case of Hate Speech Detection. In Proceedings of the ACM Web Science Conference, Troy, NY. find
  152. Orestis Papakyriakopoulos, Simon Hegelich, Juan Carlos Medina Serrano, and Fabienne Marco. 2020. Bias in word embeddings. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 446–457, Barcelona, Spain. find
  153. Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. Reducing Gender Bias in Abusive Language Detection. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 2799–2804, Brussels, Belgium. find our card
  154. Flavien Prost, Nithum Thain, and Tolga Bolukbasi. 2019. Debiasing Embeddings for Reduced Gender Bias in Text Classifcation. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 69–75, Florence, Italy. find
  155. Reid Pryzant, Richard Diehl Martinez, Nathan Dass, Sadao Kurohashi, Dan Jurafsky, and Diyi Yang. 2020. Automatically Neutralizing Subjective Bias in Text. In Proceedings of the AAAI Conference on Artifcial Intelligence (AAAI), New York, NY. find
  156. Arun K. Pujari, Ansh Mittal, Anshuman Padhi, Anshul Jain, Mukesh Jadon, and Vikas Kumar. 2019. Debiasing Gender biased Hindi Words with Word-embedding. In Proceedings of the International Conference on Algorithms, Computing and Artifcial Intelligence, pages 450–456. find
  157. Yusu Qian, Urwa Muaz, Ben Zhang, and Jae Won Hyun. 2019. Reducing gender bias in word-level language models with a gender-equalizing loss function. In Proceedings of the ACL Student Research Workshop, pages 223–228, Florence, Italy. find
  158. Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. 2020. Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection. In Proceedings of the Association for Computational Linguistics (ACL). find our card
  159. John R. Rickford and Sharese King. 2016. Language and linguistics on trial: Hearing Rachel Jeantel (and other vernacular speakers) in the courtroom and beyond. Language, 92(4):948–988. find
  160. Anthony Rios. 2020. FuzzE: Fuzzy Fairness Evaluation of Offensive Language Classifers on African-American English. In Proceedings of the AAAI Conference on Artifcial Intelligence (AAAI), New York, NY. find
  161. Gerald Roche. 2019. Articulating language oppression: colonialism, coloniality and the erasure of Ti-minority languages. Patterns of Prejudice. betâĂ Zs Alexey Romanov, Maria De-Arteaga, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, Ann… find
  162. Jonathan Rosa. 2019. Contesting Representations of Migrant “Illegality” through the Drop the I-Word Campaign: Rethinking Language Change and Social Change. In Netta Avineri, Laura R. Graham, Eric J. Johnson, Robin Conley Riner, and Jonathan Rosa, editors, Language and Social Justice in Practice. Rou… find
  163. Jonathan Rosa and Christa Burdick. 2017. Language Ideologies. In Ofelia García, Nelson Flores, and Massimiliano Spotti, editors, The Oxford Handbook of Language and Society. Oxford University Press. find
  164. Jonathan Rosa and Nelson Flores. 2017. Unsettling race and language: Toward a raciolinguistic perspective. Language in Society, 46:621–647. find
  165. Sara Rosenthal and Kathleen McKeown. 2011. Age Prediction in Blogs: A Study of Style, Content, and Online Behavior in Pre- and Post-Social Media Generations. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 763–772, Portland, OR. find
  166. Candace Ross, Boris Katz, and Andrei Barbu. 2020. Measuring Social Biases in Grounded Vision and Language Embeddings. arXiv preprint arXiv:2002.08911. find
  167. Richard Rothstein. 2017. The Color of Law: A Forgotten History of How Our Government Segregated America. Liveright Publishing. find
  168. David Rozado. 2020. Wide range screening of algorithmic bias in word embedding models using large sentiment lexicons reveals underreported bias types. PLOS One. find
  169. Elayne Ruane, Abeba Birhane, and Anthony Ven-tresque. 2019. Conversational AI: Social and Ethical Considerations. In Proceedings of the Irish Conference on Artifcial Intelligence and Cognitive Science, Galway, Ireland. find
  170. Rachel Rudinger, Chandler May, and Benjamin Van Durme. 2017. Social bias in elicited natural language inferences. In Proceedings of the Workshop on Ethics in Natural Language Processing, pages 74–79, Valencia, Spain. find our card
  171. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender Bias in Coreference Resolution. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 8–14, New Orleans, LA. find our card
  172. Elizabeth B.N. Sanders. 2002. From user-centered to participatory design approaches. In Jorge Frascara, editor, Design and the Social Sciences: Making Connections, pages 18–25. CRC Press. find
  173. Brenda Salenave Santana, Vinicius Woloszyn, and Leandro Krug Wives. 2018. Is there gender bias and stereotype in Portuguese word embeddings? In Proceedings of the International Conference on the Computational Processing of Portuguese Student Research Workshop, Canela, Brazil. find
  174. Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019. The risk of racial bias in hate speech detection. In Proceedings of the Association for Computational Linguistics (ACL), pages 1668–1678, Florence, Italy. find our card
  175. Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A. Smith, and Yejin Choi. 2020. Social Bias Frames: Reasoning about Social and Power Implications of Language. In Proceedings of the Association for Computational Linguistics (ACL). find our card
  176. Hanna Sassaman, Jennifer Lee, Jenessa Irvine, and Shankar Narayan. 2020. Creating Community-Based Tech Policy: Case Studies, Lessons Learned, and What Technologists and Communities Can Do Together. In Proceedings of the Conference on Fairness, Accountability, and Transparency, Barcelona, Spain. find
  177. Danielle Saunders and Bill Byrne. 2020. Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem. In Proceedings of the Association for Computational Linguistics (ACL). find
  178. Tyler Schnoebelen. 2017. Goal-Oriented Design for Ethical Machine Learning and NLP. In Proceedings of the Workshop on Ethics in Natural Language Processing, pages 88–93, Valencia, Spain. find
  179. Sabine Sczesny, Magda Formanowicz, and Franziska Moser. 2016. Can gender-fair language reduce gender stereotyping and discrimination? Frontiers in Psychology, 7. find
  180. João Sedoc and Lyle Ungar. 2019. The Role of Protected Class Word Lists in Bias Identifcation of Contextualized Word Representations. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 55–61, Florence, Italy. find
  181. Procheta Sen and Debasis Ganguly. 2020. Towards Socially Responsible AI: Cognitive Bias-Aware Multi-Objective Learning. In Proceedings of the AAAI Conference on Artifcial Intelligence (AAAI), New York, NY. find
  182. Deven Shah, H. Andrew Schwartz, and Dirk Hovy. 2020. Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview. In Proceedings of the Association for Computational Linguistics (ACL). find
  183. Judy Hanwen Shen, Lauren Fratamico, Iyad Rahwan, and Alexander M. Rush. 2018. Darling or Baby-girl? Investigating Stylistic Bias in Sentiment Analysis. In Proceedings of the Workshop on Fairness, Accountability, and Transparency (FAT/ML), Stockholm, Sweden. find
  184. Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2019. The Woman Worked as a Babysitter: On Biases in Language Generation. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 3398–3403, Hong Kong, China. find our card
  185. Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2020. Towards Controllable Biases in Language Generation. arXiv preprint arXiv:2005.00268. find
  186. Seungjae Shin, Kyungwoo Song, JoonHo Jang, Hyemi Kim, Weonyoung Joo, and Il-Chul Moon. 2020. Neutralizing Gender Bias in Word Embedding with Latent Disentanglement and Counterfactual Generation. arXiv preprint arXiv:2004.03133. — [arXiv:2004.03133](https://arxiv.org/abs/2004.03133) find
  187. Jesper Simonsen and Toni Robertson, editors. 2013. Routledge International Handbook of Participatory Design. Routledge. find
  188. Gabriel Stanovsky, Noah A. Smith, and Luke Zettlemoyer. 2019. Evaluating gender bias in machine translation. In Proceedings of the Association for Computational Linguistics (ACL), pages 1679–1684, Florence, Italy. find our card
  189. Yolande Strengers, Lizhe Qu, Qiongkai Xu, and Jarrod Knibbe. 2020. Adhering, Steering, and Queering: Treatment of Gender in Natural Language Generation. In Proceedings of the Conference on Human Factors in Computing Systems (CHI), Honolulu, HI. find
  190. Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019. Mitigating Gender Bias in Natural Language Processing: Literature Review. In Proceedings of the Association for Computational Linguistics (ACL), pag… find our card
  191. Adam Sutton, Thomas Lansdall-Welfare, and Nello Cristianini. 2018. Biased embeddings from wild data: Measuring, understanding and removing. In Proceedings of the International Symposium on Intelligent Data Analysis, pages 328–339, ’s-Hertogenbosch, Netherlands. find
  192. Chris Sweeney and Maryam Najafan. 2019. A Transparent Framework for Evaluating Unintended Demographic Bias in Word Embeddings. In Proceedings of the Association for Computational Linguistics (ACL), pages 1662–1667, Florence, Italy. find
  193. Chris Sweeney and Maryam Najafan. 2020. Reducing sentiment polarity for demographic attributes in word embeddings using adversarial learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 359–368, Barcelona, Spain. find
  194. Nathaniel Swinger, Maria De-Arteaga, Neil Thomas Heffernan, Mark D.M. Leiserson, and Adam Tauman Kalai. 2019. What are the biases in my word embedding? In Proceedings of the Conference on Artifcial Intelligence, Ethics, and Society (AIES), Honolulu, HI. find
  195. Samson Tan, Shafq Joty, Min-Yen Kan, and Richard Socher. 2020. It’s Morphin’ Time! Combating Linguistic Discrimination with Infectional Perturbations. In Proceedings of the Association for Computational Linguistics (ACL). find
  196. Yi Chern Tan and L. Elisa Celis. 2019. Assessing Social and Intersectional Biases in Contextualized Word Representations. In Proceedings of the Conference on Neural Information Processing Systems, Vancouver, Canada. find
  197. J. Michael Terry, Randall Hendrick, Evangelos Evan-gelou, and Richard L. Smith. 2010. Variable dialect switching among African American children: Inferences about working memory. Lingua, 120(10):2463–2475. find
  198. Joel Tetreault, Daniel Blanchard, and Aoife Cahill. 2013. A Report on the First Native Language Identifcation Shared Task. In Proceedings of the Workshop on Innovative Use of NLP for Building Educational Applications, pages 48–57, Atlanta, GA. find
  199. Mike Thelwall. 2018. Gender Bias in Sentiment Analysis. Online Information Review, 42(1):45–57. find
  200. Kristen Vaccaro, Karrie Karahalios, Deirdre K. Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Con-testability in Algorithmic Systems. In Conference Companion Publication of the 2019 on Computer Supported Cooperative Work and Social Computing, pages 523–527, Austin, TX. find
  201. Ameya Vaidya, Feng Mai, and Yue Ning. 2019. Empirical Analysis of Multi-Task Learning for Reducing Model Bias in Toxic Comment Detection. arXiv preprint arXiv:1909.09758v2. — [arXiv:1909.09758](https://arxiv.org/abs/1909.09758) find
  202. Eva Vanmassenhove, Christian Hardmeier, and Andy Way. 2018. Getting Gender Right in Neural Machine Translation. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 3003–3008, Brussels, Belgium. find our card
  203. Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020. Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias. arXiv preprint arXiv:2004.12265. — [arXiv:2004.12265](https://arxiv.org/abs/2004.12265) find
  204. Tianlu Wang, Xi Victoria Lin, Nazneen Fatema Rajani, Bryan McCann, Vicente Ordonez, and Caiming Xiong. 2020. Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigation. In Proceedings of the Association for Computational Linguistics (ACL). find
  205. Zili Wang. 2019. MSnet: A BERT-based Network for Gendered Pronoun Resolution. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 89–95, Florence, Italy. find
  206. Kellie Webster, Marta R. Costa-jussà, Christian Hardmeier, and Will Radford. 2019. Gendered Ambiguous Pronoun (GAP) Shared Task at the Gender Bias in NLP Workshop 2019. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 1–7, Florence, Italy. find
  207. Kellie Webster, Marta Recasens, Vera Axelrod, and Jason Baldridge. 2018. Mind the GAP: A balanced corpus of gendered ambiguous pronouns. Transactions of the Association for Computational Linguistics, 6:605–618. find our card
  208. Kai-Chou Yang, Timothy Niven, Tzu-Hsuan Chou, and Hung-Yu Kao. 2019. Fill the GAP: Exploiting BERT for Pronoun Resolution. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 102–106, Florence, Italy. find
  209. Zekun Yang and Juan Feng. 2020. A Causal Inference Method for Reducing Gender Bias in Word Embedding Relations. In Proceedings of the AAAI Conference on Artifcial Intelligence (AAAI), New York, NY. find
  210. Daisy Yoo, Anya Ernest, Sofa Serholt, Eva Eriksson, and Peter Dalsgaard. 2019. Service Design in HCI Research: The Extended Value Co-creation Model. In Proceedings of the Halfway to the Future Symposium, Nottingham, United Kingdom. find
  211. Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018. Mitigating unwanted biases with adversarial learning. In Proceedings of the Conference on Artifcial Intelligence, Ethics, and Society (AIES), New Orleans, LA. find
  212. Guanhua Zhang, Bing Bai, Junqi Zhang, Kun Bai, Conghui Zhu, and Tiejun Zhao. 2020a. Demographics Should Not Be the Reason of Toxicity: Mitigating Discrimination in Text Classifcations with Instance Weighting. In Proceedings of the Association for Computational Linguistics (ACL). find
  213. Haoran Zhang, Amy X. Lu, Mohamed Abdalla, Matthew McDermott, and Marzyeh Ghassemi. 2020b. Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings. In Proceedings of the ACM Conference on Health, Inference, and Learning. find
  214. Jieyu Zhao, Subhabrata Mukherjee, Saghar Hosseini, Kai-Wei Chang, and Ahmed Hassan Awadallah. 2020. Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer. In Proceedings of the Association for Computational Linguistics (ACL). find
  215. Walt Wolfram and Natalie Schilling. 2015. American English: Dialects and Variation, 3 edition. Wiley Blackwell. find
  216. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender Bias in Contextualized Word Embeddings. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 629– 634, Minneapolis, MN. find our card
  217. Austin P. Wright, Omar Shaikh, Haekyu Park, Will Epperson, Muhammed Ahmed, Stephane Pinel, Diyi Yang, and Duen Horng (Polo) Chau. 2020. RE-CAST: Interactive Auditing of Automatic Toxicity Detection Models. In Proceedings of the Conference on Human Factors in Computing Systems (CHI), Honolulu, HI. find
  218. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplifcation using corpus-level constraints. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 2979–2989, Copenhagen, Denmark. find
  219. Yinchuan Xu and Junlin Yang. 2019. Look again at the syntax: Relational graph convolutional network for gendered ambiguous pronoun resolution. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 96–101, Florence, Italy. find
  220. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018a. Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 15–20, New Orleans, LA. find our card
  221. Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018b. Learning Gender-Neutral Word Embeddings. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 4847–4853, Brussels, Belgium. find our card
  222. Alina Zhiltsova, Simon Caton, and Catherine Mulwa. 2019. Mitigation of Unintended Biases against Non-Native English Texts in Sentiment Analysis. In Proceedings of the Irish Conference on Artifcial Intelligence and Cognitive Science, Galway, Ireland. find
  223. Pei Zhou, Weijia Shi, Jieyu Zhao, Kuan-Hao Huang, Muhao Chen, and Kai-Wei Chang. 2019. Examining gender bias in languages with grammatical genders. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 5279– 5287, Hong Kong, China. find
  224. Ran Zmigrod, S. J. Mielke, Hanna Wallach, and Ryan Cotterell. 2019. Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology. In Proceedings of the Association for Computational Linguistics (ACL), pages 1651– 1661, Florence, Italy. find our card