Elamur Research⌕
☰
home / papers / 2022 / Uncertainty and Inclusivity in Gender Bias Annotation: An Annotation Taxonomy and Annotated Datasets of British English Text

Uncertainty and Inclusivity in Gender Bias Annotation: An Annotation Taxonomy and Annotated Datasets of British English Text

Paper · 2022 · Lucy Havens, Melissa Terras, Benjamin Bach et al.

Summary

Work on gender bias in NLP often defines gender and bias loosely, assumes only two genders, and treats annotations as having one correct answer. Havens and colleagues address this with a word- and phrase-level taxonomy of gender biased language. It has eleven bias types grouped into Person Name, Linguistic and Contextual categories. Its labels cover trans and gender diverse people, and annotators can mark a person's gender as unknown instead of guessing. The taxonomy grew out of linguistics, gender studies, information science and NLP literature, and was refined through participatory work with archivists at the University of Edinburgh's Centre for Research Collections. In a case study, four annotators labelled British English catalogue descriptions from the archives, which contain both historical and contemporary language. Labels could overlap, and every annotator's view was kept. The authors argue that anchoring bias in concrete text spans, rather than in abstract embedding metrics, makes bias easier to identify and measure. The planned datasets are meant to support diachronic analysis and gender bias classifiers that help archives decide which harmful descriptions to revise first.

Key points

  • Taxonomy structure: eleven gender bias types sit in three categories.
    • Person Name and Linguistic labels (e.g., Gendered Pronoun, Gendered Role) capture lexical cues.
    • Contextual labels (e.g., Generalization, Stereotype, Omission) capture how the text relates to social structures.
  • Labels added from stakeholder input:
    • Omission came from archivists, who noticed that descriptions of women were far less detailed than those of men. It became the most frequently applied Contextual label.
    • Empowering covers reclaimed terms such as "queer".
  • Annotators: four women annotated from their present-day perspective. Overlapping labels were allowed, and pre-annotation scripts supported the process.
  • Planned releases:
    • one dataset per annotator;
    • an aggregated set that keeps duplicates;
    • a deduplicated aggregate.
    • The data will be released only after further scrutiny with the archive, together with a data statement and the corpus-building code.
  • Framing: the work explicitly adopts data feminism and the Perspectivist Data Manifesto, rejecting the idea of a single ground truth.

Interesting ideas

  • I1. An explicit "unknown" value for a person's gender stops annotators from inventing information the text does not support, which keeps the data honest about its limits.
  • I2. Releasing each annotator's labels alongside both a duplicate-keeping and a deduplicated aggregate keeps disagreement visible instead of averaging it away.
  • I3. The taxonomy separates labels drawn from vocabulary inside the text from labels that require judging the text against its social context, and these two kinds call for different procedures.
  • I4. Neutral gendered words such as "he" become evidence of bias only when counted across a whole collection, so a skew can show up in aggregate even when no single sentence is biased.
  • I5. Treating omission as a bias signal means looking at what a text leaves out, for instance thin descriptions of some people compared with rich descriptions of others.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): entity attributes such as gender should allow an explicit unknown state rather than a default, so that later inferences and pronoun links do not rest on a fabricated fact.
  • I2 (learn): silver data from several witness parsers could store every witness's output plus two merged views, so that disagreements remain inspectable and usable for training or repair-rule induction.
  • I3 (frames): pragmatics could keep surface-cue detectors apart from context-dependent judgments (implicature, irony), with each layer explaining itself separately.
  • I4 (learn): evaluation could add distribution checks over outputs, such as how often pronouns resolve to male versus female entities, to reveal systematic skew that per-sentence accuracy hides.
  • I5 (world): the world model could compare how many states, events and attributes are recorded per entity, flagging entities that are mentioned but barely described.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of EdinburghEdinburgh · United Kingdom381 works · 80,747 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSocial Sciences · 1,954 worksSafety Research · 626 worksGender Studies · 52 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,379 worksBias, fairness & safety · 587 works

Cite

Cite as Lucy Havens, Melissa Terras, Benjamin Bach and Beatrice Alex. 2022. Uncertainty and Inclusivity in Gender Bias Annotation: An Annotation Taxonomy and Annotated Datasets of British English Text. Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.gebnlp-1.4

@inproceedings{havens2022uncertainty,
  title = {{Uncertainty and Inclusivity in Gender Bias Annotation: An Annotation Taxonomy and Annotated Datasets of British English Text}},
  author = {Lucy Havens, Melissa Terras, Benjamin Bach and Beatrice Alex},
  booktitle = {Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP)},
  year = {2022},
  doi = {10.18653/v1/2022.gebnlp-1.4},
  url = {https://doi.org/10.18653/v1/2022.gebnlp-1.4}
}

References

82 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Valerio Basile. 2022. The Perspectivist Data Manifesto. [Online; accessed March 21, 2022]. find
  2. Valerio Basile, Federico Cabitza, Andrea Campagner, and Michael Fell. 2021. Toward a Perspectivist Turn in Ground Truthing for Predictive Computing. CoRR, abs/2109.04270. ↗
  3. Christine Basta, Marta R Costa-jussà, and Noe Casas. 2020. Extensive Study on the Underlying Gender Bias in Contextualized Word Embeddings. Neural Computing & Applications, 33(8):3371–3384. ↗
  4. Emily M. Bender and Batya Friedman. 2018. Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science. Transactions of the Association for Computational Linguistics, 6:587–604. ↗
  5. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, page 610–623, New York, USA. Association for Computing Machinery. find
  6. Ruha Benjamin. 2019. Race after technology : abolitionist tools for the new Jim code. Polity, Cambridge, UK. github.com/thegoose20/annot Abeba Birhane and Vinay Uday Prabhu. 2021. Large image datasets: A pyrrhic win for computer vision? 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1536–1546. find
  7. Bronwyn M Bjorkman. 2017. Singular They and the Syntactic Representation of Gender in English. Glossa (London), 2(1):1. ↗
  8. Su Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (Technology) is Power: A Critical Survey of “Bias” in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454–5476.
  9. Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, and Hanna Wallach. 2021. Stereotyping Norwegian salmon: An inventory of pitfalls in fairness benchmark datasets. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1004–1015, Online. Association for Computational Linguistics. ↗
  10. Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. In Proceedings of the 30th International Conference on Neural Information Processing Systems, pages 4356–4364.
  11. Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2019. Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification. In WWW ‘19: Companion Proceedings of The 2019 World Wide Web Conference, WWW ’19, page 491–500, New York, USA. Association for Computing Machinery. find
  12. Mary Bucholtz. 1999. Gender. Journal of linguistic anthropology, 9(1-2):80–83. find
  13. Yang Trista Cao and Hal Daumé III. 2020. Toward gender-inclusive coreference resolution. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4568–4595, Online. Association for Computational Linguistics. ↗
  14. Yang Trista Cao and Hal Daumé III. 2021. Toward gender-inclusive coreference resolution: An analysis of gender and bias throughout the machine learning lifecycle*. Computational Linguistics, 47(3):615– 661. ↗
  15. Terry Cook. 2011. ’We Are What We Keep; We Keep What We Are’: Archival Appraisal Past, Present and Future. Journal of the Society of Archivists, 32(2):173–189. ↗
  16. M.Z. Corbett. 1990. Clearing the air: some thoughts on gender-neutral writing. IEEE Transactions on Professional Communication, 33(1):2–6. ↗
  17. Sasha Costanza-Chock and Nick Philip. 2018. Design Justice, A.I., and Escape from the Matrix of Domination. Journal of Design and Science. ↗
  18. Kate Crawford. 2017. The Trouble with Bias. In Neural Information Processing Systems Conference Keynote. [Online; accessed 10-July-2020]. find
  19. CRC. 2018. Collection: Papers and artwork of Yolanda Sonnabend relating to her collaboration with C.H. Waddington. [Online; accessed 19 May 2022. find
  20. Kimberlé Crenshaw. 1991. Mapping the Margins: Intersectionality, Identity Politics, and Violence against Women of Color. Stanford Law Review, 43(6):1241– 1299. ↗
  21. Aida Mostafazadeh Davani, Mark Díaz, and Vinodkumar Prabhakaran. 2022. Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations. Transactions of the Association for Computational Linguistics, 10:92–110. ↗
  22. Hannah Devinney, Jenny Björklund, and Henrik Björklund. 2022. Theories of “Gender” in NLP Bias Research. Computing Research Repository. find
  23. Catherine D’Ignazio and Lauren F. Klein. 2020. Data Feminism. Strong ideas series. MIT Press, Cambridge, USA. find
  24. Emily Dinan, Angela Fan, Adina Williams, Jack Urbanek, Douwe Kiela, and Jason Weston. 2020a. Queens are powerful too: Mitigating gender bias in dialogue generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8173–8188, Online. Association for Computational Linguistics.
  25. Emily Dinan, Angela Fan, Ledell Wu, Jason Weston, Douwe Kiela, and Adina Williams. 2020b. Multi-Dimensional Gender Bias Classification. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 314–331, Online. Association for Computational Linguistics. find
  26. Jad Doughman, Fatima Abu Salem, and Shady Elbassuoni. 2020. Time-aware word embeddings for three Lebanese news archives. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 4717–4725, Marseille, France. European Language Resources Association. find
  27. Jad Doughman, Wael Khreich, Maya El Gharib, Maha Wiss, and Zahraa Berjawi. 2021. Gender bias in text: Origin, taxonomy, and implications. In Proceedings of the 3rd Workshop on Gender Bias in Natural Language Processing, pages 34–44, Online. Association for Computational Linguistics. find
  28. Gordon Dunsire. 2018. Ethical issues in catalogue content standards. In Catalogue & Index, volume 191, pages 11–15. find
  29. Erick Elejalde, Leo Ferres, and Eelco Herder. 2017. The Nature of Real and Perceived Bias in Chilean Media. In Proceedings of the 28th ACM Conference on Hypertext and Social Media, HT ’17, page 95–104, New York, USA. Association for Computing Machinery. find
  30. Norman Fairclough. 2003. Analysing Discourse: Textual Analysis for Social Research. Routledge, London, UK. find
  31. Noah Geraci. 2019. Programmatic approaches to bias in descriptive metadata. In Code4Lib Conference 2019. [Online; accessed 28-May-2020]. find
  32. Ilka H. Gleibs. 2017. Are all “research fields” equal? Rethinking practice for the use of data from crowdsourcing market addresss. Behavior Research Methods, 49(4):1333–1342. find
  33. Seraphina Goldfarb-Tarrant, Rebecca Marchant, Ricardo Muñoz Sánchez, Mugdha Pandya, and Adam Lopez. 2021. Intrinsic bias metrics do not correlate with application bias. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1926–1940, Online. Association for Computational Linguistics.
  34. Hila Gonen and Yoav Goldberg. 2019. Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them. NAACL 2019, arXiv:1903.03862v2. ↗
  35. Gov.uk. 2022. National Minimum Wage and National Living Wage rates. find
  36. Donna Haraway. 1988. Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective. Feminist Studies, 14(3):575. ↗
  37. Sandra Harding. 1995. “Strong objectivity”: A response to the new objectivity question. Synthese, 104(3). find
  38. Lucy Havens, Melissa Terras, Benjamin Bach, and Beatrice Alex. 2020. Situated data, situated systems: A methodology to engage with power relations in natural language processing research. In Proceedings of the Second Workshop on Gender Bias in Natural Language Processing, pages 107–124, Barcelona, Spain (Online). Association for Computational Linguistics. ↗
  39. Yasmeen Hitti, Eunbee Jang, Ines Moreno, and Carolyne Pelletier. 2019. Proposed Taxonomy for Gender Bias in Text; A Filtering Methodology for the Gender Generalization Subtype. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, pages 8–17, Florence, IT. Association for Computational Linguistics. find
  40. Christoph Hube and Besnik Fetahu. 2019. Neural Based Statement Classification for Biased Language. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, pages 195– 203, Melbourne, AU. ACM. find
  41. Ben Hutchinson, Vinodkumar Prabhakaran, Emily Denton, Kellie Webster, Yu Zhong, and Stephen Denuyl. 2020. Social biases in NLP models as barriers for persons with disabilities. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5491–5501, Online. Association for Computational Linguistics.
  42. Ben Hutchinson, Andrew Smart, Alex Hanna, Emily Denton, Christina Greer, Oddur Kjartansson, Parker Barnes, and Margaret Mitchell. 2021. Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, page 560–575, New York, USA. Association for Computing Machinery. find
  43. ICA. 2011. ISAD(G): General International Standard Archival Description - Second edition. find
  44. May Jiang and Christiane Fellbaum. 2020. Interdependencies of gender and race in contextualized word embeddings. In Proceedings of the Second Workshop on Gender Bias in Natural Language Processing, pages 17–25, Barcelona, Spain (Online). Association for Computational Linguistics. ↗
  45. Eun Seo Jo and Timnit Gebru. 2020. Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, FAT* ’20, page 306–316, New York, USA. Association for Computing Machinery. ↗
  46. Os Keyes. 2018. The Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW). find
  47. Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019. Measuring Bias in Contextualized Word Representations. CoRR, abs/1906.07337. ↗
  48. Robin Lakoff. 1989. Language and Woman’s Place. Harper & Row, New York, USA. find
  49. Susan Leavy. 2018. Gender Bias in Artificial Intelligence: The Need for Diversity and Gender Theory in Machine Learning. In Proceedings of the 1st International Workshop on Gender Equality in Software Engineering, GE ’18, page 14–16, New York, USA. Association for Computing Machinery. find
  50. Jason Edward Lewis, Nick Philip, Noelani Arista, Archer Pechawis, and Suzanne Kite. 2018. Making Kin with the Machines. Journal of Design and Science. ↗
  51. Edward Loper and Steven Bird. 2002. NLTK: The Natural Language Toolkit. In Proceedings of the ACL-02 Workshop on Effective Tools and Methodologies for Teaching Natural Language Processing and Computational Linguistics - Volume 1, ETMTNLP ’02, pages 63–70, USA. Association for Computational Linguistics. ↗
  52. Bella Martin and Bruce Hanington. 2012. 11 Case studies. In Universal Methods of Design: 100 Ways to Research Complex Problems, Develop Innovative Ideas, and Design Effective Solutions, Beverly, USA. Rockport Publishers. find
  53. Melissa McCradden, Mjaye Mazwi, Shalmali Joshi, and James A. Anderson. 2020. When Your Only Tool Is A Hammer: Ethical Limitations of Algorithmic Fairness Solutions in Healthcare Machine Learning, page 109. Association for Computing Machinery, New York, USA. find
  54. Norena Shopland. 2020. A Practical Guide to Searching LGBTQIA Historical Records. Taylor & Francis Group, Milton. ↗
  55. Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model Cards for Model Reporting. Proceedings of the Conference on Fairness, Accountability, and Transparency - FAT*’19. find
  56. Dale Spencer. 2000. Language and reality: Who made the world? (1980). In Lucy Burke, Tony Crowley, and Alan Girvin, editors, The Routledge language and cultural theory reader. Routledge, London, UK. find
  57. Safiya Umoja Noble. 2018. Algorithms of Oppression: How Search Engines Reinforce Racism. New York University Press, New York, USA. find
  58. Helen O’Neill, Anne Welsh, David A Smith, Glenn Roe, and Melissa Terras. 2021. Text mining Mill: Computationally detecting influence in the writings of John Stuart Mill from library records. Digital Scholarship in the Humanities, 36(4):1013–1029. ↗
  59. Thomas Padilla. 2017. On a Collections as Data Imperative. UC Santa Barbara Previously Published Works. find
  60. Thomas Padilla. 2019. Responsible Operations: Data Science, Machine Learning, and AI in Libraries. OCLC Research. ↗
  61. Caroline Criado Perez. 2019. Invisible Women: Exposing Data Bias in a World Designed for Men. Vintage, London, UK. find
  62. Colleen Reid and Wendy Frisby. 2008. 6 Continuing the Journey: Articulating Dimensions of Feminist Participatory Action Research (FPAR). In The SAGE Handbook of Action Research, pages 93–105. SAGE Publications Ltd. find
  63. Yisi Sang and Jeffrey Stanton. 2022. The Origin and Value of Disagreement Among Data Labelers: A Case Study of Individual Differences in Hate Speech Annotation. In Information for a Better World: Shaping the Global Future, Lecture Notes in Computer Science, pages 425–444. Springer International Publishing, Cham. find
  64. Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019. The risk of racial bias in hate speech detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1668–1678, Florence, Italy. Association for Computational Linguistics. ↗
  65. Morgan Klaus Scheuerman, Katta Spiel, Oliver L. Haimson, Foad Hamidi, and Stacy M. Branham. 2020. HCI Guidelines for Gender Equity and Inclusion: Misgendering. find
  66. Muriel R. Schulz. 2000. The Semantic Derogation of Women. In Lucy Burke, Tony Crowley, and Alan Girvin, editors, The Routledge language and cultural theory reader. Routledge, London, UK. find
  67. Laurajane Smith. 2006. Uses of Heritage. Routledge, London, UK. ↗
  68. Karolina Stańczak and Isabelle Augenstein. 2021. A survey on gender bias in natural language processing. CoRR, abs/2112.14168. ↗
  69. Pontus Stenetorp, Sampo Pyysalo, Goran Topić, Sophia Ananiadou Tomoko Ohta, and Jun’ichi Tsujii. 2012. brat: a Web-based Tool for NLP-Assisted Text Annotation. In Proceedings of the Demonstrations Session at EACL 2012. Association for Computational Linguistics.
  70. Marja Liisa Swantz. 2008. 2 Participatory Action Research as Practice. In The SAGE Handbook of Action Research, pages 31–48. SAGE Publications Ltd. find
  71. Chris Sweeney and Maryam Najafian. 2019. A transparent framework for evaluating unintended demographic bias in word embeddings. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1662–1667, Florence, Italy. Association for Computational Linguistics. find
  72. Latanya Sweeney. 2013. Discrimination in online ad delivery. Communications of the ACM, 56(5):44–54. ↗
  73. Yi Chern Tan and L. Elisa Celis. 2019. Assessing Social and Intersectional Biases in Contextualized Word Representations. CoRR, abs/1911.01485. ↗
  74. G. Thomas Tanselle. 2002. The World as Archive. Common Knowledge, 8(2):402–406. ↗
  75. Trans Metadata Collective. 2022. A Mandate for Trans and Gender Diverse Metadata (draft; working title). find
  76. Sigal Vainapel, Opher Y. Shamir, Yulie Tenenbaum, and Gadi Gilam. 2015. The dark side of gendered language: The masculine-generic form as a cause for self-report bias. Psychological Assessment, 27(4):1513–1519. ↗
  77. Kellie Webster, Marta Recasens, Vera Axelrod, and Jason Baldridge. 2018. Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns. Computing Research Repository, arXiv:1810.05201. ↗
  78. Anne Welsh. 2016. The Rare Books Catalog and the Scholarly Database. Cataloging & Classification Quarterly, 54(5–6):317–337. ↗
  79. Chris Welty, Praveen Paritosh, and Lora Aroyo. 2019. Metrology for AI: From Benchmarks to Instruments. CoRR, abs/1911.01875. ↗
  80. Elizabeth Yale. 2015. The History of Archives: The State of the Discipline. Book History, 18(1):332–359. ↗
  81. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender bias in contextualized word embeddings. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 629–634, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  82. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 15–20, New Orleans, USA. Association for Computational Linguistics. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/havens-2022-uncertainty-inclusivity-gender-bias-annotation/