Elamur Research⌕
☰
home / papers / 2025 / Investigating the Capabilities and Limitations of Machine Learning for Identifying Bias in English Language Data with Information and Heritage Professionals

Investigating the Capabilities and Limitations of Machine Learning for Identifying Bias in English Language Data with Information and Heritage Professionals

Paper · 2025 · Lucy Havens, Benjamin Bach, Melissa Terras et al.

Summary

Cultural heritage institutions (galleries, libraries, archives, museums) increasingly review old catalog descriptions for harmful language, but doing this by hand across millions of records is slow. Havens et al. argue that mainstream ML treats bias as something to strip out, while heritage work needs bias kept visible and studied as a historical phenomenon. So they built classifiers that flag gender biased language in British English metadata from a Scottish archive's catalog, rather than trying to debias it.

The project had four stages: defining gendered and gender biased language, annotating a dataset, training and scoring the models, and evaluating them qualitatively. In the final stage, archivists, librarians and curators examined the models' output in a participatory workshop, and the authors analysed the discussion for its content.

Participants saw bias as contextual, shifting and unavoidable. The models broadened their sense of where bias hides, yet could not catch it fully or reliably. The study argues that for some uses, fairness-by-removal is neither achievable nor wanted.

Key points

  • The data is British English catalog metadata from a Scottish archive. Gender biased language is defined as wording that builds or reinforces unequal power through reductive, dehumanising or judgmental terms, or through favouring some gender identities.
  • The work combines HCI and ML. The workshop draws on participatory action research, critical discourse analysis and intersectional feminism, followed by content analysis of what participants said.
  • The classifiers include an Omission Classifier and a Stereotype Classifier. Both score higher on precision than on recall.
  • Participants named three practical uses for the models:
    • estimating how many flagged descriptions each collection has, so review can be prioritised;
    • improving how new descriptions are written;
    • documenting how much staff time cataloguing really needs.
  • For context, the Library of Congress takes in about 10,000 items per day and has roughly 3,200 staff, of whom only some are catalogers. Workshop participants called the bias review "intimidating" and "self-revealing."

Interesting ideas

  • I1. Turning debiasing into detection keeps the original text intact and adds a "potentially biased" label on top, so the phenomenon stays available for analysis.
  • I2. The paper separates bias types that a word list can catch (stereotyped terms) from types defined by what is missing (omission). The second kind needs reasoning about absent information.
  • I3. The authors advise running cheap traditional models inside a quantitative-plus-qualitative loop first, refining the task definition, data and model before spending on deep learning.
  • I4. When precision clearly exceeds recall, the authors plan error analysis on the missed cases to see whether they form one hidden pattern or several.
  • I5. The authors propose part-of-speech features to test whether grammatical gender marking predicts biased language. This checks how far surface morphology goes before deeper context is required.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): bias or stereotype marks could be attached as an explainable pragmatic layer on frames, leaving the source wording unchanged.
  • I2 (world): the text world could flag people who appear only through their relation to someone else, or roles with no filler, giving an explainable "absence" signal.
  • I3 (learn): new annotation layers could be prototyped with the perceptron-style learners and reviewed by people before any gparse-scale training.
  • I4 (learn): recall misses of each witness parser could be clustered to decide whether one repair rule or several distinct ones are needed.
  • I5 (tokens): gender-marked lemmas and pronouns in crate en could be exposed as features, letting higher stages measure their correlation with pragmatic labels.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of EdinburghEdinburgh · United Kingdom381 works · 80,747 citations🔬Institut national de recherche en sciences et technologies du numériqueLe Chesnay-Rocquencourt · France85 works · 25,498 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksSocial Sciences · 1,954 worksSafety Research · 626 worksPolitical Science and International Relations · 206 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,379 works

Cite

Cite as Lucy Havens, Benjamin Bach, Melissa Terras and Beatrice Alex. 2025. Investigating the Capabilities and Limitations of Machine Learning for Identifying Bias in English Language Data with Information and Heritage Professionals. arXiv:2504.00860.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1145/3706598.3713217 · arXiv 2504.00860

@misc{havens2025investigating,
  title = {{Investigating the Capabilities and Limitations of Machine Learning for Identifying Bias in English Language Data with Information and Heritage Professionals}},
  author = {Lucy Havens, Benjamin Bach, Melissa Terras and Beatrice Alex},
  year = {2025},
  doi = {10.1145/3706598.3713217},
  eprint = {2504.00860},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1145/3706598.3713217}
}

References

148 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Melissa Adler. 2017. Cruising the Library: Perversities in the Organization of Knowledge. Fordham University Press, New York. https://doi.org/10.2307/j. ctt1xhr79m ↗
  2. Christine M. Angel and Caroline Fuchs (Eds.). 2018. Organization, Representation and Description through the Digital Age: Information in Libraries, Archives and Museums. Walter de Gruyter GmbH, Boston. https://doi.org/10.1515/ 9783110337419 ↗
  3. Solon Barocas and Andrew D. Selbst. 2016. Big Data’s Disparate Impact. California Law Review 104, 3 (2016), 671–732. https://doi.org/10.15779/Z38BG31 ↗
  4. Valerio Basile, Federico Cabitza, Andrea Campagner, and Michael Fell. 2021. Toward a Perspectivist Turn in Ground Truthing for Predictive Computing. Proceedings of the AAAI Conference on Artificial Intelligence 37, 6 (June 2021), 6860–6868. https://doi.org/10.1609/aaai.v37i6.25840 ↗
  5. Christine Basta, Marta R Costa-jussà, and Noe Casas. 2020. Extensive Study on the Underlying Gender Bias in Contextualized Word Embeddings. Neural Computing & Applications 33, 8 (2020), 3371–3384. ↗
  6. Eric PS Baumer. 2017. Toward Human-Centered Algorithm Design. Big Data & Society 4, 2 (2017), 1–12. https://doi.org/10.1177/2053951717718854 ↗
  7. Mary Beard. 2017. Women & Power: A Manifesto. Profile Books Ltd, London. find
  8. Kaspar Beelen, Jon Lawrence, Daniel C. S. Wilson, and David Beavan. 2022. Bias and Representativeness in Digitized Newspaper Collections: Introducing the Environmental Scan. Digital Scholarship in the Humanities 38, 1 (July 2022), 1–22. https://doi.org/10.1093/llc/fqac037 ↗
  9. Dorothy Berry. 2020. Conscious Editing: Enhancing Diversity and Discovery. https://youtu.be/XGCTtDgNty4 ↗
  10. Abeba Birhane. 2020. Algorithmic Colonization of Africa. SCRIPT-ed 17, 2 (August 2020), 389–409. https://doi.org/10.2966/scrip.170220.389 ↗
  11. Abeba Birhane, Pratyusha Kalluri, Dallas Card, William Agnew, Ravit Dotan, and Michelle Bao. 2022. The Values Encoded in Machine Learning Research. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, KR) (FAccT ’22). ACM, New York, NY, US, 173–184. https://doi.org/10.1145/3531146.3533083 ↗
  12. Abeba Birhane and Vinay Uday Prabhu. 2021. Large Image Datasets: A Pyrrhic Win for Computer Vision?. In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, Waikoloa, HI, US, 1536–1546. https://doi.org/ 10.1109/WACV48630.2021.00158 ↗
  13. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (Technology) is Power: A Critical Survey of “Bias” in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. ACL, Online, 5454–5476. https://doi.org/10.18653/v1/2020.acl-main.485 ↗
  14. Francis X. Blouin, Jr. and William G. Rosenberg. 2011. Processing the Past: Contesting Authorities in History and the Archives (online ed.). Oxford University Press, Oxford, Chapter 8, 140–160. https://doi.org/10.1093/acprof: oso/9780199740543.001.0001 ↗
  15. Cat Bohannon. 2023. Eve: How the Female Body Drove 200 Million Years of Human Evolution (first ed.). Alfred A. Knopf, New York. find
  16. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching Word Vectors with Subword Information. Transactions of the Association for Computational Linguistics 5 (December 2017), 135–146. https://doi.org/10.1162/tacl_a_00051 ↗
  17. Geoffrey C. Bowker and Susan Leigh Star. 2000. Sorting Things Out: Classification and Its Consequences. The MIT Press, Cambridge. find
  18. Meredith Broussard. 2023. More Than a Glitch: Confronting Race, Gender, and Ability Bias in Tech. The MIT Press, Cambridge. find
  19. Mary Bucholtz. 1999. Gender. Journal of Linguistic Anthropology 9, 1-2 (1999), 80–83. find
  20. Mary Bucholtz. 2003. Theories of Discourse as Theories of Gender: Discourse Analysis in Language and Gender Studies. In The Handbook of Language and Gender. Blackwell Publishing Ltd, Oxford, 43–68. https://doi.org/10.1002/ 9780470756942.ch2 ↗
  21. Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (Proceedings of Machine Learning Research, Vol. 81). PMLR, Online, 1–15. https://proceedings.mlr.press/ v81/buolamwini18a/buolamwini18a.pdf ↗
  22. Jasmine Burns, Michelle Cronquist, Jackson Huang, Devon Murphy, K.J. Rawson, Beck Schaefer, Jamie Simons, Brian M. Watson, Adrian Williams, and The Trans Metadata Collective. 2022. Metadata Best Practices for Trans and Gender Diverse Resources. https://doi.org/10.5281/zenodo.6829167 ↗
  23. Yang Trista Cao and Hal Daumé III. 2020. Toward Gender-Inclusive Coreference Resolution. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (online). ACL, online, 4568–4595. https://doi.org/10. 18653/v1/2020.acl-main.418 ↗
  24. Michelle Caswell. 2022. Dusting for Fingerprints: Introducing Feminist Standpoint Appraisal. In Radical Empathy in Archival Practice (special issue on feminist ethics ed.), Vol. 3. Journal of Critical Library and Information Studies, online, 36 pages. Issue 1. https://journals.litwinbooks.com/index.php/jclis/article/view/ 113/67 ↗
  25. Michelle Caswell and Marika Cifor. 2016. From Human Rights to Feminist Ethics: Radical Empathy in the Archives. Archivaria 81 (May 2016), 23–43. https://archivaria.ca/index.php/archivaria/article/view/13557 ↗
  26. Chloe Ciora, Nur Iren, and Malihe Alikhani. 2021. Examining Covert Gender Bias: A Case Study in Turkish and English Machine Translation Models. In Proceedings of the 14th International Conference on Natural Language Generation. ACL, Aberdeen, Scotland, GB, 55–63. https://aclanthology.org/2021.inlg-1.7 ↗
  27. Catherine Nicole Coleman. 2020. Managing Bias When Library Collections Become Data. International Journal of Librarianship 5, 1 (July 2020), 8–19. https://doi.org/10.23974/ijol.2020.vol5.1.162 ↗
  28. Terry Cook. 2011. ‘We Are What We Keep; We Keep What We Are’: Archival Appraisal Past, Present and Future. Journal of the Society of Archivists 32, 2 (2011), 173–189. https://doi.org/10.1080/00379816.2011.619688 ↗
  29. Sam Corbett-Davies, Johann D. Gaebler, Hamed Nilforoshan, Ravi Shroff, and Sharad Goel. 2024. The Measure and Mismeasure of Fairness. The Journal of Machine Learning Research 24, 1, Article 312 (March 2024), 116 pages. find
  30. Ryan Cordell. 2020. Machine Learning + Libraries: A Report on the State of the Field. https://labs.loc.gov/static/labs/work/reports/Cordell-LOC-ML-report.pdf?loclr=blogsig ↗
  31. Kate Crawford. 2021. Atlas of AI: Power, Politics and the Planetary Costs of Artificial Intelligence. Yale University Press, New Haven. ↗
  32. Kate Crawford and Trevor Paglen. 2019. Excavating AI: The Politics of Training Sets for Machine Learning. The AI Now Institute, NYU, New York, NY, US. https://excavating.ai ↗
  33. Kimberlé Crenshaw. 1989. Demarginalizing the Intersection of Race and Sex: A Black Feminist Critique of Antidiscrimination Doctrine, Feminist Theory and Antiracist Politics. University of Chicago Legal Forum 1989, 1 (1989), 139–167. https://chicagounbound.uchicago.edu/uclf/vol1989/iss1/8 ↗
  34. Kimberlé Crenshaw. 1991. Mapping the Margins: Intersectionality, Identity Politics and Violence Against Women of Color. STANFORD LAW REVIEW 43, 6 (1991), 60. https://doi.org/10.2307/1229039 ↗
  35. Shipi Dhanorkar, Christine T. Wolf, Kun Qian, Anbang Xu, Lucian Popa, and Yunyao Li. 2021. Who Needs to Know What, When?: Broadening the Explainable AI (XAI) Design Space by Looking at Explanations Across the AI Lifecycle. In Designing Interactive Systems Conference 2021. ACM, virtual event (US), 1591– 1602. https://doi.org/10.1145/3461778.3462131 ↗
  36. Thiago Dias Oliva, Dennys Marcelo Antonialli, and Alessandra Gomes. 2021. Fighting Hate Speech, Silencing Drag Queens? Artificial Intelligence in Content Moderation and Risks to LGBTQ Voices Online. Sexuality & Culture 25, 2 (2021), 700–732. https://doi.org/10.1007/s12119-020-09790-w ↗
  37. Catherine D’Ignazio. 2024. Counting Feminicide: Data Feminism in Action. The MIT Press, Cambridge. https://doi.org/10.7551/mitpress/14671.001.0001 ↗
  38. Catherine D’Ignazio and Lauren F. Klein. 2020. Data Feminism. The MIT Press, Cambridge. https://mitpressonpubpub.mitpress.mit.edu/data-feminism ↗
  39. Emily Dinan, Angela Fan, Ledell Wu, Jason Weston, Douwe Kiela, and Adina Williams. 2020. Multi-Dimensional Gender Bias Classification. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). ACL, Online, 314–331. https://www.aclweb.org/anthology/2020. emnlp-main.23 ↗
  40. Carl DiSalvo. 2022. Design as Democratic Inquiry: Putting Experimental Civics into Practice. The MIT Press, Cambridge. https://doi.org/10.7551/mitpress/ 13372.001.0001 ↗
  41. Wendy M. Duff and Verne Harris. 2002. Stories and Names: Archival Description as Narrating Records and Constructing Meanings. Archival Science 2, 3–4 (September 2002), 263–285. https://doi.org/10.1007/BF02435625 ↗
  42. Gordon Dunsire. 2018. Ethical Issues in Catalogue Content Standards. Catalogue and Index June 2018 (2018), 11–15. Issue 191. find
  43. Norman Fairclough. 2003. Analysing Discourse: Textual Analysis for Social Research. Routledge, London. find
  44. Jonathan Furner. 2007. Dewey Deracialized: A Critical Race-Theoretic Perspective. Knowledge Organization 34, 3 (2007), 144–168. https://doi.org/10.5771/0943-7444-2007-3-144 ↗
  45. Barney G. Glaser and Anselm L. Strauss. 1980. The Discovery of Grounded Theory: Strategies for Qualitative Research. Aldine, New York. find
  46. Seraphina Goldfarb-Tarrant, Rebecca Marchant, Ricardo Muñoz Sánchez, Mugdha Pandya, and Adam Lopez. 2021. Intrinsic Bias Metrics Do Not Correlate with Application Bias. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). ACL, Online, 1926–1940. https://doi.org/10.18653/v1/2021.acl-long.150 ↗
  47. Samuel Goree and David Crandall. 2023. Situated Cameras, Situated Knowledges: Towards an Egocentric Epistemology for Computer Vision. In 3rd International Ego4D Workshop @ CVPR 2023, Vol. abs/2307.00064. ArXiv, Online, 4 pages. https://doi.org/10.48550/arxiv.2307.00064 ↗
  48. David Graeber and D. Wengrow. 2021. The Dawn of Everything: A New History of Humanity. Penguin Books Ltd., London, England, GB. ↗
  49. David Gray Widder, Sarah West, and Meredith Whittaker. 2023. Open (For Business): Big Tech, Concentrated Power, and the Political Economy of Open AI. (2023). https://doi.org/10.2139/ssrn.4543807 [accepted to appear in Nature]. ↗
  50. Mark A. Greene and Dennis Meissner. 2005. More Product, Less Process: Revamping Traditional Archival Processing. The American Archivist 68, Fall/Winter 2005 (2005), 208–263. https://www2.archivists.org/sites/all/files/ MPLP-AmericanArchivist-2005.pdf ↗
  51. Donna Haraway. 1988. Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective. Feminist Studies 14, 3 (1988), 575 pages. https://doi.org/10.2307/3178066 ↗
  52. Sandra Harding. 1995. “Strong Objectivity”: A Response to the New Objectivity Question. Synthese 104, 3 (September 1995), 331 pages. https://doi.org/10.1007/ BF01064504 ↗
  53. Lucy Havens. 2024. Recalibrating Machine Learning for Social Biases: Demonstrating a New Methodology through a Case Study Classifying Gender Biases in Archival Documentation. Ph. D. Dissertation. University of Edinburgh. ↗
  54. Lucy Havens. 2025. Data Bias and the Natural Language Processing of Metadata. In Library Catalogues as Data: Research, Practice, and Usage. Facet Publishing, London, chapter 4. ↗
  55. Lucy Havens, Melissa Terras, and Beatrice Alex. 2025. Confronting Gender Bias in Heritage Catalogues: A Natural Language Processing Approach to Revisiting Descriptive Metadata. In The Routledge Handbook of Heritage and Gender (Routledge Handbooks on Museums, Galleries and Heritage). Routledge, GB, 335–350. https://doi.org/10.4324/9781003258193-31 ↗
  56. Lucy Havens, Melissa Terras, Benjamin Bach, and Beatrice Alex. 2020. Situated Data, Situated Systems: A Methodology to Engage with Power Relations in Natural Language Processing Research. In Proceedings of the Second Workshop on Gender Bias in Natural Language Processing. ACL, Barcelona, Spain (Online), 107–124. https://aclanthology.org/2020.gebnlp-1.10 ↗
  57. Lucy Havens, Melissa Terras, Benjamin Bach, and Beatrice Alex. 2022. Uncertainty and Inclusivity in Gender Bias Annotation: An Annotation Taxonomy and Annotated Datasets of British English Text. In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP). ACL, Seattle, WA, US, 30–57. https://doi.org/10.18653/v1/2022.gebnlp-1.4 ↗
  58. Denise Hearn. 2024. Harms from Concentrated Industries: A Primer. Technical Report. Columbia Center on Sustainable Development. ↗
  59. Katy Hessel. 2023. The Story of Art Without Men (first American ed.). W.W. Norton & Company, Inc., New York, NY, US. find
  60. Mar Hicks. 2018. Introduction: Britain’s Computer “Revolution”. The MIT Press, Cambridge, 1–17. find
  61. Mar Hicks. 2021. Sexism is a Feature, Not a Bug. The MIT Press, Cambridge, Chapter 6, 135–158. ↗
  62. Patricia Hill Collins. 2000. Black Feminist Thought: Knowledge, Consciousness and the Politics of Empowerment. Routledge, New York. find
  63. Uta Hinrichs, Stefania Forlini, and Bridget Moynihan. 2019. In Defense of Sandcastles: Research Thinking Through Visualization in Digital Humanities. Digital Scholarship in the Humanities 34, Supplement 1 (Dec. 2019), i80–i99. https://doi.org/10.1093/llc/fqy051 ↗
  64. Anna Lauren Hoffmann. 2019. Where Fairness Fails: Data, Algorithms, and the Limits of Antidiscrimination Discourse. Information, Communication & Society 22, 7 (June 2019), 900–915. https://doi.org/10.1080/1369118X.2019.1573912 ↗
  65. Valentin Hofmann, Pratyusha Ria Kalluri, Dan Jurafsky, and Sharese King. 2024. AI Generates Covertly Racist Decisions About People Based on Their Dialect. Nature 633, 8028 (September 2024), 147–154. https://doi.org/10.1038/s41586-024-07856-5 ↗
  66. Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé III, Miro Dudik, and Hanna Wallach. 2019. Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19). ACM, Glasgow, Scotland, GB, 1–16. https://doi.org/10.1145/3290605.3300830 ↗
  67. Max Hort, Zhenpeng Chen, Jie M. Zhang, Mark Harman, and Federica Sarro. 2024. Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey. ACM Journal of Responsible Computing 1, 2 (2024), 52 pages. https://doi.org/10.1145/3631326 ↗
  68. Hilary Hutchinson, Wendy Mackay, Bo Westerlund, Benjamin B. Bederson, Allison Druin, Catherine Plaisant, Michel Beaudouin-Lafon, Stéphane Conversy, Helen Evans, Heiko Hansen, Nicolas Roussel, and Björn Eiderbäck. 2003. Technology Probes: Inspiring Design For and With Families. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, FL, US) (CHI ’03). ACM, New York, NY, US, 17–24. https://doi.org/10.1145/642611.642616 ↗
  69. ICA. 2011. ISAD(G): General International Standard Archival Description -Second Edition. https://www.ica.org/en/isadg-general-international-standard-archival-description-second-edition ↗
  70. Lise Jaillant (Ed.). 2022. Archives, Access and Artificial Intelligence: Working with Born-Digital and Digitized Archival Collections. Bielefeld University Press, Bielefeld, DE. https://doi.org/10.14361/9783839455845 ↗
  71. Sophie Jentzsch and Cigdem Turan. 2022. Gender Bias in BERT - Measuring and Analysing Biases through Sentiment Rating in a Realistic Downstream Classification Task. In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP). ACL, Seattle, WA, US, 184–199. https://doi.org/ 10.18653/v1/2022.gebnlp-1.20 ↗
  72. Xisen Jin, Francesco Barbieri, Brendan Kennedy, Aida Mostafazadeh Davani, Leonardo Neves, and Xiang Ren. 2021. On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. ACL, Online, 3770–3783. https://doi.org/10. 18653/v1/2021.naacl-main.296 ↗
  73. Eun Seo Jo and Timnit Gebru. 2020. Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning. In Proceedings of the 2020 Conference on Fairness, Accountability and Transparency (Barcelona, Spain) (FAT∗’20). ACM, New York, NY, US, 306–316. https://doi.org/10.1145/3351095.3372829 ↗
  74. Samia Kabir, Lixiang Li, and Tianyi Zhang. 2024. STILE: Exploring and Debugging Social Biases in Pre-trained Text Representations. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, US) (CHI ’24). ACM, New York, NY, US, Article 293, 20 pages. https://doi.org/10.1145/3613904.3642111 ↗
  75. Atoosa Kasirzadeh. 2022. Algorithmic Fairness and Structural Injustice: Insights from Feminist Political Philosophy. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. ACM, Oxford, England, GB, 349–356. https://doi.org/10.1145/3514094.3534188 ↗
  76. Atoosa Kasirzadeh and Iason Gabriel. 2023. In Conversation with Artificial Intelligence: Aligning Language Models with Human Values. Philosophy & Technology 36, 27 (April 2023), 1–24. https://doi.org/10.1007/s13347-023-00606-x ↗
  77. Lena Kástner and Barnaby Crook. 2024. Explaining AI through mechanistic interpretability. European Journal for Philosophy of Science 14, 4 (2024), 52. https://doi.org/10.1007/s13194-024-00614-4 ↗
  78. Os Keyes. 2018. The Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition. Proceedings of the ACM on Human-Computer Interaction 2, CSCW, Article 88 (November 2018), 22 pages. https://doi.org/10. 1145/3274357 ↗
  79. Lauren Klein and Catherine D’Ignazio. 2024. Data Feminism for AI. In The 2024 ACM Conference on Fairness, Accountability, and Transparency. ACM, Rio de Janeiro, Brazil, 100–112. https://doi.org/10.1145/3630106.3658543 ↗
  80. Mikhail Korobov. 2015. sklearn-crfsuite. https://sklearn-crfsuite.readthedocs.io/en/latest [online; accessed June 6, 2023]. ↗
  81. Emmanouil Krasanakis, Eleftherios Spyromitros-Xioufis, Symeon Papadopoulos, and Yiannis Kompatsiaris. 2018. Adaptive Sensitive Reweighting to Mitigate Bias in Fairness-aware Classification. In Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW ’18. ACM Press, Lyon, France, 853–862. https://doi.org/10.1145/3178876.3186133 ↗
  82. Klaus Krippendorff. 2018. Content Analysis: An Introduction to its Methodology. SAGE, Los Angeles. ↗
  83. Kaysi Eastlick Kushner and Raymond Morrow. 2003. Grounded Theory, Feminist Theory, Critical Theory: Toward Theoretical Triangulation. Advances in Nursing Science 26 (2003), 30–43. https://doi.org/10.1097/00012272-200301000-00006 ↗
  84. Faisal Ladhak, Esin Durmus, Mirac Suzgun, Tianyi Zhang, Dan Jurafsky, Kathleen McKeown, and Tatsunori Hashimoto. 2023. When Do Pre-Training Biases Propagate to Downstream Tasks? A Case Study in Text Summarization. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, Dubrovnik, Croatia, 3206–3219. https://aclanthology.org/2023.eacl-main.234 ↗
  85. Vivian Lai, Chacha Chen, Alison Smith-Renner, Q. Vera Liao, and Chenhao Tan. 2023. Towards a Science of Human-AI Decision Making: An Overview of Design Space in Empirical Human-Subject Studies. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (Chicago, IL, US) (FAccT ’23). ACM, New York, NY, US, 1369–1385. https://doi.org/10.1145/ 3593013.3594087 ↗
  86. Anne Lauscher and Goran Glavaš. 2019. Are We Consistently Biased? Multidimensional Analysis of Biases in Distributional Word Vectors. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), Rada Mihalcea, Ekaterina Shutova, Lun-Wei Ku, Kilian Evang, and Soujanya Poria (Eds.). Association for Computational Linguistics, Minneapolis, Minnesota, 85–91. https://doi.org/10.18653/v1/S19-1010 ↗
  87. LC Labs. n.d.. Exploring Computational Description. https://labs.loc.gov/work/ experiments/ECD/?loclr=blogsig [Accessed November 20, 2024]. ↗
  88. Patricia Leavy. 2017. Qualitative Research Design. Guilford Publications, New York, 131–163. https://ebookcentral.proquest.com/lib/ed/reader.action?docID= 4832778&ppg=244 ↗
  89. Q. Vera Liao, Daniel Gruen, and Sarah Miller. 2020. Questioning the AI: Informing Design Practices for Explainable AI User Experiences. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, Honolulu, HI, US, 1–15. https://doi.org/10.1145/3313831.3376590 ↗
  90. Q. Vera Liao and Kush R. Varshney. 2022. Human-Centered Explainable AI (XAI): From Algorithms to User Experiences. arXiv:2110.10790 [cs.AI] https://arxiv.org/abs/2110.10790 ↗
  91. Tom Liberum, Senthooran Rajamanoharan, Conmy Arthur, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah, and Neel Nanda. 2024. Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2. Technical report. Google DeepMind. https://storage.googleapis.com/gemma-scope/gemma-scope-report.pdf ↗
  92. Library of Congress. n.d.. Fascinating Facts | About the Library | Library of Congress. https://www.loc.gov/about/fascinating-facts [Accessed December 1, 2024]. ↗
  93. Library of Congress. n.d.. General Information | About the Library | Library of Congress. https://www.loc.gov/about/general-information/ [Accessed December 1, 2024]. ↗
  94. Sebastian Linxen, Christian Sturm, Florian Brühlmann, Vincent Cassau, Klaus Opwis, and Katharina Reinecke. 2021. How WEIRD is CHI?. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, Yokohama, Japan, 1–14. https://doi.org/10.1145/3411764.3445488 ↗
  95. Edward Loper and Steven Bird. 2002. NLTK: The Natural Language Toolkit. In Proceedings of the ACL-02 Workshop on Effective Tools and Methodologies for Teaching Natural Language Processing and Computational Linguistics - Volume 1 (Philadelphia, PA) (ETMTNLP ’02). ACL, US, 63–70. https://doi.org/10.3115/ 1118108.1118117 ↗
  96. Elizabeth Lorang, Leen-Kiat Soh, Yi Liu, and Chulwoo Pack. 2020. Digital Libraries, Intelligent Data Analytics, and Augmented Description: A Demonstration Project. https://labs.loc.gov/static/labs/work/experiments/final-report-revised_june-2020.pdf ↗
  97. Li Lucy and David Bamman. 2021. Gender and Representation Bias in GPT-3 Generated Stories. In Proceedings of the Third Workshop on Narrative Understanding. ACL, Virtual, 48–55. https://doi.org/10.18653/v1/2021.nuse-1.5 ↗
  98. Richard Marciano. 2022. Afterword: Towards a New Discipline of Computational Archival Science (CAS). Bielefeld University Press, Bielefeld, 205–218. ↗
  99. Nina Markl and Catherine Lai. 2021. Context-Sensitive Evaluation of Automatic Speech Recognition: Considering User Experience & Language Variation. In Proceedings of the First Workshop on Bridging Human–Computer Interaction and Natural Language Processing. ACL, Online, 34–40. https://aclanthology.org/ 2021.hcinlp-1.6 ↗
  100. Melissa McCradden, Mjaye Mazwi, Shalmali Joshi, and James A. Anderson. 2020. When Your Only Tool is a Hammer: Ethical Limitations of Algorithmic Fairness Solutions in Healthcare Machine Learning. ACM, New York, NY, US, 109. https://doi.org/10.1145/3375627.3375824 ↗
  101. Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A Survey on Bias and Fairness in Machine Learning. Comput. Surveys 54, 6, Article 115 (July 2021), 35 pages. https://doi.org/10.1145/3457607 ↗
  102. Wilhelmina Nekoto, Vukosi Marivate, Tshinondiwa Matsila, Timi Fasubaa, Tajudeen Kolawole, Taiwo Fagbohungbe, Solomon Oluwole Akinola, Shamsuddeen Hassan Muhammad, Salomon Kabongo, Salomey Osei, Sackey Freshia, Rubungo Andre Niyongabo, Ricky Macharm, Perez Ogayo, Orevaoghene Ahia, Musie Meressa, Mofe Adeyemi, Masabata Mokgesi-Selinga, Lawrence Okegbemi, Laura Jane Martinus, Kolawole Tajudeen, Kevin Degila, Kelechi Ogueji, Kathleen Siminyu, Julia Kreutzer, Jason Webster, Jamiil Toure Ali, Jade Abbott, Iroro Orife, Ignatius Ezeani, Idris Abdulkabir Dangana, Herman Kamper, Hady Elsahar, Goodness Duru, Ghollah Kioko, Espoir Murhabazi, Elan van Biljon, Daniel Whitenack, Christopher Onyefuluchi, Chris Emezue, Bonaventure Dossou, Blessing Sibanda, Blessing Itoro Bassey, Ayodele Olabiyi, Arshath Ramkilowan, Alp Öktem, Adewale Akinfaderin, and Abdallah Bashir. 2020. Participatory Research for Low-Resourced Machine Translation: A Case Study in African Languages. In Findings of the Association for Computational Linguistics: EMNLP 2020. ACL, Online, 2144–2160. https://www.aclweb.org/anthology/2020.findings-emnlp.195 ↗
  103. Safiya Umoja Noble. 2018. Algorithms of Oppression: How Search Engines Reinforce Racism. New York University Press, New York. find
  104. Mahsan Nourani, Amal Hashky, and Eric D. Ragan. 2024. User Profiling in Human-AI Design: An Empirical Case Study of Anchoring Bias, Individual Differences, and AI Attitudes. Proceedings of the Twelfth AAAI Conference on Human Computation and Crowdsourcing (HCOMP 2024) 12, 1 (October 2024), 137–146. https://doi.org/10.1609/hcomp.v12i1.31608 ↗
  105. Society of American Archivists. 2020. SAA Core Values Statement and Code of Ethics. https://www2.archivists.org/statements/saa-core-values-statement-and-code-of-ethics [online; accessed September 10, 2024]. ↗
  106. Hope A. Olson. 2001. The Power to Name: Representation in Library Catalogs. Signs: Journal of Women in Culture and Society 26, 3 (April 2001), 639–668. https://doi.org/10.1086/495624 ↗
  107. OpenAI. 2024. ChatGPT. https://chat.openai.com [online; accessed January 19, 2024]. ↗
  108. Hadas Orgad and Yonatan Belinkov. 2022. Choose Your Lenses: Flaws in Gender Bias Evaluation. In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP). ACL, Seattle, WA, 151–167. https://doi.org/10. 18653/v1/2022.gebnlp-1.17 ↗
  109. Alexandra Ortolja-Baird and Julianne Nyhan. 2022. Encoding the Haunting of an Object Catalogue: On the Potential of Digital Technologies to Perpetuate or Subvert the Silence and Bias of the Early-Modern Archive. Digital Scholarship in the Humanities 37, 3 (August 2022), 844–867. https://doi.org/10.1093/llc/fqab065 ↗
  110. Rock Yuren Pang, Jack Cenatempo, Franklyn Graham, Bridgette Kuehn, Maddy Whisenant, Portia Botchway, Katie Stone Perez, and Allison Koenecke. 2023. Auditing Cross-Cultural Consistency of Human-Annotated Labels for Recommendation Systems. In 2023 ACM Conference on Fairness, Accountability, and Transparency. ACM, Chicago, IL, US, 1531–1552. https://doi.org/10.1145/3593013. 3594098 ↗
  111. Mihir Parmar, Swaroop Mishra, Mor Geva, and Chitta Baral. 2023. Don’t Blame the Annotator: Bias Already Starts in the Annotation Instructions. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, Dubrovnik, Croatia, 1779–1789. https://doi.org/10.18653/v1/2023.eacl-main.130 ↗
  112. Amandalynne Paullada, Inioluwa Deborah Raji, Emily M. Bender, Emily Denton, and Alex Hanna. 2021. Data and Its (Dis)contents: A Survey of Dataset Development and Use in Machine Learning Research. Patterns 2, 11 (November 2021), 14 pages. https://doi.org/10.1016/j.patter.2021.100336 ↗
  113. Caroline Criado Perez. 2019. Invisible Women: Exposing Data Bias in a World Designed for Men. Vintage, London. find
  114. Dana Pessach and Erez Shmueli. 2022. A Review on Fairness in Machine Learning. ACM Computing Surveys (CSUR) 55, 3, Article 51 (February 2022), 44 pages. https://doi.org/10.1145/3494672 ↗
  115. David Pilgrim. 2024. Why I Collect Racist Objects - Jim Crow Museum. https://jimcrowmuseum.ferris.edu/collect.htm [online; accessed February 11, 2025]. ↗
  116. Miriam Posner. 2016. What’s Next: The Radical, Unrealized Potential of Digital Humanities. University of Minnesota Press, Minneapolis, 32–41. https://doi.org/10.5749/j.ctt1cn6thb.6 ↗
  117. Deborah Raji, Emily Denton, Emily M. Bender, Alex Hanna, and Amandalynne Paullada. 2021. AI and the Everything in the Whole Wide World Benchmark. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, Vol. 1. Curran, Online, 17 pages. https://datasets-benchmarks-proceedings.neurips.cc/paper_files/ paper/2021/file/084b6fbb10729ed4da8c3d3f5a3ae7c9-Paper-round2.pdf ↗
  118. Jane Ritchie and Jane Lewis. 2003. In-Depth Interviews. SAGE Publications Ltd, London, 138–168. find
  119. Colin Robson and Kieran McCartan. 2016. The Analysis and Interpretation of Qualitative Data. Wiley-Blackwell, Chichester, 459–486. find
  120. Kit T. Rodolfa, Erika Salomon, Lauren Haynes, Iván Higuera Mendieta, Jamie Larson, and Rayid Ghani. 2020. Case Study: Predictive Fairness to Reduce Misdemeanor Recidivism Through Social Service Interventions. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. ACM, Barcelona, Spain, 142–153. https://doi.org/10.1145/3351095.3372863 ↗
  121. Nihar Sahoo, Himanshu Gupta, and Pushpak Bhattacharyya. 2022. Detecting Unintended Social Bias in Toxic Language Datasets. In Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL). ACL, Abu Dhabi, United Arab Emirates (Hybrid), 132–143. https://aclanthology.org/2022. conll-1.10 ↗
  122. Nithya Sambasivan, Erin Arnesen, Ben Hutchinson, Tulsee Doshi, and Vinodkumar Prabhakaran. 2021. Re-Imagining Algorithmic Fairness in India and Beyond. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (Virtual Event, CA) (FAccT ‘21). ACM, New York, NY, US, 315–328. https://doi.org/10.1145/3442188.3445896 ↗
  123. Michele Samorani, Shannon L. Harris, Linda Goler Blount, Haibing Lu, and Michael A. Santoro. 2022. Overbooked and Overlooked: Machine Learning and Racial Bias in Medical Appointment Scheduling. Manufacturing & Service Operations Management 24, 6 (2022), 2825–2842. https://doi.org/10.1287/msom. 2021.0999 ↗
  124. Anna Schmidt and Michael Wiegand. 2017. A Survey on Hate Speech Detection using Natural Language Processing. In Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media. ACL, Valencia, ES, 1–10. https://doi.org/10.18653/v1/W17-1101 ↗
  125. Muriel R. Schulz. 2000. The Semantic Derogation of Women. In The Routledge Language and Cultural Theory Reader. Routledge, London. find
  126. Joan M. Schwartz and Terry Cook. 2002. Archives, Records, and Power: The Making of Modern Memory. Archival Science 2, 1–2 (2002), 1–19. https://doi.org/10.1007/BF02435628 ↗
  127. Ben Shneiderman. 2022. Human-Centered AI. Oxford University Press, New York. ↗
  128. Norena Shopland. 2020. A Practical Guide to Searching LGBTQIA Historical Records. Taylor & Francis Group, Milton. https://doi.org/10.4324/9781003006787 ↗
  129. Laurajane Smith. 2006. Uses of Heritage. Routledge, London. ↗
  130. Susan Leigh Star and James R. Griesemer. 1989. Institutional Ecology, ‘Translations’ and Boundary Objects: Amateurs and Professionals in Berkeley’s Museum of Vertebrate Zoology, 1907-39. Social Studies of Science 19, 3 (1989), 387–420. https://doi.org/10.1177/030631289019003001 ↗
  131. Ryan Steed, Swetasudha Panda, Ari Kobren, and Michael Wick. 2022. Upstream Mitigation Is Not All You Need: Testing the Bias Transfer Hypothesis in Pre-Trained Language Models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). ACL, Dublin, Ireland, 3524–3542. https://doi.org/10.18653/v1/2022.acl-long.247 ↗
  132. Ann Laura Stoler. 2002. Colonial archives and the arts of governance. Archival Science 2, 1–2 (2002), 87–109. https://doi.org/10.1007/BF02435632 ↗
  133. Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019. Mitigating Gender Bias in Natural Language Processing: Literature Review. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. ACL, Florence, Italy, 1630–1640. https://doi.org/10.18653/v1/P19-1159 ↗
  134. Marja Liisa Swantz. 2008. Participatory Action Research as Practice. SAGE Publications Ltd, London, Chapter 2, 31–48. https://doi.org/10.4135/9781848607934.n8 ↗
  135. Mohammad Tahaei, Marios Constantinides, Daniele Quercia, Sean Kennedy, Michael Muller, Simone Stumpf, Q. Vera Liao, Ricardo Baeza-Yates, Lora Aroyo, Jess Holbrook, Ewa Luger, Michael Madaio, Ilana Golbin Blumenfeld, Maria De-Arteaga, Jessica Vitak, and Alexandra Olteanu. 2023. Human-Centered Responsible Artificial Intelligence: Current & Future Trends. In CHI EA ’23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI EA ’23). ACM, New York, NY, US, Article 515, 4 pages. https://doi.org/10.1145/3544549.3583178 ↗
  136. Jessica Tai. 2021. Cultural Humility as a Framework for Anti-Oppressive Archival Description. Journal of Critical Library and Information Studies 3, 2 (October 2021), 23 pages. https://doi.org/10.24242/jclis.v3i2.120 ↗
  137. Melissa Terras, James Baker, James Hetherington, David Beavan, Martin Zaltz Austwick, Anne Welsh, Helen O’Neill, Will Finley, Oliver Duke-Williams, and Adam Farquhar. 2018. Enabling Complex Analysis of Large-Scale Digital Collections: Humanities Research, High-Performance Computing, and Transforming Access to British Library Digital Collections. Digital Scholarship in the Humanities 33, 2 (June 2018), 456–466. https://doi.org/10.1093/llc/fqx020 ↗
  138. Theo Thomassen. 2002. A First Introduction to Archival Science. Archival Science 1 (2002), 373–385. ↗
  139. Nanna Bonde Thylstrup, Daniela Agostinho, Annie Ring, Catherine D’Ignazio, and Kristin Veel (Eds.). 2021. Uncertain Archives: Critical Keywords for Big Data. The MIT Press, Cambridge. find
  140. Ciaran B. Trace. 2022. Archival Infrastructure and the Information Backlog. Archival Science 22, 1 (2022), 75–93. https://doi.org/10.1007/s10502-021-09368-x ↗
  141. Kiri Wagstaff. 2012. Machine Learning that Matters. In Proceedings of the 29th International Conference on Machine Learning (Edinburgh, Scotland, UK) (ICML ‘12). icml.cc / Omnipress, Madison, WI, US, 6 pages. https://icml.cc/2012/papers/ 298.pdf ↗
  142. Zijie J. Wang, Dongjin Choi, Shenyu Xu, and Diyi Yang. 2021. Putting Humans in the Natural Language Processing Loop: A Survey. In Proceedings of the First Workshop on Bridging Human–Computer Interaction and Natural Language Processing, Su Lin Blodgett, Michael Madaio, Brendan O’Connor, Hanna Wallach, and Qian Yang (Eds.). Association for Computational Linguistics, Online, 47–52. https://aclanthology.org/2021.hcinlp-1.8 ↗
  143. Kellie Webster, Marta Recasens, Vera Axelrod, and Jason Baldridge. 2018. Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns. Transactions of the Association for Computational Linguistics 6 (2018), 605–617. https://doi.org/10.1162/tacl_a_00240 ↗
  144. Anne Welsh. 2016. The Rare Books Catalog and the Scholarly Database. Cataloging & Classification Quarterly 54, 5–6 (August 2016), 317–337. https://doi.org/10.1080/01639374.2016.1188433 ↗
  145. Anne Welsh and Sue Batley. 2009. Practical Cataloguing: AACR, RDA and MARC21. Facet Publishing, London. find
  146. Iris Marion Young. 2011. Responsibility for Justice. Oxford University Press, Cary. https://doi.org/10.1093/acprof:oso/9780195392388.001.0001 ↗
  147. Chien Wen (Tina) Yuan, Nanyi Bi, Ya-Fang Lin, and Yuen-Hsien Tseng. 2023. Contextualizing User Perceptions about Biases for Human-Centered Explainable Artificial Intelligence. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, Hamburg Germany, 1–15. https://doi.org/10.1145/ 3544548.3580945 ↗
  148. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers). ACL, New Orleans, LA, US, 15–20. https://doi.org/10.18653/v1/ N18-2003 ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/havens-2025-investigating-capabilities-limitations-machine-learning/