Elamur Research⌕
☰
home / papers / 2021 / Gender and Representation Bias in GPT-3 Generated Stories

Gender and Representation Bias in GPT-3 Generated Stories

Summary

The paper asks whether stories written by GPT-3 reproduce familiar gender stereotypes. The authors drew single-sentence prompts naming a main character from 402 English contemporary fiction books, using BookNLP to identify characters. They had the davinci model continue each prompt several times. As a control, they took human-written excerpts from the same books that start with the same sentence. Character gender, as a reader would likely perceive it, was inferred from pronouns, then honorifics, then U.S. birth-name statistics. The team compared topics using LDA and measured appearance, intellect and power in character descriptions using lexicon-based embedding similarity, including SemAxis for power.

GPT-3 features more masculine than feminine characters, as the books do. Feminine characters lean toward family, emotions and looks, and they come across as weaker. These gaps persist when the identical prompt yields a "she" in one story and a "he" in another. Cognitive verbs in prompts raised intellect scores, but high-power verbs did not make characters more powerful. The study shows that bias can surface without any explicit cue in the prompt. This is a concern for anyone using large language models for creative writing.

Key points

  • Data: 2154 main characters, each with 10 prompts and 5 completions per prompt. Generation used davinci with temperature 0.9 and an 1800-token cap. The generated stories total over 161M tokens, and the matched book excerpts total over 32M tokens.
  • Gender labelling: A character was labelled by pronouns if at least 75% pointed to one gender. Otherwise, honorifics were used, followed by Social Security Administration names from 1990–2019 with a 90% threshold.
  • Controlled comparison: A matched subset of 7334 GPT-3 stories pairs outputs from the same prompt where the character was gendered differently.
  • Topic analysis: LDA with 50 topics was trained in MALLET. Gendered topic gaps in matched stories correlate with those in all generated stories (Pearson r = 0.91). Generated-story gaps also mirror those in books (r = 0.84).
  • Prompt steering: Bloom's Taxonomy cognitive verbs raised intellect scores. Sap et al. (2017) high-power verbs left non-masculine characters less powerful than masculine ones. Code is available at github.com/lucy3/gpt3_gender.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

98 papers of our library cite it (of the 140884 whose bibliographies we read); 17 of them have a page here, 81 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.nuse-1.5

@inproceedings{lucy2021gender,
  title = {{Gender and Representation Bias in GPT-3 Generated Stories}},
  author = {Lucy, Li and Bamman, David},
  booktitle = {Proceedings of the Third Workshop on Narrative Understanding},
  year = {2021},
  doi = {10.18653/v1/2021.nuse-1.5},
  url = {https://doi.org/10.18653/v1/2021.nuse-1.5}
}

References

48 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abubakar Abid, Maheen Farooqi, and James Zou. 2021. Persistent anti-Muslim bias in large language models. arXiv preprint arXiv:2101.05783. find our card
  2. Bronwyn M Bjorkman. 2017. Singular they and the syntactic representation of gender in English. Glossa: A Journal of General Linguistics, 2(1). find
  3. Lauren Ackerman. 2019. Syntactic and cognitive issues in investigating gendered coreference. Glossa: A Journal of General Linguistics, 4(1). find
  4. Cameron Blevins and Lincoln Mullen. 2015. Jane, john... leslie? a historical method for algorithmic gender prediction. DHQ: Digital Humanities Quarterly, 9(3). find
  5. Nader Akoury, Shufan Wang, Josh Whiting, Stephen Hood, Nanyun Peng, and Mohit Iyyer. 2020. STO-RIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6470–6484, Online.… find
  6. Jordan Boyd-Graber, Yuening Hu, and David Mimno. 2017. Applications of topic models. Foundations and Trends® in Information Retrieval, 11(2-3):143– 296. find
  7. Gwern Branwen. 2020. GPT-3 creative fiction. find
  8. Greg Brockman, Mira Murati, Peter Welinder, and OpenAI. 2020. OpenAI API. OpenAI Blog. find
  9. Vladimir Alexeev. 2020. GPT-3: Creative potential of NLP. Towards Data Science. find
  10. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… find
  11. Jisun An, Haewoon Kwak, and Yong-Yeol Ahn. 2018. SemAxis: A lightweight framework to characterize domain-specific word semantics beyond sentiment. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2450–2461, Melbourne, Australia… find
  12. L.W. Anderson, B.S. Bloom, D.R. Krathwohl, P. Airasian, K. Cruikshank, R. Mayer, P. Pintrich, J. Raths, and M. Wittrock. 2001. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. Longman. find
  13. Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. find
  14. David Bamman, Olivia Lewke, and Anya Mansoor. 2020. An annotated dataset of coreference in English literature. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 44–54, Marseille, France. European Language Resources Association. find our card
  15. Yang Trista Cao and Hal Daumé III. 2020. Toward gender-inclusive coreference resolution. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4568–4595, Online. Association for Computational Linguistics. find our card
  16. David Bamman, Brendan O’Connor, and Noah A. Smith. 2013. Learning latent personas of film characters. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 352–361, Sofia, Bulgaria. Association for Computational Linguistics. find our card
  17. Elizabeth Clark, Anne Spencer Ross, Chenhao Tan, Yangfeng Ji, and Noah A. Smith. 2018. Creative writing with a machine in the loop: Case studies on slogans and stories. In 23rd International Conference on Intelligent User Interfaces, IUI ’18, page 329–340, New York, NY, USA. Association for Computin… find
  18. David Bamman, Ted Underwood, and Noah A. Smith. 2014. A Bayesian mixed effects model of literary Max Kreminski, Melanie Dickinson, Michael Mateas, and Noah Wardrip-Fruin. 2020. Why are we like this?: The AI architecture of a co-creative storytelling game. In International Conference on the Foundatio… find
  19. Katherine Elkins and Jon Chun. 2020. Can GPT-3 pass a writer’s Turing Test? Journal of Cultural Analytics. find
  20. Ethan Fast, Binbin Chen, and Michael S Bernstein. 2016a. Empath: Understanding topic signals in large-scale text. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pages 4647–4657. find
  21. Brian Larson. 2017. Gender as a variable in natural-language processing: Ethical considerations. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, pages 1–11, Valencia, Spain. Association for Computational Linguistics. find
  22. Ethan Fast, Tina Vachovsky, and Michael Bernstein. 2016b. Shirtless and dangerous: Quantifying linguistic signals of gender bias in an online fiction writing community. In Proceedings of the International AAAI Conference on Web and Social Media, volume 10. find
  23. Marilee Long, Jocelyn Steinke, Brooks Applegate, Maria Knight Lapinski, Marne J. Johnson, and Sayani Ghosh. 2010. Portrayals of male and female scientists in television programs popular among middle school-age children. Science Communication, 32(3):356–382. find
  24. Eva Flicker. 2003. Between brains and breasts—women scientists in fiction film: On the marginalization and sexualization of scientific competence. Public Understanding of Science, 12(3):307–318. find
  25. Li Lucy, Dorottya Demszky, Patricia Bromley, and Dan Jurafsky. 2020. Content analysis of textbooks via natural language processing: Findings on gender, race, and ethnicity in Texas U.S. history textbooks. AERA Open, 6(3):2332858420940312. find
  26. Dhruvil Gala, Mohammad Omar Khursheed, Hannah Lerner, Brendan O’Connor, and Mohit Iyyer. 2020. Analyzing gender bias within narrative tropes. In Proceedings of the Fourth Workshop on Natural Language Processing and Computational Social Science, pages 212–217, Online. Association for Computational Li… find
  27. Tomas Mikolov, Greg Corrado, Kai Chen, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In Proceedings of the International Conference on Learning Representations (ICLR), pages 1–12. find
  28. Andrew Goldstone and Ted Underwood. 2014. The quiet transformations of literary studies: What thirteen thousand scholars could tell us. New Literary History, 45(3):359–384. find
  29. Corinne A. Moss-Racusin, John F. Dovidio, Victoria L. Brescoll, Mark J. Graham, and Jo Handels-man. 2012. Science faculty’s subtle gender biases favor male students. Proceedings of the National Academy of Sciences, 109(41):16474–16479. find
  30. Alexander Miserlis Hoyle, Lawrence Wolf-Sonkin, Hanna Wallach, Isabelle Augenstein, and Ryan Cotterell. 2019. Unsupervised discovery of gendered language through latent-variable modeling. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1706– 1716, Fl… find
  31. Nikola Mrkšić, Diarmuid Ó Séaghdha, Blaise Thomson, Milica Gašić, Lina M. Rojas-Barahona, Pei-Hao Su, David Vandyke, Tsung-Hsien Wen, and Steve Young. 2016. Counter-fitting word vectors to linguistic constraints. In Proceedings of the 2016 Conference of the North American Chapter of the Associatio… find
  32. Sherrie A. Inness. 2008. Geek Chic: Smart Women in Popular Culture. Palgrave Macmillan. find
  33. Brendan T. Johns and Melody Dye. 2019. Gender bias at scale: Evidence from the usage of personal names. Behavior Research Methods, 51(4). find
  34. Brian A Nosek, Mahzarin R Banaji, and Anthony G Greenwald. 2002. Harvesting implicit group attitudes and beliefs from a demonstration web site. Group Dynamics: Theory, Research, and Practice, 6(1):101. find
  35. Hannah Kirk, Yennie Jun, Haider Iqbal, Elias Benussi, Filippo Volpin, Frederic A. Dreyer, Aleksandar Shtedritski, and Yuki M. Asano. 2021. How true is gpt-2? an empirical analysis of intersectional occupational biases. find
  36. Rita Raley and Minh Hua. 2020. Playing with unicorns: AI dungeon and citizen NLP. Digital Humanities Quarterly, 14(4). find
  37. Austin C. Kozlowski, Matt Taddy, and James A. Evans. 2019. The geometry of culture: Analyzing the meanings of class through word embeddings. American Sociological Review, 84(5):905–949. find
  38. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), Eve Kraic… find our card
  39. Maarten Sap, Marcella Cindy Prasettio, Ari Holtzman, Hannah Rashkin, and Yejin Choi. 2017. Connotation frames of power and agency in modern films. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2329–2334, Copenhagen, Denmark. Association for Computat… find
  40. Kevin Scott. 2020. Microsoft teams up with OpenAI to exclusively license GPT-3 language model. The Official Microsoft Blog. find
  41. Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. 2020. Towards Controllable Biases in Language Generation. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3239–3254, Online. Association for Computational Linguistics. find our card
  42. Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2019. The woman worked as a babysitter: On biases in language generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Process… find our card
  43. Vered Shwartz, Rachel Rudinger, and Oyvind Tafjord. 2020. “you are grounded!”: Latent name artifacts in pre-trained language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6850–6861, Online. Association for Computational Linguistics. find
  44. Stacy L Smith, Marc Choueiti, Ashley Prescott, and Katherine Pieper. 2012. Gender roles & occupations: A look at character attributes and job-related aspirations in film and television. Geena Davis Institute on Gender in Media, pages 1–46. find
  45. Social Security Administration. 2020. Popular baby names: Beyond the top 1000 names. National Data. find
  46. Peter D. Turney and Michael L. Littman. 2003. Measuring praise and criticism: Inference of semantic orientation from association. ACM Trans. Inf. Syst., 21(4):315–346. find
  47. William E Underwood, David Bamman, and Sabrina Lee. 2018. The transformation of gender in English-language fiction. Journal of Cultural Analytics, 1(1). find
  48. Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. Universal adversarial triggers for attacking and analyzing NLP. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Proce… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lucy-2021-gender-representation-bias-gpt-3/