Elamur Research⌕
☰
home / papers / 2026 / Caged Birds and Cute Bookworms: Feminine Tropes and Implicit Gender Bias in Large Language Models

Caged Birds and Cute Bookworms: Feminine Tropes and Implicit Gender Bias in Large Language Models

Paper · 2026 · Sachita Nishal and Jack Bandy

Summary

The paper asks whether LLMs carry hidden gender assumptions into creative writing when a prompt never mentions gender. The authors built a benchmark from the crowd-sourced TV Tropes wiki. They scored each trope's genderedness using the method of Gala et al. (2020) and kept strongly feminine-leaning tropes. They then removed tropes with gendered words in their titles. Both authors annotated the rest by hand, keeping only implicitly gendered tropes about a single character.

Each trope became 12 story prompts: three wordings crossed with four pronoun conditions (none, they, she, he). The prompts were given to four open-weight instruction-tuned models, and protagonist gender was inferred mainly from pronouns.

All models defaulted to female protagonists when no gender was given. They usually ignored requested they/them pronouns, and sometimes made a character female even when asked for he/him.

The authors argue that bias in generated stories is a matter of narrative and character construction, not just word choice. This matters as LLMs power storytelling tools.

Key points

  • Dataset filtering: 19,727 de-duplicated TV Tropes entries were narrowed in stages.
    • 988 tropes in the top 5th percentile of feminine genderedness
    • 522 tropes after removing gendered titles
    • 211 manually verified tropes (Cohen's Kappa = 0.65 on a 200-trope sample)
  • Prompt design: Plot, Summary, and Sentences wordings were crossed with Ambiguous, They, She, and He conditions. This gave 2,532 stories per model.
  • Models tested: Gemma 3 (12B), Llama 3.1 (8B), Phi-4 (14B), and Qwen 3 (14B).
  • Main findings: The feminine default held across all three wordings, though Plot prompts produced somewhat fewer female protagonists. Phi-4 often avoided the problem by choosing neutral names such as Alex or Jamie, which the authors see as a surface-level fix.
  • Limitations: The data is English-language and Global North–centric, only gender is studied (no intersectional identities), and the dataset is released on GitHub.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Social Sciences · 1,772 worksSafety Research · 543 worksGender Studies · 52 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksBias, fairness & safety · 517 works

Cite

Cite as Sachita Nishal and Jack Bandy. 2026. Caged Birds and Cute Bookworms: Feminine Tropes and Implicit Gender Bias in Large Language Models. Proceedings of the Workshop on Evaluating Evaluations (EvalEval).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.evaleval-1.22

@inproceedings{nishal2026caged,
  title = {{Caged Birds and Cute Bookworms: Feminine Tropes and Implicit Gender Bias in Large Language Models}},
  author = {Sachita Nishal and Jack Bandy},
  booktitle = {Proceedings of the Workshop on Evaluating Evaluations (EvalEval)},
  year = {2026},
  doi = {10.18653/v1/2026.evaleval-1.22},
  url = {https://doi.org/10.18653/v1/2026.evaleval-1.22}
}

References

30 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abubakar Abid, Maheen Farooqi, and James Zou. 2021. Persistent anti-muslim bias in large language models. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 298–306.
  2. Nader Akoury, Shufan Wang, Josh Whiting, Stephen Hood, Nanyun Peng, and Mohit Iyyer. 2020. STO-RIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6470–6484, Online.… find
  3. Connor Baumler and Rachel Rudinger. 2022. Recognition of they/them as singular personal pronouns in coreference resolution. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3426–3432, Seattle, Uni…
  4. Amanda Bertsch, Ashley Oh, Sanika Natu, Swetha Gangu, Alan W. Black, and Emma Strubell. 2022. Evaluating Gender Bias Transfer from Film Data. In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 235–243, Seattle, Washington. Association for Computational L… find
  5. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29.
  6. Mandar S. Chaudhary and Arnav Jhala. 2022. Computational Support for Trope Analysis of Textual Narratives. In Interactive Storytelling, Lecture Notes in Computer Science, pages 529–540, Cham. Springer International Publishing. find
  7. Myra Cheng, Tiziano Piccardi, and Diyi Yang. 2023. Compost: Characterizing and evaluating caricature in llm simulations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10853–10875. find
  8. Jean-Peïc Chou, Alexa Fay Siu, Nedim Lipka, Ryan Rossi, Franck Dernoncourt, and Maneesh Agrawala. 2023. TaleStream: Supporting Story Ideation with Trope Knowledge. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST ’23, pages 1–12, New York, NY, USA. Asso… find
  9. Chengyu Chuang and Yi Yang. 2022. Buy tesla, sell ford: Assessing implicit stock market preference in pre-trained language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 100–105. find
  10. Ruadhán Mac Cormaic. 2023. A message from the Editor. The Irish Times. find
  11. Sorelle Friedler, Ranjit Singh, Borhane Blili-Hamelin, Jacob Metcalf, and Brian J Chen. 2023. AI Red-Teaming Is Not a One-Stop Solution to AI Harms: Recommendations for Using Red-Teaming for AI Accountability. Technical report, Data and Society. find
  12. Dhruvil Gala, Mohammad Omar Khursheed, Hannah Lerner, Brendan O’Connor, and Mohit Iyyer. 2020. Analyzing Gender Bias within Narrative Tropes. In Proceedings of the Fourth Workshop on Natural Language Processing and Computational Social Science, pages 212–217, Online. Association for Computational Li… find
  13. Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. 2018. Word embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences, 115(16):E3635– E3644. find
  14. Tarleton Gillespie. 2024. Generative ai and the politics of visibility. Big Data & Society, 11(2):20539517241252131. find
  15. Hadas Kotek, Rikker Dockum, and David Sun. 2023. Gender bias and stereotypes in large language models. In Proceedings of The ACM Collective Intelligence Conference, pages 12–24. find
  16. Eve Kraicer and Andrew Piper. 2019. Social characters: the hierarchy of gender in contemporary english-language fiction. Journal of Cultural Analytics, 3(2). find
  17. Li Lucy and David Bamman. 2021. Gender and Representation Bias in GPT-3 Generated Stories. In Proceedings of the Third Workshop on Narrative Understanding, pages 48–55, Virtual. Association for Computational Linguistics.
  18. Moin Nadeem, Anna Bethke, and Siva Reddy. 2021. Stereoset: Measuring stereotypical bias in pretrained language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long…
  19. Will Oremus. 2023. He wrote a book on a rare subject. Then a ChatGPT replica appeared on Amazon. Washington Post. find
  20. Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel Bowman. 2022. Bbq: A hand-built bias benchmark for question answering. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2086–2105.
  21. Mia Sato. 2023. AI-generated fiction is flooding literary magazines — but not fooling anyone. find
  22. Hanieh Shakeri, Carman Neustaedter, and Steve Di-Paola. 2021. SAGA: Collaborative Storytelling with GPT-3. In Companion Publication of the 2021 Conference on Computer Supported Cooperative Work and Social Computing, CSCW ’21 Companion, pages 163–166, New York, NY, USA. Association for Computing Mach… find
  23. Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. 2019. The woman worked as a babysitter: On biases in language generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (…
  24. Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. 2020. Towards controllable biases in language generation. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3239–3254.
  25. Lihao Sun, Chengzhi Mao, Valentin Hofmann, and Xuechunzi Bai. 2025. Aligned but blind: Alignment increases implicit bias by reducing awareness of race. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 22167–22184, Vienna, Austr…
  26. Ted Underwood, David Bamman, and Sabrina Lee. 2018. The transformation of gender in english-language fiction. Journal of Cultural Analytics, 3(2). find
  27. Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. DIALOGPT : Large-scale generative pre-training for conversational response generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Ling…
  28. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Gender bias in coreference resolution: Evaluation and debiasing methods. Preprint, arxiv:1804.06876 [cs].
  29. Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018. Learning Gender-Neutral Word Embeddings. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4847–4853, Brussels, Belgium. Association for Computational Linguistics.
  30. Yachao Zhao, Bo Wang, Yan Wang, Dongming Zhao, Ruifang He, and Yuexian Hou. 2025. Explicit vs. implicit: Investigating social bias in large language models through self-reflection. In Findings of the Association for Computational Linguistics: ACL 2025, pages 1–12, Vienna, Austria. Association for Co… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nishal-2026-caged-birds-cute-bookworms-feminine/