Elamur Research⌕
☰
home / papers / 2026 / Investigating Gender Stereotypes in Large Language Models via Social Determinants of Health

Investigating Gender Stereotypes in Large Language Models via Social Determinants of Health

Paper · 2026 · Trung Hieu Ngo, Adrien Bazoge, Solen Quiniou et al.

Summary

Clinical LLM bias benchmarks usually test one attribute, such as gender, in isolation and outside real clinical context. The authors instead ask whether models link gender to combinations of social determinants of health (SDoH). They use social-history sections from anonymized French hospital notes, annotated with 14 SDoH. Explicit gender cues are removed, for example by rewriting occupations in inclusive masculine/feminine forms based on the INSEE PCS-2020 nomenclature. Nine instruction-tuned LLMs then guess the patient's gender on a 7-point scale, where 4 means "uncertain." Any systematic drift away from 4 is treated as evidence of stereotyping.

Bias is measured with a signed, modified RMSE. Fisher's exact tests check which SDoH are associated with gendered outputs. Model predictions are also compared with those of human annotators. Occupation emerges as the strongest driver of gendered guesses. Larger models are steadier and less stereotyped, while medical fine-tuning can raise bias. Both humans and models depend on similar profession–gender stereotypes. The framework works across models and could be reused on health records in other languages.

Key points

  • Data: 1,700 social-history sections from a French University Hospital, annotated with 14 SDoH. Notes were kept only if they had at least three SDoH plus occupation information, leaving 958 notes (52% male, 48% female).
  • Models: Nine LLMs were tested.
    • General models: Llama-3.1-8b/3.3-70b-Instruct, Qwen2.5-Instruct 7B/72B, Mistral-v0.3-Instruct and Mistral-Small-24b-Instruct-2501.
    • Medical adaptations: OpenBioLLM, Med42 and HuatuoGPT.
    • Each model was run three times with top-k 100, top-p 0.9 and temperature 1.0.
  • Analysis method: Bias is scored with a sign-adjusted RMSE (negative means leaning female, positive means leaning male). Associations are tested with Fisher's exact tests at p < 0.05. Occupations are grouped into PCS-2020 categories plus a Homemakers group.
  • Main findings: Occupation is the most influential SDoH. Domain adaptation can amplify stereotypes. The nine human annotators showed profession–gender stereotypes similar to those of the models.
  • Mitigation prompt: When explicitly told to ignore gender markers, Llama refused in about 80% of cases, and Qwen answered "uncertain" every time. This suggests that prompting could help reduce bias.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Centre National de la Recherche ScientifiqueParis · France168 works · 62,369 citations🎓Nantes UniversitéNantes · France5 works · 470 citations🔬InsermParis · France6 works · 462 citations🔬Laboratoire des Sciences du Numérique de NantesNantes · France4 works · 417 citations🏥Centre Hospitalier Universitaire de NantesNantes · France2 works · 409 citations🔬Centre d'Investigation Clinique de NantesNantes · France2 works · 409 citations🎓École Centrale de NantesNantes · France2 works · 409 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSafety Research · 543 worksMedicine · 241 worksHealth Informatics · 184 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksBias, fairness & safety · 517 works

Cite

Cite as Trung Hieu Ngo, Adrien Bazoge, Solen Quiniou, Pierre-Antoine Gourraud and Emmanuel Morin. 2026. Investigating Gender Stereotypes in Large Language Models via Social Determinants of Health. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-eacl.114

@inproceedings{ngo2026investigating,
  title = {{Investigating Gender Stereotypes in Large Language Models via Social Determinants of Health}},
  author = {Trung Hieu Ngo, Adrien Bazoge, Solen Quiniou, Pierre-Antoine Gourraud and Emmanuel Morin},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.114},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.114}
}

References

32 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. AI@Meta. 2024. Llama 3 model card. find
  2. Jacy Reese Anthis, Kristian Lum, Michael Ekstrand, Avi Feller, and Chenhao Tan. 2025. The impossibility of fair llms. Association for Computational Linguistics, 63:105–120. find
  3. Solon Barocas, Kate Crawford, Aaron Shapiro, and Hanna Wallach. 2017. The problem with bias: Allocative versus representational harms in machine learning. In 9th Annual conference of the special interest group for computing, information and society, page 1. New York, NY. find
  4. Adrien Bazoge, Pacôme Constant dit Beaufils, Mohammed Hmitouch, Romain Bourcier, Emmanuel Morin, Richard Dufour, Béatrice Daille, Pierre-Antoine Gourraud, and Matilde Karakachoff. 2025. Improving social determinants of health documentation in french electronic health records using large language mod… find
  5. Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610–623. find
  6. Clément Christophe, Praveen K Kanithi, Tathagata Raha, Shadab Khan, and Marco AF Pimentel. 2024. Med42-v2: A suite of clinical llms. find
  7. Fanny Ducel, Nicolas Hiebel, Olivier Ferret, Karën Fort, and Aurélie Névéol. 2025. " women do not have heart attacks!" gender biases in automatically generated clinical cases in french. In Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics. find
  8. Fanny Ducel, Aurélie Névéol, and Karën Fort. 2024. “you’ll be a nurse, my son!” automatically assessing gender biases in autoregressive language models in french and italian. Language Resources and Evaluation, pages 1–29. find
  9. Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K. Ahmed. 2024. Bias and Fairness in Large Language Models: A Survey. Computational Linguistics, 50(3):1097– 1179.
  10. Hongzhi Huang, Defa Zhu, Banggu Wu, Yutao Zeng, Ya Wang, Qiyang Min, and Xun Zhou. 2025. Over-tokenized transformer: Vocabulary is generally worth scaling. find
  11. INSEE. 2024. PCS 2020: Professions and Socio-Professional Categories. Accessed: 2025-07-23. find
  12. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, and 1 others. 2023. Mistral 7b. Preprint, arXiv:2310.06825. find
  13. Ankur Joshi, Saket Kale, Satish Chandel, and D Kumar Pal. 2015. Likert scale: Explored and explained. British journal of applied science & technology, 7(4):396. find
  14. Matilde Karakachoff, Thomas Goronflot, Sandrine Coudol, Delphine Toublant, Adrien Bazoge, Pacôme Constant Dit Beaufils, Emilie Varey, Christophe Leux, Nicolas Mauduit, Matthieu Wargny, and 1 others. 2024. Implementing a biomedical data warehouse from blueprint to bedside in a regional french univers… find
  15. Hannah Rose Kirk, Yennie Jun, Filippo Volpin, Haider Iqbal, Elias Benussi, Frederic Dreyer, Aleksandar Shtedritski, and Yuki Asano. 2021. Bias out-of-the-box: An empirical analysis of intersectional occupational biases in popular generative language models. Advances in neural information processing… find
  16. Charaka Vinayak Kumar, Ashok Urlana, Gopichand Kanumolu, Bala Mallikarjunarao Garlapati, and Pruthwik Mishra. 2025. No llm is free from bias: A comprehensive study of bias evaluation in large language models. Preprint, arXiv:2503.11985. — [arXiv:2503.11985](https://arxiv.org/abs/2503.11985) find
  17. Tao Li, Daniel Khashabi, Tushar Khot, Ashish Sabharwal, and Vivek Srikumar. 2020. UNQOVERing stereotyping biases via underspecified questions. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3475–3489, Online. Association for Computational Linguistics.
  18. Lucía Vicente and Helena Matute. 2023. Humans inherit artificial intelligence biases. Scientific reports, 13(1):15737. find
  19. Kristian Lum, Jacy Reese Anthis, Kevin Robinson, Chirag Nagpal, and Alexander D’Amour. 2025. Bias in language models: Beyond trick tests and towards ruted evaluation. Association for Computational Linguistics, 63:137–161. find
  20. Xidong Wang, Jianquan Li, Shunian Chen, Yuxuan Zhu, Xiangbo Wu, Zhiyi Zhang, Xiaolong Xu, Junying Chen, Jie Fu, Xiang Wan, Anningzhe Gao, and Benyou Wang. 2025. Huatuo-26M, a large-scale Chinese medical QA dataset. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 3828–… find
  21. B. Merino, P. Campos, M. Santaolaya, A. Gil, J. Vega, and T. Swift. 2013. Integration of social determinants of health and equity into health strategies, programmes and activities: health equity training process in spain. Technical Report 9 (Case studies), World Health Organization, Geneva. find
  22. Moin Nadeem, Anna Bethke, and Siva Reddy. 2021. StereoSet: Measuring stereotypical bias in pretrained language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long…
  23. Jesutofunmi A. Omiye, Jenna C. Lester, Simon Spichak, Veronica Rotemberg, and Roxana Daneshjou. 2023. Large language models propagate race-based medicine. npj Digital Medicine, 6(1):1–4. Publisher: Nature Publishing Group. find
  24. Ankit Pal and Malaikannan Sankarasubbu. 2024. Openbiollms: Advancing open-source large language models for healthcare and life sciences. https://huggingface.co/aaditya/ OpenBioLLM-Llama3-70B. ↗
  25. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 23 others. 2025. Qwen2.5 technical report. Preprint, arXiv:241… find
  26. Travis Zack, Eric Lehman, Mirac Suzgun, Jorge A Rodriguez, Leo Anthony Celi, Judy Gichoya, Dan Jurafsky, Peter Szolovits, David W Bates, Raja-Elie E Abdulnour, Atul J Butte, and Emily Alsentzer. 2024. Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model eva… find
  27. Yubo Zhang, Shudi Hou, Mingyu Derek Ma, Wei Wang, Muhao Chen, and Jieyu Zhao. 2024. Climb: A benchmark of clinical bias in large language models. arXiv e-prints, pages arXiv–2407. find
  28. Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel Bowman. 2022. BBQ: A hand-built bias benchmark for question answering. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2086–2105, Dublin, Ireland. Asso…
  29. Raphael Poulain, Hamed Fayyaz, and Rahmatollah Beheshti. 2024. Bias patterns in the application of llms for clinical decision support: A comprehensive study. arXiv e-prints, pages arXiv–2404. find
  30. Ricardo Trainotti Rabonato and Lilian Berton. 2025. A systematic review of fairness in machine learning. AI and Ethics, 5(3):1943–1954. find
  31. Paolo Raile. 2024. The usefulness of ChatGPT for psychotherapists and patients. Humanities and Social Sciences Communications, 11(1):1–8. Publisher: Palgrave. find
  32. Chaofan Tao, Qian Liu, Longxu Dou, Niklas Muennighoff, Zhongwei Wan, Ping Luo, Min Lin, and Ngai Wong. 2024. Scaling laws with vocabulary: Larger models deserve larger vocabularies. Advances in Neural Information Processing Systems, 37:114147–114179.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ngo-2026-investigating-gender-stereotypes-large-language/