Elamur Research⌕
☰
home / papers / 2026 / When Demographic Sensitivity Isn't What It Seems: Baseline-Aware Counterfactual Audits for Clinical NLP

When Demographic Sensitivity Isn't What It Seems: Baseline-Aware Counterfactual Audits for Clinical NLP

Paper · 2026 · Hyunwoo Yoo

Summary

Counterfactual fairness audits for clinical language models usually change a patient's age or sex and treat any resulting output shift as evidence of demographic bias. This paper argues that the assumption is unsafe, because modern LLMs can also change their answers when the input is altered in ways that have nothing to do with demographics. The author proposes a baseline-aware audit. Demographic edits are compared against paraphrase controls that keep both the clinical facts and the demographic attributes intact. Differences are then manually labelled as negligible, plausibly justified, or clinically worrying.

The framework was tested on 12 synthetic, triage-oriented EHR-style cases, mostly borderline, using three LLMs. Paraphrase controls often moved outputs as much as, or more than, demographic swaps did. Generated text also proved highly unstable on its own. Age edits had stronger effects than sex edits. The main lesson is that a demographic effect should count as meaningful only when it clearly rises above the model's intrinsic noise. Without that baseline, bias can be overstated or misattributed.

Key points

  • Models and outputs: gpt-4o-mini, gemini-2.5-flash and Qwen3-30B-A3B-Thinking-2507 were evaluated. Each produced a structured triage output with a recommendation, urgency, red flags, rationale and immediate action.

  • Test conditions: each case had four versions (original, age swap, sex swap, paraphrase control). Robustness checks added three extra independent paraphrase controls, plus variation in decoding temperature and prompts.

  • Noise floor: intrinsic generative instability measured 0.46–0.71 by Jaccard. Action change rates under control vs. demographic edits were:

    • gpt-4o-mini: 0.472 vs. 0.268
    • Qwen3: 0.211 vs. 0.278
    • gemini-2.5-flash: 0.444 vs. 0.727

  • Age vs. sex: urgency changed more often under age edits than under sex edits:

    • gpt-4o-mini: 0.056 vs. 0.000
    • gemini-2.5-flash: 0.333 vs. 0.154
    • Qwen3: 0.190 vs. 0.000

    Stable labels often concealed shifts in rationales and recommendations.

  • Practical rule: if the demographic effect is no larger than the control effect (Δdem ≤ Δcontrol), treat it as noise. Main limitations are the small synthetic set of 12 cases and a Jaccard metric that cannot separate stylistic variation from clinical drift.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Drexel UniversityPhiladelphia · United States6 works · 1,443 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksSocial Sciences · 1,954 worksSafety Research · 626 works

Cite

Cite as Hyunwoo Yoo. 2026. When Demographic Sensitivity Isn't What It Seems: Baseline-Aware Counterfactual Audits for Clinical NLP. BioNLP 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.bionlp-1.14

@inproceedings{yoo2026when,
  title = {{When Demographic Sensitivity Isn't What It Seems: Baseline-Aware Counterfactual Audits for Clinical NLP}},
  author = {Hyunwoo Yoo},
  booktitle = {BioNLP 2026},
  year = {2026},
  doi = {10.18653/v1/2026.bionlp-1.14},
  url = {https://doi.org/10.18653/v1/2026.bionlp-1.14}
}

References

25 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Amber M. Angell, Yongqiu Li, Jiang Bian, Camille Parchment, Larry Yin, Srikar Chamala, Hesamedin Hakimjavadi, Lindsay Thompson, and Yi Guo. 2025. Algorithmic fairness in machine learning prediction of autism using electronic health records. Studies in Health Technology and Informatics. find
  2. Mingchen Li, Zaifu Zhan, Han Yang, Yongkang Xiao, Huixue Zhou, Jiatan Huang, and Rui Zhang. 2025. Benchmarking retrieval-augmented large language models in biomedical nlp: Application, robustness, and self-awareness. Science Advances. find
  3. Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A survey on bias and fairness in machine learning. ACM Computing Surveys.
  4. OpenAI. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. Version 6; last revised 4 Mar 2024.
  5. Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020. Beyond accuracy: Behavioral testing of NLP models with CheckList. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4902–4912, Online. Association for Computational Linguistic… find
  6. Varuni Sarwal, Aditya Pimplaskar, Misty Richards, Kunmi Sobowale, Jeffrey N. Chiang, and Loes Olde Loohuis. 2025. Early prediction and fairness evaluation of perinatal depression using ehr: A study of 18,000+ pregnancies. medRxiv. find
  7. Miguel Contreras, Parisa Rashidi, and Sumit Kapoor. 2025. 353: Large language models for mortality prediction using structured ehr and unstructured clinical notes. Critical Care Medicine. find
  8. Ravi Shankar, Janani Kannan, Yih Hng Tan, and Qian Xu. 2025. Natural language processing techniques to detect delirium in hospitalized patients from clinical notes: a systematic review. npj Digital Medicine. find
  9. Amanda Coston, Alan Mishler, Edward H. Kennedy, and Alexandra Chouldechova. 2020. Counterfactual risk assessments, evaluation, and fairness. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. find
  10. Zhecheng Sheng. 2022. Nlp system for mining social determinant of health from clinical notes and its fairness evaluations. 2022 IEEE 10th International Conference on Healthcare Informatics (ICHI). find
  11. Gemini Team, Google. 2023. Gemini: A family of highly capable multimodal models. arXiv preprint arXiv:2312.11805.
  12. Abhishek Shivanna, Adam Spannaus, Jordan Tschida, John Gounley, Patrycja Krawczuk, and Heidi Hanson. 2026. Mitigating algorithmic bias in cancer site classification models. JCO Clinical Cancer Informatics. find
  13. Dong-Gil Ko. 2026. Advancing fairness in clinical ai decision-making through a sociotechnical threshold bias audit. Proceedings of the 19th International Joint Conference on Biomedical Engineering Systems and Technologies. find
  14. Katherine Krieger, Irbaz Hameed, Giorgio Quer, Charles Mack, Marco Savic, Polina Mantaj, Aina Hirofuji, Alexander Gregg, Giovanni Soletti, Camilla S Rossi, Mohamed Rahouma, and Mario Gaudino. 2025. Generative pre-trained transformer reinforces historical gender bias in diagnosing women’s cardiovascu… find
  15. Ida Surakka, Brooke N Wolford, Scott C Ritchie, Whitney E Hornsby, Nadia R Sutton, Maiken Elvenstad Gabrielsen, Anne Heidi Skogholt, Laurent Thomas, Michael Inouye, Kristian Hveem, and Cristen J Willer. 2023. Sex-specific survival bias and interaction modeling in coronary artery disease risk predict… find
  16. Helena Kudiabor. 2026. Sex bias in autism drops as age at diagnosis rises. find
  17. Simon Suster, Timothy Baldwin, and Karin Verspoor. 2023. Promoting fairness in classification of quality of medical evidence. In Proceedings of the 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks, pages 413–426, Toronto, Canada. Association for Computational Linguisti… find
  18. Matt J. Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. 2017. Counterfactual fairness. In Advances in Neural Information Processing Systems 30 (NeurIPS 2017). find
  19. Dandan Wang and Shiqing Zhang. 2024. Large language models in medical and healthcare fields: applications, advances, and challenges. Artificial Intelligence Review. find
  20. Xuezhi Wang, Haohan Wang, and Diyi Yang. 2022. Measure and improve robustness in NLP models: A survey. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4569–4586, Seattle, United States. Associati…
  21. Yifan Wang, Laura Sikstrom, Robert Xiao, Zoe Findlay, Juveria Zaheer, Sean L. Hill, and Marta M. Maslej. 2026. Fairness analysis of machine learning predictions of aggression in acute psychiatric care. npj Mental Health Research. find
  22. Site Xu and Mu Sun. 2025. Natural language processing (nlp): Identifying linguistic gender bias in electronic medical records (emrs). Journal of Patient Experience. find
  23. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  24. Kunyu Yu, Rui Yang, Jingchi Liao, Siqi Li, Huitao Li, Irene Li, Yifan Peng, Rishikesan Kamaleswaran, and Nan Liu. 2025. Benchmarking foundation models with multimodal public electronic health records. IEEE Journal of Biomedical and Health Informatics. find
  25. Yubo Zhang, Shudi Hou, Mingyu Derek Ma, Wei Wang, Muhao Chen, and Jieyu Zhao. 2024. Climb: A benchmark of clinical bias in large language models. Preprint, arXiv:2407.05250. — [arXiv:2407.05250](https://arxiv.org/abs/2407.05250) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yoo-2026-when-demographic-sensitivity-isn-t/