Elamur Research⌕
☰
home / papers / 2026 / The Point of View of a Sentiment: Towards Clinician Bias Detection in Psychiatric Notes

The Point of View of a Sentiment: Towards Clinician Bias Detection in Psychiatric Notes

Paper · 2026 · Alissa A. Valentine, Lauren Lepow, Lili Chan et al.

Summary

Clinical notes can carry stigmatizing descriptions of patients. These descriptions can damage patients' trust when they read their own records and can bias later clinicians. Earlier NLP work on harmful clinical language relied on consensus labels that reflect only the institution's or physicians' view. This study asks whether physicians and non-physicians judge the sentiment of psychiatric patient descriptions differently, and whether LLMs can be steered toward either viewpoint. The authors built a small psychiatric lexicon of patient descriptors and pulled 39 note sentences containing those terms. Ten physicians and ten non-physicians then labeled each sentence as negative, neutral, or positive. Physicians answered as the note's author, and non-physicians answered as the patient being described. GPT-3.5, Llama-3.1, and Mistral were tested in zero-shot and few-shot in-context learning setups to reproduce each group's labels. With ICL, GPT-3.5 matched physicians most closely, while Mistral matched non-physicians best. The work argues that annotator subjectivity should be modeled explicitly when detecting clinician bias in psychiatric text.

Key points

  • Lexicon construction: a psychiatrist and a fairness-focused computational scientist started from 54 candidate words. They drew these from prior work by Sun et al. (2022), along with "never words" that their medical center's Institute for Health Equity Research had compiled. The list was then narrowed to descriptors that apply only to patients, are typical in psychiatry, and carry subjective meaning, such as "compliant," "malingering," "agitated," and "cooperative."
  • Dataset: 39 sentences, each longer than 30 characters, with protected health information masked by hand. The annotations are released on GitHub.
  • Annotators: 10 physicians with clinical note experience and 10 non-physicians with no medical training. Each group received a different perspective-taking instruction.
  • Models: GPT-3.5, Llama-3.1, and Mistral, tested with zero-shot and few-shot ICL prompting.
  • Main finding: with ICL, GPT-3.5 aligned best with the physician perspective and Mistral aligned best with the non-physician perspective. This suggests prompting can shift a model toward a particular reader's point of view.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of CopenhagenCopenhagen · Denmark116 works · 15,292 citations🎓Icahn School of Medicine at Mount SinaiNew York · United States2 works · 190 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksSocial Sciences · 1,954 worksPsychology · 1,649 worksSafety Research · 626 worksSocial Psychology · 330 works

🧭 Topics 1Research subjects and languages the work is about.Sentiment & emotion · 1,035 works

Cite

Cite as Alissa A. Valentine, Lauren Lepow, Lili Chan, Alexander Charney and Isotta Landi. 2026. The Point of View of a Sentiment: Towards Clinician Bias Detection in Psychiatric Notes. Proceedings of the Second Workshop of Identity Aware AI.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2dxaipemahpd

@inproceedings{valentine2026point,
  title = {{The Point of View of a Sentiment: Towards Clinician Bias Detection in Psychiatric Notes}},
  author = {Alissa A. Valentine, Lauren Lepow, Lili Chan, Alexander Charney and Isotta Landi},
  booktitle = {Proceedings of the Second Workshop of Identity Aware AI},
  year = {2026},
  doi = {10.63317/2dxaipemahpd},
  url = {https://doi.org/10.63317/2dxaipemahpd}
}

References

32 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Valerio Basile. 2020. It’s the end of the gold standard as we know it. on the impact of pre-aggregation on the evaluation of highly subjective tasks. In CEUR workshop proceedings, volume 2776, pages 31–40. CEUR-WS. find
  2. Data Availability Isabel Bilotta, Scott Tonidandel, Winston R Liaw, Eden King, Diana N Carvajal, Ayana Taylor, Julie Thamby, Yang Xiang, Cui Tao, and Michael Hansen. 2024. Examining linguistic differences in electronic health records for diverse patients The dataset referenced in this paper with ann… find
  3. André Bittar, Sumithra Velupillai, Angus Roberts, and Rina Dutta. 2021. Using general-purpose sentiment lexicons for suicide risk assessment in electronic health records: Corpus-based analysis. JMIR medical informatics, 9(4):e22397. find
  4. Eben Holderness, Philip Cawkwell, Kirsten Bolton, James Pustejovsky, and Mei-Hua Hall. 2019. Distinguishing clinical sentiment: The importance of domain adaptation in psychiatric patient health records. arXiv preprint arXiv:1904.03225. find
  5. Sean Boley, Abbey Sidebottom, Marc Vacquier, David Watson, Bailey Van Eyll, Sara Friedman, and Scott Friedman. 2024. Racial differences in stigmatizing and positive language in emergency medicine notes. Journal of Racial and Ethnic Health Disparities, pages 1–11. find
  6. Sune Holm, Eike Petersen, Melanie Ganz, and Aasa Feragen. 2023. Bias in context: What to do when complete bias removal is not an option. Proceedings of the National Academy of Sciences, 120(23):e2304710120. find
  7. Kerstin Denecke and Daniel Reichenpfader. 2023. Sentiment analysis of clinical narratives: a scoping review. Journal of Biomedical Informatics, 140:104336. find
  8. Adam O Horvath and B Dianne Symonds. 1991. Relation between working alliance and outcome in psychotherapy: A meta-analysis. Journal of counseling psychology, 38(2):139. find
  9. Wenhan Dong, Yuemeng Zhao, Zhen Sun, Yule Liu, Zifan Peng, Jingyi Zheng, Zongmin Zhang, Ziyi Zhang, Jun Wu, Ruiming Wang, Shengmin Xu, Xinyi Huang, and Xinlei He. 2025. Humanizing llms: A survey of psychological measurements with tools, datasets, and human-agent applications. find
  10. Marc S Karver, Alessandro S De Nadai, Maureen Monahan, and Stephen R Shirk. 2018. Meta-analysis of the prospective relation between alliance and outcome in child and adolescent psychotherapy. Psychotherapy, 55(4):341. find
  11. Patrick JA Kelly, Andrew M Snyder, Madina Agénor, Cassandra R Navalta, Chelsea Misquith, Josiah D Rich, and Jaclyn MW Hughto. 2023. A scoping review of methodological approaches to detect bias in the electronic health record. Stigma and Health. find
  12. Leonor Fernández, Alan Fossa, Zhiyong Dong, Tom Delbanco, Joann Elmore, Patricia Fitzgerald, Kendall Harcourt, Jocelyn Perez, Jan Walker, and Catherine DesRoches. 2021. Words matter: what do patients find judgmental or offensive in outpatient notes? Journal of general internal medicine, pages 1–8. find
  13. Thomas H McCoy, Victor M Castro, Andrew Cagan, Ashlee M Roberson, Isaac S Kohane, and Roy H Perlis. 2015. Sentiment measured in hospital discharge notes is associated with readmission and mortality risk: an electronic health record study. PloS one, 10(8):e0136341. find
  14. Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378. find
  15. Tabor E Flickinger, Somnath Saha, Richard D Moore, and Mary C Beach. 2013. Higher quality communication and relationships are associated with improved patient engagement in hiv care. JAIDS Journal of Acquired Immune Deficiency Syndromes, 63(3):362–366. find
  16. Mary L. McHugh. 2012. Interrater reliability: the kappa statistic. BIOCHEMIA MEDICA, 22(3):276–282. find
  17. Roberto Navigli, Simone Conia, and Björn Ross. 2023. Biases in large language models: origins, inventory, and discussion. ACM Journal of Data and Information Quality, 15(2):1–21. find
  18. Christoph Flückiger, Aaron C Del Re, Bruce E Wampold, and Adam O Horvath. 2018. The alliance in adult psychotherapy: A meta-analytic synthesis. Psychotherapy, 55(4):316. find
  19. Mahmud Omar, Shelly Soffer, Reem Agbareia, Nicola Luigi Bragazzi, Donald U Apakama, Carol R Horowitz, Alexander W Charney, Robert Freeman, Benjamin Kummer, and Benjamin S Glicksberg. 2025. Sociodemographic biases in medical decision making by large language models. Nature Medicine, pages 1–9. find
  20. Kai He, Rui Mao, Qika Lin, Yucheng Ruan, Xiang Lan, Mengling Feng, and Erik Cambria. 2023. A survey of large language models for healthcare: from data, technology, and applications to accountability and ethics. arXiv preprint arXiv:2310.05694.
  21. Anna P Goddu, Katie J O’Conor, Sophie Lanzkron, Mustapha O Saheed, Somnath Saha, Monica E Peek, Carlton Haywood, and Mary Catherine Gracie Himmelstein, David Bates, and Li Zhou. 2022. Examination of stigmatizing language Beach. 2018. Do words matter? stigmatizing language and the transmission of bia… find
  22. Yunze Xiao, Lynnette Hui Xian Ng, Jiarui Liu, and Mona T. Diab. 2025. Humanizing machines: Rethinking LLM anthropomorphism through a multi-level framework of design. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 3331–3350, Suzhou, China. Association… find
  23. Ridam Pal, Hardik Garg, Shashwat Patel, and Tavpritesh Sethi. 2023. Bias amplification in intersectional subpopulations for clinical phenotyping by large language models. medRxiv, page 2023.03. 22.23287585. find
  24. Travis Zack, Eric Lehman, Mirac Suzgun, Jorge A Rodriguez, Leo Anthony Celi, Judy Gichoya, Dan Jurafsky, Peter Szolovits, David W Bates, and Raja-Elie E Abdulnour. 2024. Assessing the potential of gpt-4 to perpetuate racial and gender biases in health care: a model evaluation study. The Lancet Digit… find
  25. Jenny Park, Somnath Saha, Brant Chee, Janiece Taylor, and Mary Catherine Beach. 2021. Physician use of stigmatizing language in patient medical records. JAMA Network Open, 4(7):e2117052–e2117052. find
  26. Reid Pryzant, Richard Diehl Martinez, Nathan Dass, Sadao Kurohashi, Dan Jurafsky, and Diyi Yang. Automatically neutralizing subjective bias in text. In Proceedings of the aaai conference on artificial intelligence, volume 34, pages 480–489. find
  27. Haoran Zhang, Amy X Lu, Mohamed Abdalla, Matthew McDermott, and Marzyeh Ghassemi. 2020. Hurtful words: quantifying biases in clinical contextual word embeddings. In proceedings of the ACM Conference on Health, Inference, and Learning, pages 110–120. find
  28. Jorge A Rodriguez, Cheryl R Clark, and David W Bates. 2020. Digital health equity as a necessity in the 21st century cures act era. Jama, 323(23):2381–2382. find
  29. Jennifer M Silva, T Elizabeth Durden, and Annemarie Hirsch. 2023. Erasing inequality: Examining discrepancies between electronic health records and patient narratives to uncover perceived stigma and dismissal in clinical encounters. Social Science Medicine, 323:115837. find
  30. Malliga Subramanian, Veerappam-palayam Easwaramoorthy Sathiskumar, G Deepalakshmi, Jaehyuk Cho, and G Manikandan. 2023. A survey on hate speech detection and sentiment analysis using machine learning and deep learning models. Alexandria Engineering Journal, 80:110–121. find
  31. M. Sun, T. Oliwa, M. E. Peek, and E. L. Tung. 2022. Negative patient descriptors: Documenting racial bias in the electronic health record. Health Aff (Millwood), 41(2):203–211. Sun, Michael Oliwa, Tomasz Peek, Monica E Tung, Elizabeth L eng K23 HL145090/HL/NHLBI NIH HHS/ L30 HL148782/HL/NHLBI NIH HH… — [doi:10.1377/hlthaff.2021.01423](https://doi.org/10.1377/hlthaff.2021.01423) find
  32. Gary E Weissman, Lyle H Ungar, Michael O Harhay, Katherine R Courtright, and Scott D Halpern. 2019. Construct validity of six sentiment analysis methods in the text of encounter notes of patients with critical illness. Journal of biomedical informatics, 89:114–121. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/valentine-2026-point-view-sentiment-clinician-bias/