Elamur Research⌕
☰
home / papers / 2026 / Empathy Applicability Modeling for General Health Queries

Empathy Applicability Modeling for General Health Queries

Paper · 2026 · Shan Randhawa, Agha Ali Raza, Kentaro Toyama et al.

Summary

LLMs are entering clinical messaging, but they often fail to judge whether a patient's message calls for empathy at all. Earlier NLP work mostly scores empathy after the fact in responses, or assumes support is always needed. This paper asks the question before any reply is written. It proposes the Empathy Applicability Framework (EAF), which labels single-turn, asynchronous health queries along two dimensions, Emotional Reactions and Interpretations, as Applicable or Not Applicable. The dimensions are adapted from EPITOME, and the cues are tied to Patient-Centered Communication theory. The authors built the cue set through thematic coding of HealthCareMagic and iCliniq queries. They then annotated a benchmark with two human annotators and GPT-4o and found substantial human–GPT agreement on cases where the humans agreed. RoBERTa-base classifiers trained on human or GPT-only labels beat heuristic and zero-shot LLM baselines, which suggests the labels are learnable. Error analysis points to implicit distress, unclear clinical severity and contextual hardship as hard cases. The work lays groundwork for empathy-aware response planning in asynchronous healthcare.

Key points

  • Problem: Clinicians reportedly miss 70–90% of empathic opportunities. EAF predicts empathy needs before a response is generated. If both dimensions are labeled Not Applicable, the patient mainly needs information.
  • Data sampling: 9,500 queries were drawn evenly from HealthCareMagic and iCliniq (Li et al., 2023), 4,750 from each. Of these, 1,500 were dual-annotated by humans and GPT-4o, and 8,000 were labeled by GPT-4o only.
  • Released benchmark: The final benchmark contains 1,296 queries. It is released with annotation scripts, training code and model outputs on GitHub (shanmrandhawa/Empathy-Applicability-Framework).
  • Framework design: EAF drops EPITOME's Explorations dimension because it targets multi-turn dialogue. The cue taxonomy was derived from 300 coded queries.
  • Limitations: Only two non-clinician annotators were used, and GPT-4o was the only LLM tested. Humans picked one subcategory per label while GPT-4o picked several. The classifier is limited to RoBERTa-base, and labels are binary rather than graded.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of MichiganAnn Arbor · United States70 works · 13,693 citations🎓Lahore University of Management SciencesLahore · Pakistan3 works · 61 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksMedicine · 241 worksHealth Informatics · 184 worksPsychiatry and Mental health · 9 works

Cite

Cite as Shan Randhawa, Agha Ali Raza, Kentaro Toyama, Julie Hui and Mustafa Naseem. 2026. Empathy Applicability Modeling for General Health Queries. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1983

@inproceedings{randhawa2026empathy,
  title = {{Empathy Applicability Modeling for General Health Queries}},
  author = {Shan Randhawa, Agha Ali Raza, Kentaro Toyama, Julie Hui and Mustafa Naseem},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1983},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1983}
}

References

43 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Iqra Ali, Jesse Atuhurra, Hidetaka Kamigaito, and Taro Watanabe. 2025. Hlu: Human vs llm generated text detection dataset for urdu at multiple granularities. In Proceedings of the 31st International Conference on Computational Linguistics, pages 3495–3510. find
  2. Maria Antoniak, Aakanksha Naik, Carla S Alvarado, Lucy Lu Wang, and Irene Y Chen. 2024. Nlp for maternal healthcare: Perspectives and guiding principles in the age of llms. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 1446–1463. find
  3. Monica Oliveira Bernardo, Dario Cecílio-Fernandes, Patrício Costa, Thelma A Quince, Manuel João Costa, and Marco Antonio Carvalho-Filho. 2018. Physicians’ self-assessed empathy levels do not correlate with patients’ assessments. PloS one, 13(5):e0198488. find
  4. Yong Cao, Li Zhou, Seolhwa Lee, Laura Cabello Piqueras, Min Chen, and Daniel Hershcovich. 2023. Assessing cross-cultural alignment between chatgpt and human societies: An empirical study. In Proceedings of the first workshop on cross-cultural considerations in NLP (C3NLP), pages 53–67.
  5. Yibo Chai, Fengyang Wu, Rui Sun, Zhongliang Zhang, Jie Bao, Runxin Ma, Qizhou Peng, Danqin Wu, Yexing Wan, and Keyu Li. 2019. Predicting future alleviation of mental illness in social media: an empathy-based social network perspective. In 2019 IEEE Intl Conf on Parallel & Distributed Processing with… find
  6. Xinhao Chen, Chong Yang, Man Lan, Li Cai, Yang Chen, Tu Hu, Xinlin Zhuang, and Aimin Zhou. 2024. Cause-aware empathetic response generation via chain-of-thought fine-tuning. arXiv preprint arXiv:2408.11599. — [arXiv:2408.11599](https://arxiv.org/abs/2408.11599) find
  7. Aida Mostafazadeh Davani, Mark Díaz, and Vinodkumar Prabhakaran. 2022. Dealing with disagreements: Looking beyond the majority vote in subjective annotations. Transactions of the Association for Computational Linguistics, 10:92–110.
  8. Joost Dekker, Jeanet Karchoud, Annemarie MJ Braamse, Hilde Buiting, Inge RHM Konings, Myra E van Linde, Claudia SEW Schuurhuizen, Mirjam AG Sprangers, Aartjan TF Beekman, and Henk MW Verheul. 2020. Clinical management of emotions in patients with cancer: introducing the approach “emotional support a… find
  9. Quentin Eichbaum, Charles-Antoine Barbeau-Meunier, Mary White, Revathi Ravi, Elizabeth Grant, Helen Riess, and Alan Bleakley. 2023. Empathy across cultures–one size does not fit all: from the ego-logical to the eco-logical of relational empathy. Advances in Health Sciences Education, 28(2):643–657. find
  10. Ronald M Epstein and Richard L Street Jr. 2007. Patient-centered communication in cancer care: promoting healing and reducing suffering. find
  11. Jun Gao, Yuhan Liu, Haolin Deng, Wei Wang, Yu Cao, Jiachen Du, and Ruifeng Xu. 2021. Improving empathetic response generation by recognizing emotion cause in conversations. In Findings of the association for computational linguistics: EMNLP 2021, pages 807–819. find
  12. Xiang Gao and Kamalika Das. 2024. Customizing language model responses with contrastive in-context learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18039– 18046. find
  13. Mitchell L Gordon, Kaitlyn Zhou, Kayur Patel, Tatsunori Hashimoto, and Michael S Bernstein. 2021. The disagreement deconvolution: Bringing machine learning performance metrics in line with reality. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–14. find
  14. Clarissa Guidi and Chiara Traversa. 2021. Empathy in patient care: from ‘clinical empathy’to ‘empathic concern’. Medicine, Health Care and Philosophy, 24:573–585. find
  15. Lianne Hermans, Tim Olde Hartman, and Patrick W Dielissen. 2018. Differences between gp perception of delivered empathy and patient-perceived empathy: a cross-sectional study in primary care. British Journal of General Practice. find
  16. Hinke Hoffstädt, Jacqueline Stouthard, Maartje C Meijers, Janine Westendorp, Inge Henselmans, Peter Spreeuwenberg, Paul de Jong, Sandra van Dulmen, and Liesbeth M van Vliet. 2020. Patients’ and clinicians’ perceptions of clinician-expressed empathy in advanced cancer consultations and associations w… find
  17. Mahshid Hosseini and Cornelia Caragea. 2021. It takes two to empathize: One to seek and one to provide. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 13018–13026. find
  18. Ian Hsu, Somnath Saha, Phillip Todd Korthuis, Victoria Sharp, Jonathon Cohn, Richard D Moore, and Mary Catherine Beach. 2012. Providing support to patients in emotional encounters: a new perspective on missed empathic opportunities. Patient education and counseling, 88(3):436–442. find
  19. Yan Hu, Qingyu Chen, Jingcheng Du, Xueqing Peng, Vipina Kuttichi Keloth, Xu Zuo, Yujia Zhou, Zehan Li, Xiaoqian Jiang, Zhiyong Lu, et al. 2024. Improving large language models for clinical named entity recognition via prompt engineering. Journal of the American Medical Informatics Association, 31(9)… find
  20. Liting Jiang, Di Wu, Bohui Mao, Yanbing Li, and Wushour Slamu. 2023. Empathy intent drives empathy detection. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6279–6290. find
  21. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  22. Rebecca L Johnson, Giada Pistilli, Natalia Menédez-González, Leslye Denisse Dias Duran, Enrico Panai, Julija Kalpokiene, and Donald Jay Bertulfo. 2022. The ghost in the machine has an american accent: value conflict in gpt-3. arXiv preprint arXiv:2203.07785. find
  23. Lauren A McCormack, Katherine Treiman, Douglas Rupert, Pamela Williams-Piehota, Eric Nadler, Neeraj K Arora, William Lawrence, and Richard L Street Jr. 2011. Measuring patient-centered communication in cancer care: a literature review and the development of a systematic approach. Social science & me… find
  24. Hannah Rose Kirk, Bertie Vidgen, Paul Röttger, and Scott A Hale. 2024. The benefits, risks and bounds of personalizing the alignment of large language models to individuals. Nature Machine Intelligence, 6(4):383–392. find
  25. Erica Koranteng, Arya Rao, Efren Flores, Michael Lev, Adam Landman, Keith Dreyer, and Marc Succi. 2023. Empathy and equity: Key considerations for large language model adoption in health care. JMIR Medical Education, 9:e51199. find
  26. Aakriti Kumar, Nalin Poungpeth, Diyi Yang, Erina Farrell, Bruce L Lambert, and Matthew Groh. 2026. When large language models are reliable for judging empathic communication. Nature Machine Intelligence, pages 1–13. find
  27. Allison Lahnala, Charles Welch, David Jurgens, and Lucie Flek. 2022. A critical reflection and forward perspective on empathy and natural language processing. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 2139–2158. find
  28. Allison Claire Lahnala, Béla Neuendorf, Alexander Thomin, Charles Welch, Tina Stibane, and Lucie Flek. 2024. Appraisal framework for clinical empathy: A novel application to breaking bad news conversations. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Langu… find
  29. Gyeongeun Lee, Zhu Wang, Sathya N Ravi, and Natalie Parde. 2025. From heart to words: Generating empathetic responses via integrated figurative language and semantic context signals. In Findings of the Association for Computational Linguistics: ACL 2025, pages 4490–4502. find
  30. Yunxiang Li, Zihan Li, Kai Zhang, Ruilong Dan, Steve Jiang, and You Zhang. 2023. Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge. Cureus, 15(6).
  31. Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, and Minlie Huang. 2021. Towards emotional support dialog systems. In Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural lan…
  32. Quinn McNemar. 1947. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153–157. find
  33. Negar Mokhberian, Myrl Marmarelis, Frederic Hopp, Valerio Basile, Fred Morstatter, and Kristina Lerman. 2024. Capturing perspectives of crowdsourced annotators in subjective learning tasks. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Lingu… find
  34. Diane S Morse, Elizabeth A Edwardsen, and Howard S Gordon. 2008. Missed opportunities for interval empathy in lung cancer communication. Archives of internal medicine, 168(17):1853–1858. find
  35. Nature Machine Intelligence. 2024. Empathic ai can’t get under the skin. Nature Machine Intelligence, 6:495. find
  36. Joanne K Olson. 1995. Relationships between nurse-expressed empathy, patient-perceived empathy and patient distress. Image: The Journal of Nursing Scholarship, 27(4):317–322. find
  37. Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards empathetic open-domain conversation models: A new benchmark and dataset. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 5370–5381. find
  38. Torkel Richert, Björn Johnson, and Bengt Svensson. 2018. Being a parent to an adult child with drug problems: Negative impacts on life situation, health, and emotions. Journal of Family Issues, 39(8):2311– 2335. find
  39. Sahand Sabour, Chujie Zheng, and Minlie Huang. 2022. Cem: Commonsense-aware empathetic response generation. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 11229–11237. find
  40. Sermo Team. 2025. Can physicians and patients trust AI doctor apps like ChatGPT? https://www.sermo.com/resources/ai-doctor-app/. Blog post; accessed 22 July 2025. ↗
  41. Ashish Sharma, Adam Miner, David Atkins, and Tim Althoff. 2020. A computational approach to understanding empathy expressed in text-based mental health support. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5263–5276.
  42. Lanrui Wang, Jiangnan Li, Chenxu Yang, Zheng Lin, Hongyin Tang, Huan Liu, Yanan Cao, Jingang Wang, and Weiping Wang. 2025. Sibyl: Empowering empathetic dialogue generation in large language models via sensible and visionary commonsense inference. In Proceedings of the 31st International Conference o…
  43. Joshua D Wondra and Phoebe C Ellsworth. 2015. An appraisal theory of empathy and other vicarious emotional experiences. Psychological review, 122(3):411. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/randhawa-2026-empathy-applicability-modeling-general-health/