Elamur Research⌕
☰
home / papers / 2026 / Almost Clinical: Linguistic properties of synthetic electronic health records

Almost Clinical: Linguistic properties of synthetic electronic health records

Paper · 2026 · Serge Sharoff, John Baker, Dr David Francis Hunt et al.

Summary

Real mental health records are hard to study at scale because privacy rules restrict access and their personal narratives make re-identification more likely. This work asks whether LLM-generated records could replace them for linguistic research. The authors built patient profiles by combining age, gender, sexuality, ethnicity, diagnosis, medication, risk factors and treatment history. They then prompted two open-weight models to write four document types: initial assessments, GP correspondence, referral/handover letters and DIALOG+-style care plans. A sample of the outputs was analysed with Systemic Functional Linguistics, covering transitivity and agency, modality and connectors. Features were detected with CorpusTool and then checked by hand. The generated texts read fluently and use suitable clinical vocabulary. However, they shift register, lack clinical detail and get medication and diagnostic procedures wrong. Llama also shows ethnic bias: it brings up drug use far more often for Afro-Caribbean patients. Because the profiles are identical except for controlled variables, such biases can be measured directly. This makes the corpus useful both for linguistic research and for auditing LLMs before clinical use. The prompts and annotated corpora are released.

Key points

  • Models and corpus size: Four instruction-tuned models were piloted: DeepSeek V2 16B, Llama 3.2 3B, MediPhi 4B and Mistral v0.3 7B. Experts judged Llama and Mistral closest to clinical reality. Each produced 12,960 stories per genre, built from 2×2×3×3×6×3×5×4 parameter combinations.
  • Annotated sample: For manual linguistic annotation, 24 texts per genre per model were selected.
  • Text length: Llama writes longer and less consistent texts, with one reaching 8,098 words. Referrals are Llama's longest genre (median 973 words) but Mistral's shortest (473).
  • Care plans: Material clauses make up most of the care plans, accounting for 83% of clauses in Llama's output and 89% in Mistral's, against 61–67% in the other genres.
  • Ethnic bias in Llama: Marijuana appears 25 times in White British stories versus 1,189 times across the two Afro-Caribbean variants. For Mistral the figures are 26 versus 77. Llama shows a similar gap for cocaine (259 versus 1,857).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓King's College LondonLondon · United Kingdom42 works · 1,617 citations🎓University of ExeterExeter · United Kingdom12 works · 203 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksLiterature and Literary Theory · 519 worksMedicine · 241 worksHealth Informatics · 184 works

Cite

Cite as Serge Sharoff, John Baker, Dr David Francis Hunt and Alan Simpson. 2026. Almost Clinical: Linguistic properties of synthetic electronic health records. Proceedings of the 1st Workshop on Linguistic Analysis for Health (HeaLing 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.healing-1.10

@inproceedings{sharoff2026almost,
  title = {{Almost Clinical: Linguistic properties of synthetic electronic health records}},
  author = {Serge Sharoff, John Baker, Dr David Francis Hunt and Alan Simpson},
  booktitle = {Proceedings of the 1st Workshop on Linguistic Analysis for Health (HeaLing 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.healing-1.10},
  url = {https://doi.org/10.18653/v1/2026.healing-1.10}
}

References

19 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Karen Ball and Wendy Burn. 2021. Writing clinic letters: College guidance on improving engagement with patients. Royal College of Psychiatrists. find
  2. Jenny Chim, Adam Tsakalidis, Dimitris Gkoumas, Dana Atzil-Slonim, Yaakov Ophir, Ayah Zirikly, Philip Resnik, and Maria Liakata. 2024. Overview of the CLPsych 2024 shared task: Leveraging large language models to identify evidence of suicidality risk in online posts. In Proceedings of the 9th Worksho… find
  3. Barbara Draghi, Zhenchen Wang, Puja Myles, and Allan Tucker. 2024. Identifying and handling data bias within primary healthcare data using synthetic data generators. Heliyon, 10(2). find
  4. Abinitha Gourabathina, Walter Gerych, Eileen Pan, and Marzyeh Ghassemi. 2025. The medium is the message: How non-clinical information shapes clinical decisions in LLMs. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, pages 1805–1828. find
  5. Michael A. K. Halliday and Christian M. I. M. Matthiessen. 1999. Construing experience through meaning: a language-based approach to cognition. Cassell, London. find
  6. Michael A. K. Halliday and Christian M. I. M. Matthiessen. 2014. Introduction to Functional Grammar, 4th edition edition. Routledge, London. find
  7. Andrew Hider, Lesa Wright, and Jacob Needle. 2025. Clinical reach into the cognitive space (CRITiCS): outline conceptual framework for safe use of generative artificial intelligence in mental health decision-making. BJPsych Bulletin, page 1–6. find
  8. Nicolas Hiebel, Olivier Ferret, Karën Fort, and Aurélie Névéol. 2025. Clinical text generation: Are we there yet? Annual Review of Biomedical Data Science, 8. find
  9. Glenn Holmes, Biya Tang, Sunil Gupta, Svetha Venkatesh, Helen Christensen, and Alexis Whitton. 2025. Applications of large language models in the field of suicide prevention: Scoping review. Journal of Medical Internet Research, 27:e63126. find
  10. Ivana Kvapilíková, Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Ondřej Bojar. 2020. Unsupervised multilingual sentence embeddings for parallel corpus mining. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 255– 262, Onli… find
  11. Sunjun Kweon, Junu Kim, Jiyoun Kim, Sujeong Im, Eunbyeol Cho, Seongsu Bae, Jungwoo Oh, Gyubok Lee, Jong Hak Moon, Seng Chan You, Seungjin Baek, Chang Hoon Han, Yoon Bin Jung, Yohan Jo, and Edward Choi. 2023. Publicly shareable clinical large language model built on synthetic clinical notes. find
  12. Bolei Ma, Berk Yoztyurk, Anna-Carolina Haensch, Xinpeng Wang, Markus Herklotz, Frauke Kreuter, Barbara Plank, and Matthias Assenmacher. 2025. Algorithmic fidelity of large language models in generating synthetic German public opinions: A case study. In Proceedings of the 63rd Annual Meeting of the A…
  13. Christian MIM Matthiessen. 2015. Register in the round: registerial cartography. Functional Linguistics, 2(1):1–48. find
  14. NHS Digital. 2025. Advice and guidance for referrers and referring clinician teams. find
  15. Mick O’Donnell. 2008. Demonstration of the UAM CorpusTool for text and image annotation. In Proceedings of the ACL-08: HLT Demo Session, pages 13–16, Columbus, Ohio. Association for Computational Linguistics. find
  16. Stefan Priebe, Eoin Golden, David Kingdon, Serif Omer, Sophie Walsh, Kleomenis Katevas, Paul Mc-Crone, Sandra Eldridge, and Rose McCabe. 2017. Effective patient–clinician interaction to improve treatment outcomes for patients with psychosis: a mixed-methods design. Programme Grants for Applied Resea… find
  17. Elena Rey Velasco, Hanne Sæderup Pedersen, and Timothy Skinner. 2022. Analysis of patient cues in asynchronous health interactions: pilot study combining empathy appraisal and systemic functional linguistics. JMIR formative research, 6(12):e40058. find
  18. Yiqing Xie, Sheng Zhang, Hao Cheng, Pengfei Liu, Zelalem Gero, Cliff Wong, Tristan Naumann, Hoifung Poon, and Carolyn Rose. 2024. DocLens: Multi-aspect fine-grained evaluation for medical text generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Vol… find
  19. Kailai Yang, Tianlin Zhang, Ziyan Kuang, Qianqian Xie, and Sophia Ananiadou. 2023. Mentalllama: Interpretable mental health analysis on social media with large language models. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sharoff-2026-almost-clinical-linguistic-properties-synthetic/