Elamur Research⌕
☰
home / papers / 2026 / Medical Summarization in Practice: Design, Deployment, and Analysis of a Clinical Summarization System for a German Hospital

Medical Summarization in Practice: Design, Deployment, and Analysis of a Clinical Summarization System for a German Hospital

Paper · 2026 · Moiz Rauf and Sean Papay

Summary

Doctors must condense a patient's many electronic health records (EHRs) into a discharge summary. This is slow and adds to clinician workload, and most research on automating it targets English. This industry-track paper describes an LLM assistant piloted in a German hospital. It drafts German-language discharge summaries, which physicians then check and revise. GDPR rules and local hardware exclude cloud services and fine-tuning, so the system runs Mistral Small 3.2 on-premise in a zero-shot setup. As a result, the main design lever is how FHIR patient data is presented to the model. Events are bucketed into fixed time windows and, inside each window, grouped by clinical category. The structured record is then serialized as flat text, JSON, Markdown, or XML-style tags. The authors run three experiments:

  • how the encoding choice affects clinician-rated quality,
  • how automatic metrics relate to expert judgments,
  • how stable outputs are across repeated generations.

Automatic metrics track clinician ratings only moderately. The study shows that local LLM integration into hospital workflows is feasible and offers practical guidance for non-English clinical settings.

Key points

  • Model and deployment: Mistral Small 3.2 serves as the backbone; it is a decoder-only LLM that handles multiple languages and offers a 128K context window. It runs zero-shot, fully inside the hospital under GDPR, and follows a strict human-in-the-loop workflow.
  • Input structuring: Each record is a sequence of (category, value, timestamp) events. A "temporal-hierarchical hybrid" combines time windowing with grouping into vitals, labs, meds, procedures, diagnoses, and notes. Four encodings are compared: Flat Text (baseline), JSON, Markdown, and XML tags.
  • Human evaluation: Three clinicians scored summaries on a 5-point Likert scale, converted from German school grades. The axes were Logical Clarity, Completeness, Medical Precision, and Readability. Patients with short and longer hospital stays were included.
  • Metrics vs. experts: Automatic metrics correspond only moderately to clinician judgments. The regression analyses explain limited variance (R² ≈ 0.33).
  • Limitations: The study uses one model, data from one hospital, and a small evaluation volume. It includes no robustness tests on degraded EHR data. Under the EU AI Act, such systems count as high-risk applications.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of BambergBamberg · Germany5 works · 65 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksMedicine · 241 worksHealth Informatics · 184 worksHealth Professions · 69 worksHealth Information Management · 50 works

🧭 Topics 1Research subjects and languages the work is about.Summarization · 232 works

Cite

Cite as Moiz Rauf and Sean Papay. 2026. Medical Summarization in Practice: Design, Deployment, and Analysis of a Clinical Summarization System for a German Hospital. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-industry.34

@inproceedings{rauf2026medical,
  title = {{Medical Summarization in Practice: Design, Deployment, and Analysis of a Clinical Summarization System for a German Hospital}},
  author = {Moiz Rauf and Sean Papay},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-industry.34},
  url = {https://doi.org/10.18653/v1/2026.eacl-industry.34}
}

References

42 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Griffin Adams, Emily Alsentzer, Mert Ketenci, Jason Zucker, and Noémie Elhadad. 2021. What’s in a summary? laying the groundwork for advances in hospital-course summarization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4794–4811, Online. Association for Computational Linguistics. find
  2. Rana Alissa, Jennifer A Hipp, and Kendall Webb. 2022. Saving time for patient care by optimizing physician note templates: a pilot study. Frontiers in Digital Health, 3:772356. ↗
  3. Brian G Arndt, John W Beasley, Michelle D Watkinson, Jonathan L Temte, Wen-Jan Tuan, Christine A Sinsky, and Valerie J Gilchrist. 2017. Tethered to the ehr: primary care physician workload assessment using ehr event log data and time-motion observations. The Annals of Family Medicine, 15(5):419–426. ↗
  4. Lydie Bednarczyk, Daniel Reichenpfader, Christophe Gaudet-Blavignac, Amon Kenna Ette, Jamil Zaghir, Yuanyuan Zheng, Adel Bensahla, Mina Bjelogrlic, and Christian Lovis. 2025. Scientific evidence for clinical text summarization using large language models: Scoping review. J Med Internet Res, 27:e68998. ↗
  5. Kristine Ann M. Carandang, Jasper Meynard Arana, Ethan Robert Casin, Christopher Monterola, Daniel Stanley Tan, Jesus Felix B. Valenzuela, and Christian Alis. 2025. Are LLMs reliable? an exploration of the reliability of large language models in clinical note generation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track), pages 1413–1422, Vienna, Austria. Association for Computational Linguistics. https://artificialintelligenceact.eu/ ↗
  6. Tanise Ceron, Neele Falk, Ana Barić, Dmitry Nikolaev, and Sebastian Padó. 2024. Beyond prompt brittleness: Evaluating the reliability and consistency of political worldviews in LLMs. Transactions of the Association for Computational Linguistics, 12:1378– 1400. ↗
  7. Yu-Neng Chuang, Ruixiang Tang, Xiaoqian ng, and Xia Hu. 2024. Spec: a soft prompt-based calibration on performance variability of large language model in clinical notes summarization. Journal of biomedical informatics, 151:104606. find
  8. Daniel Deutsch, Rotem Dror, and Dan Roth. 2021. A statistical analysis of summarization evaluation metrics using resampling methods. Transactions of the Association for Computational Linguistics, 9:1132– 1146. ↗
  9. Ori Ernst, Ori Shapira, Ido Dagan, and Ran Levy. 2023. Re-examining summarization evaluation across multiple quality criteria. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13829–13838, Singapore. Association for Computational Linguistics. find
  10. Matthias Ganzinger, Nicola Kunz, Pascal Fuchs, Cornelia K Lyu, Martin Loos, Martin Dugas, and Thomas M Pausch. 2025. Automated generation of discharge summaries: leveraging large language models with clinical data. Scientific Reports, 15(1):1– 13. ↗
  11. V Hartman and T R Campion. 2022. A Day-to-Day approach for automating the hospital course section of the discharge summary. AMIA Jt Summits Transl Sci Proc, 2022:216–225. find
  12. Berk Atil, Alexa Chittams, Liseng Fu, Ferhan Ture, Lixinyu Xu, and Breck Baldwin. 2024. Llm stability: A detailed analysis with some surprises. CoRR, abs/2408.04667. ↗
  13. Silvia Casola, Yang Janet Liu, Siyao Peng, Oliver Kraus, Albert Gatt, and Barbara Plank. 2025. References matter: Investigating the impact of reference set variation on summarization evaluation. pages 274–291. find
  14. Vince C Hartman, Sanika S Bapat, Mark G Weiner, Babak B Navi, Evan T Sholle, and Thomas R Campion Jr. 2023. A method to automate the discharge summary hospital course for neurology patients. Journal of the American Medical Informatics Association, 30(12):1995–2003. ↗
  15. Michael Haycock, Laura Stuttaford, Oliver Ruscombe-King, Zoe Barker, Kathryn Callaghan, and Timothy Davis. 2014. Improving the percentage of electronic discharge summaries completed within 24 hours of discharge. BMJ Quality Improvement Reports, 3(1). ↗
  16. Jia He, Mukund Rungta, David Koleczek, Arshdeep Sekhon, Franklin X Wang, and Sadid Hasan. 2024. Does prompt formatting have any impact on llm performance? arXiv preprint arXiv:2411.10541. ↗
  17. Felix Heilmeyer, Daniel Böhringer, Thomas Reinhard, Sebastian Arens, Lisa Lyssenko, and Christian Haverkamp. 2024. Viability of open large language models for clinical documentation in german health care: Real-world model evaluation study. JMIR Medical Informatics, 12:e59617. ↗
  18. Elias Hossain, Rajib Rana, Niall Higgins, Jeffrey Soar, Prabal Datta Barua, Anthony R. Pisani, and Kathryn Turner. 2023. Natural language processing in electronic health records in relation to healthcare decision-making: A systematic review. Computers in Biology and Medicine, 155:106649. ↗
  19. Steven Euijong Whang. 2025. Better think with tables: Tabular structures enhance llm comprehension for data-analytics requests. Preprint, arXiv:2412.17189. ↗
  20. Daniel Keszthelyi, Christophe Gaudet-Blavignac, Mina Bjelogrlic, Christian Lovis, and 1 others. 2023. Patient information summarization in clinical settings: scoping review. JMIR Medical Informatics, 11(1):e44639. find
  21. Maya Kruse, Shiyue Hu, Nicholas Derby, Yifu Wu, Samantha Stonbraker, Bingsheng Yao, Dakuo Wang, Elizabeth Goldberg, and Yanjun Gao. 2025. Zero-shot large language models for long clinical text summarization with temporal reasoning. medRxiv. ↗
  22. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  23. Yuanye Liu, Jiahang Xu, Li Lyna Zhang, Qi Chen, Xuan Feng, Yang Chen, Zhongxin Guo, Yuqing Yang, and Peng Cheng. 2025. Beyond prompt content: Enhancing llm performance via content-format integrated prompt optimization. arXiv preprint arXiv:2502.04295. ↗
  24. Ivan Lopez, Akshay Swaminathan, Karthik Vedula, Sanjana Narayanan, Fateme Nateghi Haredasht, Stephen P Ma, April S Liang, Steven Tate, Manoj Maddali, Robert Joseph Gallo, and 1 others. 2025. Clinical entity augmented retrieval for clinical information extraction. npj Digital Medicine, 8(1):45. find
  25. Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2022. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8086–8098, Dublin, Ireland. Association for Computational Linguistics.
  26. Skatje Myers, Timothy A Miller, Yanjun Gao, Matthew M Churpek, Anoop Mayampurath, Dmitriy Dligach, and Majid Afshar. 2025. Lessons learned on information retrieval in electronic health records: a comparison of embedding models and pooling strategies. Journal of the American Medical Informatics Association, 32(2):357–364. ↗
  27. Lilian Ngweta, Kiran Kate, Jason Tsay, and Yara Rizk. 2025. Towards LLMs robustness to changes in prompt format styles. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 4: Student Research Workshop), pages 529–537, Albuquerque, USA. Association for Computational Linguistics. ↗
  28. Jio Oh, Geon Heo, Seungjun Oh, Hyunjin Kim, JinYeong Bak, Jindong Wang, Xing Xie, and J Marc Overhage and David McCallie Jr. 2020. Physician time spent using the electronic health record during outpatient encounters: a descriptive study. Annals of internal medicine, 172(3):169–174. find
  29. Koyena Pal, Seyed Ali Bahrainian, Laura Mercurio, and Carsten Eickhoff. 2023. Neural summarization of electronic health records. CoRR, abs/2305.15222. ↗
  30. Maxime Peyrard. 2019. Studying summarization evaluation metrics in the appropriate scoring range. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5093– 5100. find
  31. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Belgium, Brussels. Association for Computational Linguistics. ↗
  32. Sandy L Robertson, Mark D Robinson, and Alfred Reid. 2017. Electronic health record effects on work-life balance and burnout within the i3 population collaborative. Journal of graduate medical education, 9(4):479–484. ↗
  33. Walid Saba, Suzanne Wendelken, and James Shanahan. 2024. Question-answering based summarization of electronic health records using retrieval augmented generation. CoRR, abs/2401.01469. ↗
  34. Tait D Shanafelt, Omar Hasan, Lotte N Dyrbye, Christine Sinsky, Daniel Satele, Jeff Sloan, and Colin P West. 2015. Changes in burnout and satisfaction with work-life balance in physicians and the general us working population between 2011 and 2014. In Mayo clinic proceedings, volume 90, pages 1600– 1613. Elsevier. ↗
  35. Han-Chin Shing, Chaitanya Shivade, Nima Pourdamghani, Feng Nan, Philip Resnik, Douglas W. Oard, and Parminder Bhatia. 2021. Towards clinical encounter summarization: Learning to compose discharge summaries from prior notes. CoRR, abs/2104.13498. ↗
  36. Sonish Sivarajkumar, Mark Kelley, Alyssa Samolyk-Mazzanti, Shyam Visweswaran, and Yanshan Wang. 2024. An empirical evaluation of prompting strategies for large language models in zero-shot clinical natural language processing: algorithm development and validation study. JMIR Medical Informatics, 12:e55318. ↗
  37. Vimig Socrates, Thomas Huang, Xuguang Ai, Soraya Fereydooni, Qingyu Chen, R Andrew Taylor, and David Chartash. 2024. Yale at “discharge me!”: Evaluating constrained generation of discharge summaries with unstructured and structured information. In Proceedings of the 23rd Workshop on Biomedical Natural Language Processing, pages 724–730. find
  38. An Quang Tang, Xiuzhen Zhang, and Minh Ngoc Dinh. 2024. IgnitionInnovators at “discharge me!”: Chain-of-thought instruction finetuning large language models for discharge summaries. In Proceedings of the 23rd Workshop on Biomedical Natural Language Processing, pages 731–739, Bangkok, Thailand. Association for Computational Linguistics. find
  39. Dave Van Veen, Cara Van Uden, Louis Blankemeier, Jean-Benoit Delbrouck, Asad Aali, Christian Bluethgen, Anuj Pareek, Malgorzata Polacin, Eduardo Pontes Reis, Anna Seehofnerová, and 1 others. 2024. Adapted large language models can outperform medical experts in clinical text summarization. Nature medicine, 30(4):1134–1142.
  40. Justin Xu, Zhihong Chen, Andrew Johnston, Louis Blankemeier, Maya Varma, Jason Hom, William J. Collins, Ankit Modi, Robert Lloyd, Benjamin Hopkins, Curtis Langlotz, and Jean-Benoit Delbrouck. 2024. Overview of the first shared task on clinical text generation: RRG24 and “discharge me!”. pages 85–98. find
  41. Pranjul Yadav, Michael Steinbach, Vipin Kumar, and Gyorgy Simon. 2018. Mining electronic health records (ehrs): A survey. ACM Comput. Surv., 50(6). ↗
  42. Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rauf-2026-medical-summarization-practice-design-deployment/