Elamur Research⌕
☰
home / papers / 2026 / Systematic Evaluation of the Quality of Synthetic Clinical Notes Rephrased by LLMs at Million-Note Scale

Systematic Evaluation of the Quality of Synthetic Clinical Notes Rephrased by LLMs at Million-Note Scale

Paper · 2026 · Jinghui Liu, Sarvesh Soni and Anthony Nguyen

Summary

Studies of LLM-generated clinical text usually judge it on a single axis, such as similarity to a reference or usefulness for one task. This paper evaluates one large synthetic corpus from several angles at once. The authors took 4.6M deduplicated notes from MIMIC-III and MIMIC-IV and had three compact open-source LLMs rewrite them. Each note was rewritten either whole or in roughly 150-word chunks.

The evaluation combined:

  • linguistic and similarity metrics (ROUGE, BERTScore, UMLS concept overlap);
  • downstream prediction tasks, with a fairness check;
  • LLM-as-judge fact-checking;
  • manual error analysis.

The rewritten notes differ noticeably in wording but keep the core clinical content. They also stay useful for coarse prediction tasks. They lose fine detail, however, which hurts ICD coding. Chunk-wise rewriting recovers much of that detail, but lack of surrounding context lowers factual precision. The most common errors are misreading clinical context, followed by temporal mix-ups, wrong measurements and invented claims. Although the notes were not built for any particular task, they help as augmentation data for rare ICD codes. This makes the released corpus a practical resource for clinical NLP.

Key points

  • Source data: MIMIC-III (v1.4) and MIMIC-IV (v2.2), giving 4.6M notes and 1.4B words after exact-duplicate removal.
  • Generation: notes were rephrased with Llama-3.1 (8B), Mistral-v0.3 (7B) and Qwen-2.5 (7B) using vLLM.
    • This took about 1.8K H100 GPU hours.
    • Chunked rephrasing averaged 2.3 chunks per note.
  • Released resource: MIMIC-Ext-SynNotes, a synthetic corpus of about 9.1B words, available on GitHub.
  • Evaluation tools:
    • Similarity: ROUGE and a DeBERTa-based BERTScore.
    • Medical concepts: CUI precision, recall and F1 via QuickUMLS, following MedCON.
    • Downstream tasks: in-hospital mortality, 30-day readmission, DRG prediction and ICD coding.
    • Fact-checking: Gemini via Vertex AI.
  • Robustness check: extra runs with Gemma and Mixtral produced broadly similar results.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏛️National Institutes of HealthBethesda · United States12 works · 3,846 citations🔬United States National Library of MedicineBethesda · United States7 works · 5,342 citations🏛️Commonwealth Scientific and Industrial Research OrganisationCanberra · Australia15 works · 1,898 citations

Also on the byline: Australian e-Health Research Centre.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksMedicine · 241 worksHealth Informatics · 184 works

🧭 Topics 1Research subjects and languages the work is about.Evaluation & benchmarks · 1,220 works

Cite

Cite as Jinghui Liu, Sarvesh Soni and Anthony Nguyen. 2026. Systematic Evaluation of the Quality of Synthetic Clinical Notes Rephrased by LLMs at Million-Note Scale. BioNLP 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.bionlp-1.28

@inproceedings{liu2026systematic,
  title = {{Systematic Evaluation of the Quality of Synthetic Clinical Notes Rephrased by LLMs at Million-Note Scale}},
  author = {Jinghui Liu, Sarvesh Soni and Anthony Nguyen},
  booktitle = {BioNLP 2026},
  year = {2026},
  doi = {10.18653/v1/2026.bionlp-1.28},
  url = {https://doi.org/10.18653/v1/2026.bionlp-1.28}
}

References

74 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Basel Alshaikhdeeb, Ahmed Abdelmonem Hemedan, Soumyabrata Ghosh, Irina Balaur, and Venkata Satagopam. 2025. Generation of synthetic clinical text: A systematic review. arXiv [cs.CL]. https://physionet.org/news/post/gpt-responsible-use ↗
  2. Philippe Besse, Eustasio del Barrio, Paula Gordaliza, Jean-Michel Loubes, and Laurent Risser. 2021. A survey of bias in machine learning through the prism of statistical parity. The American statistician, pages 1–11. find
  3. William Boag, Olga Kovaleva, Thomas H McCoy, Jr, Anna Rumshisky, Peter Szolovits, and Roy H Perlis. 2021. Hard for humans, hard for machines: predicting readmission after psychiatric hospitalization using narrative notes. Translational psychiatry, 11(1):32. find
  4. Katherine E Brown, Chao Yan, Zhuohang Li, Xinmeng Zhang, Benjamin X Collins, You Chen, Ellen Wright Clayton, Murat Kantarcioglu, Yevgeniy Vorobeychik, and Bradley A Malin. 2025. Large language models are less effective at clinical prediction tasks than locally trained machine learning models. Journa… find
  5. Kristine Ann M Carandang, Jasper Meynard Arana, Ethan Robert Casin, Christopher Monterola, Daniel Stanley Tan, Jesus Felix B Valenzuela, and Christian Alis. 2025. Are LLMs reliable? an exploration of the reliability of large language models in clinical note generation. In Proceedings of the 63rd Ann… find
  6. Philip Chung, Akshay Swaminathan, Alex J Goodell, Yeasul Kim, S Momsen Reincke, Lichy Han, Ben Deverett, Mohammad Amin Sadeghi, Abdel-Badih Ariss, Marc Ghanem, David Seong, Andrew A Lee, Caitlin E Coombes, Brad Bradshaw, Mahir A Sufian, Hyo Jung Hong, Teresa P Nguyen, Mohammad R Rasouli, Komal Kamra… find
  7. Tiago K Colicchio, James J Cimino, and Guilherme Del Fiol. 2019. Unintended consequences of nationwide electronic health record adoption: Challenges and opportunities in the Post-Meaningful use era. Journal of medical Internet research, 21(6):e13313. find
  8. Emma Croxford, Yanjun Gao, Nicholas Pellegrino, Karen Wong, Graham Wills, Elliot First, Frank Liao, Cherodeep Goswami, Brian Patterson, and Majid Afshar. 2025. Current and future state of evaluation of large language models for medical summarization tasks. npj Health Systems, 2(1):1–13. find
  9. James C Douglas, Yidong Gan, Ben Hachey, and Jonathan K Kummerfeld. 2025. Less is more: Explainable and efficient ICD code prediction with clinical entities. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 30835–30847, Strouds… find
  10. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, and 516 others. 202… — The Llama 3 Herd of Models
  11. Joakim Edin, Alexander Junge, Jakob D Havtorn, Lasse Borgholt, Maria Maistro, Tuukka Ruotsalo, and Lars Maaløe. 2023. Automated medical coding on MIMIC-III and MIMIC-IV: A critical review and replicability study. In Proceedings of the 46th International ACM SIGIR Conference on Research and Developme… find
  12. Yidong Gan, Maciej Rybinski, Ben Hachey, and Jonathan K Kummerfeld. 2025. Aligning AI research with the needs of clinical coding workflows: Eight recommendations based on US data analysis and critical review. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics… find
  13. Yanjun Gao, Dmitriy Dligach, Timothy Miller, Dongfang Xu, Matthew M M Churpek, and Majid Afshar. 2022. Summarizing patients’ problems from hospital progress notes using pre-trained sequence-to-sequence models. In Proceedings of the 29th International Conference on Computational Linguistics, pages 29… find
  14. Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Harkirat Singh Behl, Xin Wang, Sébastien Bubeck, Ronen Eldan, Adam Tauman Kalai, Yin Tat Lee, and Yuanzhi… find
  15. Moritz Hardt, Eric Price, and Nathan Srebro. 2016. Equality of opportunity in supervised learning. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16, pages 3323–3331, Red Hook, NY, USA. Curran Associates Inc.
  16. G Harry and Mc Laughlin. 1969. SMOG grading - a new readability formula. The Journal of Reading. find
  17. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. DeBERTaV3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing. In The Eleventh International Conference on Learning Representations.
  18. Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, and Boris Ginsburg. 2024. RULER: What’s the real context size of Chao-Wei Huang, Shang-Chi Tsai, and Yun-Nung Chen. 2022. PLM-ICD: Automatic ICD coding with pre-trained language models. In Proceedings of the 4th Cli… find
  19. Kexin Huang, Jaan Altosaar, and Rajesh Ranganath. 2019. ClinicalBERT: Modeling clinical notes and predicting hospital readmission. arXiv [cs.CL].
  20. Shaoxiong Ji, Xiaobo Li, Wei Sun, Hang Dong, Ara Taalas, Yijia Zhang, Honghan Wu, Esa Pitkänen, and Pekka Marttinen. 2024. A unified review of deep learning for automated medical coding. ACM computing surveys. find
  21. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… find
  22. Lavender Yao Jiang, Xujin Chris Liu, Nima Pour Nejatian, Mustafa Nasir-Moin, Duo Wang, Anas Abidin, Kevin Eaton, Howard Antony Riina, Ilya Laufer, Paawan Punjabi, Madeline Miceli, Nora C Kim, Cordelia Orillac, Zane Schnurman, Christopher Livia, Hannah Weiss, David Kurland, Sean Neifert, Yosef Dastag… find
  23. Alistair E W Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J Pollard, Benjamin Moody, Brian Gow, Li-Wei H Lehman, Leo A Celi, and Roger G Mark. 2023. MIMIC-IV, a freely accessible electronic health record dataset. Scientific data, 10(1):1.
  24. Alistair E W Johnson, Tom J Pollard, Lu Shen, Li-Wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016. MIMIC-III, a freely accessible critical care database. Scientific data, 3:160035.
  25. Lamin Juwara, Alaa El-Hussuna, and Khaled El Emam. 2024. An evaluation of synthetic data augmentation for mitigating covariate bias in health data. Patterns (New York, N.Y.), 5(4):100946. find
  26. Nicolas M Kahl, Marshall J Frieden, Zach R Pope, Marlene M Millen, Vaishal M Tolia, Theodore C Chan, Christopher A Longhurst, Karandeep Singh, and Alan X You. 2026. Evaluation of electronic health record-integrated artificial intelligence chart review. Npj Health Systems, 3(1):6. find
  27. Feiyang Kang, Newsha Ardalani, Michael Kuchnik, Youssef Emad, Mostafa Elhoushi, Shubhabrata Sengupta, Shang-Wen Li, Ramya Raghavendra, Ruoxi Jia, and Carole-Jean Wu. 2025. Demystifying synthetic data in LLM pre-training: A systematic study of scaling laws, benefits, and pitfalls. In Proceedings of t… find
  28. Ira Ktena, Olivia Wiles, Isabela Albuquerque, Sylvestre-Alvise Rebuffi, Ryutaro Tanno, Abhijit Guha Roy, Shekoofeh Azizi, Danielle Belgrave, Pushmeet Kohli, Taylan Cemgil, Alan Karthikesalingam, and Sven Gowal. 2024. Generative models improve fairness of medical classifiers under distribution shifts… find
  29. Sunjun Kweon, Junu Kim, Jiyoun Kim, Sujeong Im, Eunbyeol Cho, Seongsu Bae, Jungwoo Oh, Gyubok Lee, Jong Hak Moon, Seng Chan You, Seungjin Baek, Chang Hoon Han, Yoon Bin Jung, Yohan Jo, and Edward Choi. 2024. Publicly shareable clinical large language model built on synthetic clinical notes. In Findi… find
  30. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles, pages 611–626, New…
  31. Patrick Lewis, Myle Ott, Jingfei Du, and Veselin Stoyanov. 2020. Pretrained language models for biomedical and clinical tasks: Understanding and extending the State-of-the-Art. In Proceedings of the 3rd Clinical Natural Language Processing Workshop, pages 146–157, Online. Association for Computation…
  32. Jeffrey Li, Alex Fang, Georgios Smyrnis, Maor Ivgi, Matt Jordan, Samir Yitzhak Gadre, Hritik Bansal, Etash Kumar Guha, Sedrick Keh, Kushal Arora, Saurabh Garg, Rui Xin, Niklas Muennighoff, Reinhard Heckel, Jean Mercat, Mayee F Chen, Suchin Gururangan, Mitchell Wortsman, Alon Albalak, and 40 others.… — DataComp-LM: In search of the next generation of training sets for language models
  33. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 28779–28796, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  34. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  35. Jinghui Liu, Daniel Capurro, Anthony Nguyen, and Karin Verspoor. 2022a. “note bloat” impacts deep learning-based NLP models for clinical prediction tasks. Journal of biomedical informatics, 133:104149. find
  36. Jinghui Liu, Bevan Koopman, Nathan J Brown, Kevin Chu, and Anthony Nguyen. 2024a. Generating synthetic clinical text with local large language models to identify misdiagnosed limb fractures in radiology reports. Artificial intelligence in medicine, 159(103027):103027. find
  37. Jinghui Liu and Anthony Nguyen. 2024. Rephrasing electronic health records for pretraining clinical language models. In Proceedings of the 22nd Annual Workshop of the Australasian Language Technology Association, pages 164–172. find
  38. Leibo Liu, Oscar Perez-Concha, Anthony Nguyen, Vicki Bennett, and Louisa Jorm. 2022b. Hierarchical label-wise attention transformer model for explainable ICD coding. Journal of biomedical informatics, 133:104161. find
  39. Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024b. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173.
  40. Mohammad Loni, Fatemeh Poursalim, Mehdi Asadi, and Arash Gharehbaghi. 2025. A review on generative AI models for synthetic medical text, time series, and longitudinal data. npj digital medicine, 8(1):281. find
  41. Pratyush Maini, Vineeth Dorna, Parth Doshi, Aldo Carranza, Fan Pan, Jack Urbanek, Paul Burstein, Alex Fang, Alvin Deng, Amro Abbas, Brett Larsen, Cody Blakeney, Charvi Bannur, Christina Baek, Darren Teh, David Schwab, Haakon Mongstad, Haoli Yin, Josh Wills, and 10 others. 2025. BeyondWeb: Lessons fr… find
  42. Rumeng Li, Xun Wang, and Hong Yu. 2023. Two directions for clinical data generation with large language models: Data-to-label and label-to-data. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7129–7143, Singapore. Association for Computational Linguistics. find
  43. Pratyush Maini, Skyler Seto, Richard Bai, David Grangier, Yizhe Zhang, and Navdeep Jaitly. 2024. Rephrasing the web: A recipe for compute and data-efficient language modeling. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14… find
  44. Zihan Liang, Ziwen Pan, and Ruoxuan Xiong. 2025. Causal representation learning from multimodal clinical records under non-random modality missingness. find
  45. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-Tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural L…
  46. Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 6252–6278, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  47. Avijit Mitra, Zhichao Yang, Emily Druhl, Raelene Goodwin, and Hong Yu. 2025. Synth-SBDH: A synthetic dataset of social and behavioral determinants of health for clinical text. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 27887–27923, Stroudsburg, P… find
  48. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. 2024. AI models collapse when trained on recursively generated data. Nature, 631(8022):755–759. find
  49. Monica Munnangi, Akshay Swaminathan, Jason Alan Fries, Jenelle A Jindal, Sanjana Narayanan, Ivan Lopez, Lucia Tu, Philip Chung, Jesutofunmi Omiye, Mehr Kashyap, and Nigam Shah. 2025. FactEHR: A dataset for evaluating factuality in clinical notes using LLMs. In Machine Learning for Healthcare Confere… find
  50. Aaron Nicolson, Jinghui Liu, Jason Dowling, Anthony Nguyen, and Bevan Koopman. 2024. e-health CSIRO at RRG24: Entropy-augmented self-critical sequence training for radiology report generation. In Proceedings of the 23rd Workshop on Biomedical Natural Language Processing, pages 99–104. find
  51. Guilherme Penedo, Hynek Kydlíček, Loubna Ben Allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. 2024. The FineWeb datasets: Decanting the web for the finest text data at scale. arXiv [cs.CL].
  52. Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, and 24 others. 2024. Qwen2.5 technical report. arXiv [cs.CL]. find
  53. Kirk Roberts. 2024. Large language models for reducing clinicians’ documentation burden. Nature medicine. find
  54. Ananya B Sai, Akash Kumar Mohankumar, and Mitesh M Khapra. 2022. A survey of evaluation metrics used for NLG systems. ACM Comput. Surv., 55(2):1–39. find
  55. Arne Schwieger, Katrin Angst, Mateo de Bardeci, Achim Burrer, Flurin Cathomas, Stefano Ferrea, Franziska Grätz, Marius Knorr, Golo Kronenberg, Tobias Spiller, David Troi, Erich Seifritz, Samantha Weber, and Sebastian Olbrich. 2024. Large language models can support generation of standardized dischar… find
  56. Yijia Shao, Yucheng Jiang, Theodore Kanell, Peter Xu, Omar Khattab, and Monica Lam. 2024. Assisting in writing wikipedia-like articles from scratch with large language models. In Proceedings of the 2024
  57. Yuqi Si and Kirk Roberts. 2019. Deep patient representation of clinical notes via Multi-Task learning for mortality prediction. AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science, 2019:779–788. find
  58. William R Small, Jonathan Austrian, Luke O’Donnell, Jesse Burk-Rafel, Katherine A Hochman, Adam Goodman, Jonah Zaretsky, Jacob Martin, Stephen Johnson, Vincent J Major, Simon Jones, Christian Henke, Benjamin Verplanke, Jwan Osso, Ian Larson, Archana Saxena, Aron Mednick, Choumika Simonis, Joseph Han… find
  59. Luca Soldaini. 2016. QuickUMLS: a fast, unsupervised approach for medical concept extraction. MedIR workshop, sigir. find
  60. M Solnyshkina, R Zamaletdinov, L Gorodetskaya, and A Gabitov. 2017. Evaluating text complexity and flesch-kincaid grade level. Journal of social studies education research, 8(3):238–248. find
  61. Sarvesh Soni and Dina Demner-Fushman. 2024. Toward relieving clinician burden by automatically generating progress notes using interim hospital data. AMIA Annual Symposium Proceedings, 2024:1059– 1068. find
  62. Sarvesh Soni and Kirk Roberts. 2020. Paraphrasing to improve the performance of electronic health records question answering. AMIA Summits on Translational Science proceedings AMIA Summit on Translational Science, 2020:626–635. find
  63. Ruixiang Tang, Xiaotian Han, Xiaoqian Jiang, and Xia Hu. 2023. Does synthetic data generation of LLMs help clinical text mining? arXiv [cs.CL]. find
  64. Dave Van Veen, Cara Van Uden, Louis Blankemeier, Jean-Benoit Delbrouck, Asad Aali, Christian Bluethgen, Anuj Pareek, Malgorzata Polacin, Eduardo Pontes Reis, Anna Seehofnerová, Nidhi Rohatgi, Poonam Hosamani, William Collins, Neera Ahuja, Curtis P Langlotz, Jason Hom, Sergios Gatidis, John Pauly, an… — Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarization
  65. Hanyin Wang, Chufan Gao, Christopher Dantona, Bryan Hull, and Jimeng Sun. 2024a. DRG-LLaMA : tuning A LLaMA model to predict diagnosis-related group for hospitalized patients. NPJ digital medicine, 7(1):16. find
  66. Junda Wang, Zonghai Yao, Zhichao Yang, Huixue Zhou, Rumeng Li, Xun Wang, Yucheng Xu, and Hong Yu. 2024b. NoteChat: A dataset of synthetic patient-physician conversations conditioned on clinical notes. In Findings of the Association for Computational Linguistics ACL 2024, pages 15183–15201, Stroudsbu… find
  67. For clinical text synthesis, each LLM was provided with a system prompt consistent with its chat template and was instructed to synthesize a new version using a paraphrasing prompt found effective in prior studies (Liu and Nguyen, 2024; Maini et al., 2024), denoted as the “high quality” prompt: Reph…
  68. Yiqing Xie, Sheng Zhang, Hao Cheng, Pengfei Liu, Zelalem Gero, Cliff Wong, Tristan Naumann, Hoifung Poon, and Carolyn Rose. 2024. DocLens: Multi-aspect fine-grained medical text evaluation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap… find
  69. Justin Xu, Zhihong Chen, Andrew Johnston, Louis Blankemeier, Maya Varma, Jason Hom, William J Collins, Ankit Modi, Robert Lloyd, Benjamin Hopkins, Curtis Langlotz, and Jean-Benoit Delbrouck. 2024. Overview of the first shared task on clinical text generation: RRG24 and “discharge me!”. In Proceeding… find
  70. Wen-Wai Yim, Yujuan Fu, Asma Ben Abacha, Neal Snider, Thomas Lin, and Meliha Yetisgen. 2023. Aci-bench: a novel ambient clinical intelligence dataset for benchmarking automatic visit note generation. Scientific data, 10(1):586. find
  71. Feiyang Yu, Mark Endo, Rayan Krishnan, Ian Pan, Andy Tsai, Eduardo Pontes Reis, Eduardo Kaiser Ururahy Fonseca, Henrique Min Ho Lee, Zahra Shakeri Hossein Abad, Andrew Y Ng, Curtis P Langlotz, Vasantha Kumar Venugopal, and Pranav Rajpurkar. 2023. Evaluating progress in automatic chest X-ray radiolog… find
  72. Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating text generation with BERT. In International Conference on Learning Representations. find
  73. Hendrik Šuvalov, Mihkel Lepson, Veronika Kukk, Maria Malk, Neeme Ilves, Hele-Andra Kuulmets, and Raivo Kolde. 2025. Using synthetic health care data to leverage large language models for named entity recognition: Development and validation study. Journal of medical internet research, 27(1):e66279. find
  74. Details of Rephrasing MIMIC Notes [System Prompt] You are a medical artificial intelligence assistant. The assistant gives truthful, detailed, and professional answers to the requests. For the following paragraph give me a diverse paraphrase of the same in high-quality English language as in clinica… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2026-systematic-evaluation-quality-synthetic-clinical/