Elamur Research⌕
☰
home / papers / 2026 / X-ray Made Simple: Lay Radiology Report Generation and Robust Evaluation

X-ray Made Simple: Lay Radiology Report Generation and Robust Evaluation

Paper · 2026 · Kun Zhao, Chenghao Xiao, Sixing Yan et al.

Summary

Chest X-ray reports are written in dense clinical jargon and follow rigid templates. This creates two problems. Patients struggle to understand reports they now often see before talking to a doctor. Models can also score well on n-gram metrics like BLEU simply by copying recurring report phrasing. The authors propose the Layman's RRG framework, which uses plain-language reports to address both issues. Starting from MIMIC-CXR, they build two layman-style resources:

  • a sentence-level set, used for evaluation;
  • a report-level set, used for training.

Both come from a pipeline that removes near-duplicate sentences, rewrites them with LLMs, checks and refines the output, and ends with human verification. For evaluation, professional sentences are mapped to semantically matching layman equivalents via embeddings, and a semantics-based score is computed. This reduces the score inflation that templates produce. Models trained on layman-style reports capture clinical meaning better. Their performance keeps rising as layman data grows, whereas professional-language training shows the opposite trend. Automatic metrics on layman outputs also track human judgments more closely. The work therefore supports more patient-friendly and more reliably evaluated report generation.

Key points

  • Deduplication: GritLM embeddings with a cosine-similarity cutoff of 0.8 shrank roughly 490,000 MIMIC-CXR sentences to about 50,000 before layman rewriting.
  • Rewriting and refinement: ChatGPT-4o, Kimi and DeepSeek produced the rewrites. Refinement combined an embedding-based semantic check with LLM self-verification of correctness.
  • Human evaluation: Inter-annotator agreement (Cohen's Kappa) was 0.63 for professional outputs and 0.58 for layman outputs.
  • Metric correlation: Layman-style outputs aligned better with human scores on every metric tested, including BLEU, ROUGE, METEOR, CheXbert-F1, RadGraph-F1, RadCliQ, GREEN and BERTScore.
  • Release and limitations: Datasets and code are available at github.com/hegehongcha/LaymanRRG. The authors note that the references are LLM-generated "silver-standard" data and that the work relies on MIMIC-CXR as its only data source.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of PittsburghPittsburgh · United States59 works · 22,324 citations🎓University of ManchesterManchester · United Kingdom87 works · 44,938 citations🎓Hong Kong Baptist UniversityHong Kong · China7 works · 246 citations🎓Durham UniversityDurham · United Kingdom8 works · 561 citations

Also on the byline: The University of Texas Rio Grande Valley.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 worksMedicine · 241 worksHealth Informatics · 184 works

Cite

Cite as Kun Zhao, Chenghao Xiao, Sixing Yan, Haoteng Tang, William K. Cheung, Noura Al Moubayed, Liang Zhan and Chenghua Lin. 2026. X-ray Made Simple: Lay Radiology Report Generation and Robust Evaluation. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1726

@inproceedings{zhao2026x,
  title = {{X-ray Made Simple: Lay Radiology Report Generation and Robust Evaluation}},
  author = {Kun Zhao, Chenghao Xiao, Sixing Yan, Haoteng Tang, William K. Cheung, Noura {Al Moubayed}, Liang Zhan and Chenghua Lin},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1726},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1726}
}

References

30 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. 21st Century Cures Act. 2016. 21st century cures act. An Act to accelerate the discovery, development, and delivery of 21st century cures, and for other purposes. find
  2. Rebecca Anhang Price, Marc N Elliott, Alan M Zaslavsky, Ron D Hays, William G Lehrman, Lise Ry-bowski, Susan Edgman-Levitan, and Paul D Cleary. 2014. Examining the role of patient experience surveys in measuring health care quality. Medical Care Research and Review, 71(5):522–554. find
  3. Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72.
  4. Chris Callison-Burch, Miles Osborne, and Philipp Koehn. 2006. Re-evaluating the role of bleu in machine translation research. In 11th conference of the european chapter of the association for computational linguistics, pages 249–256.
  5. Siyuan Dai, Lunxiao Li, Kun Zhao, Eardi Lila, Paul K Crane, Heng Huang, Dongkuan Xu, Haoteng Tang, and Liang Zhan. 2025a. Why text prevails: Vision may undermine multimodal medical decision making. arXiv preprint arXiv:2512.13747. — [arXiv:2512.13747](https://arxiv.org/abs/2512.13747) find
  6. Siyuan Dai, Kai Ye, Guodong Liu, Haoteng Tang, and Liang Zhan. 2025b. Zeus: zero-shot llm instruction for union segmentation in multimodal medical imaging. International Journal of Machine Learning and Cybernetics, 16(11):9067–9078. find
  7. Francesco Dalla Serra, William Clackett, Hamish MacKinnon, Chaoyang Wang, Fani Deligianni, Jeff Dalton, and Alison Q O’Neil. 2022. Multimodal generation of radiology reports using knowledge-grounded extraction of entities and relations. In Proceedings of the 2nd Conference of the Asia-Pacific Chapte… find
  8. Jane Domingo, Galal Galal, Jonathan Huang, Priyanka Soni, Vladislav Mukhin, Camila Altman, Tom Bayer, Thomas Byrd, Stacey Caron, Patrick Creamer, et al. 2022. Preventing delayed and missed care by applying artificial intelligence to trigger radiology imaging follow-up. NEJM Catalyst Innovations in C… find
  9. Wenjun Hou, Kaishuai Xu, Yi Cheng, Wenjie Li, and Jiang Liu. 2023. Organ: observation-guided radiology report generation via tree reasoning. arXiv preprint arXiv:2306.06466. find
  10. Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. 2019. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAAI conference on art… find
  11. Chang Liu, Yuanhe Tian, Weidong Chen, Yan Song, and Yongdong Zhang. 2024. Bootstrapping large language models for radiology report generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18635–18643. find
  12. Chang Liu, Yuanhe Tian, and Yan Song. 2023. A systematic review of deep learning-based research on radiology report generation. arXiv preprint arXiv:2311.14199. — [arXiv:2311.14199](https://arxiv.org/abs/2311.14199) find
  13. Pilar López-Úbeda, Teodoro Martín-Noguerol, Jorge Escartín, Alberto Cabrera-Zubizarreta, and Antonio Luna. 2024. Automated mri pituitary structured reporting from free-text using a fine-tuned llama model: a feasibility study. Japanese Journal of Radiology, pages 1–9. find
  14. Saahil Jain, Ashwin Agrawal, Adriel Saporta, Steven QH Truong, Du Nguyen Duong, Tan Bui, Pierre Chambon, Yuhao Zhang, Matthew P. Lungren, Andrew Y. Ng, Curtis P. Langlotz, and Pranav Rajpurkar. 2021. Radgraph: Extracting clinical entities and relations from radiology reports. Preprint, arXiv:2106.14… — [arXiv:2106.14463](https://arxiv.org/abs/2106.14463) find
  15. Thusitha Mabotuwana, Christopher S Hall, Vadiraj Hombal, Prashanth Pai, Usha Nandini Raghavan, Shawn Regis, Brady McKee, Sandeep Dalal, Christoph Wald, and Martin L Gunn. 2019. Automated tracking of follow-up imaging recommendations. American Journal of Roentgenology, 212(6):1287–1294. find
  16. Alistair EW Johnson, Tom J Pollard, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Yifan Peng, Zhiyong Lu, Roger G Mark, Seth J Berkowitz, and Steven Horng. 2019. Mimic-cxr-jpg, a large publicly available database of labeled chest radiographs. arXiv preprint arXiv:1901.07042. — [arXiv:1901.07042](https://arxiv.org/abs/1901.07042) find
  17. Thusitha Mabotuwana, Christopher S Hall, Joel Tieder, and Martin L Gunn. 2018. Improving quality of follow-up imaging recommendations in radiology. In AMIA annual symposium proceedings, volume 2017, page 1196. find
  18. Kaveri Kale, Pushpak Bhattacharyya, and Kshitij Sharad Jadhav. 2023. Replace and report: Nlp assisted radiology report generation. ArXiv, abs/2306.17180. find
  19. Suhyeon Lee, Won Jun Kim, Jinho Chang, and Jong Chul Ye. 2023. Llm-cxr: Instruction-finetuned llm for cxr image understanding and generation. In The Twelfth International Conference on Learning Representations. find
  20. Christy Y Li, Xiaodan Liang, Zhiting Hu, and Eric P Xing. 2019. Knowledge-driven encode, retrieve, paraphrase for medical image report generation. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 6666–6673. find
  21. Yaowei Li, Bang Yang, Xuxin Cheng, Zhihong Zhu, Hongxiang Li, and Yuexian Zou. 2023. Unify, align and refine: Multi-level semantic alignment for radiology report generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2863–2874. find
  22. Chen Lin, Shuai Zheng, Zhizhe Liu, Youru Li, Zhenfeng Zhu, and Yao Zhao. 2022. Sgt: Scene graph-guided transformer for surgical report generation. In International conference on medical image computing and computer-assisted intervention, pages 507–518. Springer. find
  23. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81.
  24. Teresa Martin-Carreras, Tessa S Cook, and Charles E Kahn Jr. 2019. Readability of radiology reports: implications for patient-centered care. Clinical imaging, 54:116–120. find
  25. Niklas Muennighoff, Hongjin Su, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. 2024. Generative representational instruction tuning. arXiv preprint arXiv:2402.09906.
  26. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
  27. Yifan Peng, Xiaosong Wang, Le Lu, Mohammadhadi Bagheri, Ronald Summers, and Zhiyong Lu. 2018. Negbio: a high-performance tool for negation and uncertainty detection in radiology reports. AMIA Summits on Translational Science Proceedings, 2018:188.
  28. Akshay Smit, Saahil Jain, Pranav Rajpurkar, Anuj Pareek, Andrew Ng, and Matthew Lungren. 2020. Combining automatic labelers and expert annotations for accurate radiology report labeling using BERT. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pag…
  29. Aaron Smith, Christian Hardmeier, and Jörg Tiedemann. 2016. Climbing mont bleu: the strange world of reachable high-bleu translations. In Proceedings of the 19th annual conference of the European association for machine translation, pages 269–281. find
  30. Amanda Stent, Matthew Marge, and Mohit Singhai. 2005. Evaluating evaluation methods for generation in the presence of variation. In International conference on intelligent text processing and computational linguistics, pages 341–351. Springer. 15421–15435, Bangkok, Thailand and virtual meeting. Asso… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhao-2026-x-ray-made-simple-lay/