Elamur Research⌕
☰
home / papers / 2026 / Enhancing Reinforcement Learning for Radiology Report Generation with Evidence-aware Rewards and Self-correcting Preference Learning

Enhancing Reinforcement Learning for Radiology Report Generation with Evidence-aware Rewards and Self-correcting Preference Learning

Paper · 2026 · Qin Zhou, Guoyan Liang, Qianyi Yang et al.

Summary

The paper addresses two weaknesses of reinforcement-learning approaches to automatic radiology report generation (RRG). First, rewards computed over the whole report give little guidance tied to image evidence. Second, existing methods have no built-in way to fix their own unreliable statements. The authors propose ESC-RL, which has two parts.

  • GEAR (evidence-aware reward): CheXbert labels from generated and reference reports sort findings into true positives, false negatives, and false positives. A frozen grounding model (MAVL) then produces disease-specific response maps. The reward favors matching maps for correct findings, pushes the model to recover omitted findings, and penalizes claims lacking visual support.
  • SPL (self-correcting preference learning): The policy samples several candidate reports. A lightweight predictor, trained on an automatically built disease-level preference set, scores each disease description and discards unreliable ones. An LLM then combines the remaining descriptions into a refined report, with no human annotation.

On the MIMIC-CXR and IU-Xray chest X-ray benchmarks, ESC-RL reports state-of-the-art results on both lexical and clinical metrics. This suggests that evidence-grounded rewards can reduce hallucinated and missed findings.

Key points

  • Data: Evaluation uses two public, de-identified chest X-ray datasets, MIMIC-CXR and IU-Xray. Clinical efficacy is measured over 14 disease categories.
  • Disease labels: Each disease has one of four states (blank, positive, negative, uncertain). Only positive labels are used to form the TP, FN, and FP groups.
  • Ablation on the MIMIC-CXR test set (full model vs. baseline):
    • BLEU-1: 0.465 → 0.487
    • RadGraphF1: 0.279 → 0.304
    • CheXbertF1: 0.514 → 0.608
    • GREEN: 0.344 → 0.394
    • RadCliQ: 2.48 → 2.39 (lower is better)
  • Individual contributions: Each component improves GREEN on its own. Plain RL adds 0.5 points, GEAR adds 1.4, and SPL adds 1.2. Combining them gives complementary gains.
  • Limitations: The method has been tested only on chest X-rays, so transfer to CT, MRI, or other body regions is unverified. It also depends on extra pretrained models (CheXbert, MAVL), which adds computational cost.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Zhejiang UniversityHangzhou · China224 works · 13,587 citations🔬Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of EducationShanghai · China3 works · 41 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksComputer Vision and Pattern Recognition · 3,486 worksMedicine · 275 worksHealth Informatics · 214 worksRadiology, Nuclear Medicine and Imaging · 5 works

Cite

Cite as Qin Zhou, Guoyan Liang, Qianyi Yang, Jingyuan Chen, Sai Wu, Chang Yao and Zhe Wang. 2026. Enhancing Reinforcement Learning for Radiology Report Generation with Evidence-aware Rewards and Self-correcting Preference Learning. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1718

@inproceedings{zhou2026enhancing,
  title = {{Enhancing Reinforcement Learning for Radiology Report Generation with Evidence-aware Rewards and Self-correcting Preference Learning}},
  author = {Qin Zhou, Guoyan Liang, Qianyi Yang, Jingyuan Chen, Sai Wu, Chang Yao and Zhe Wang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1718},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1718}
}

References

32 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Asma Alkhaldi, Raneem Alnajim, Layan Alabdullatef, Rawan Alyahya, Jun Chen, Deyao Zhu, Ahmed Alsi-nan, and Mohamed Elhoseiny. 2024. Minigpt-med: Large language model as a general interface for radiology diagnosis. Preprint, arXiv:2407.04106. — [arXiv:2407.04106](https://arxiv.org/abs/2407.04106) find
  2. Zhihong Chen, Yaling Shen, Yan Song, and Xiang Wan. 2022. Cross-modal memory networks for radiology report generation. Preprint, arXiv:2204.13258. find
  3. Zhihong Chen, Yan Song, Tsung-Hui Chang, and Xiang Wan. 2020. Generating radiology reports via memory-driven transformer. arXiv preprint arXiv:2010.16056.
  4. Zhihong Chen, Maya Varma, Jean-Benoit Delbrouck, Magdalini Paschali, Louis Blankemeier, Dave Van Veen, Jeya Maria Jose Valanarasu, Alaa Youssef, Joseph Paul Cohen, Eduardo Pontes Reis, Emily B. Tsai, Andrew Johnston, Cameron Olsen, Tanishq Mathew Abraham, Sergios Gatidis, Akshay S Chaudhari, and Cur… — [arXiv:2401.12208](https://arxiv.org/abs/2401.12208) find
  5. Jie Cheng, Gang Xiong, Xingyuan Dai, Qinghai Miao, Yisheng Lv, and Fei-Yue Wang. 2024. RIME: Robust preference-based reinforcement learning with noisy preferences. In Proceedings of the 41st International Conference on Machine Learning, volume 235, pages 8229–8247. PMLR. find
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  7. Demner Fushman Dina, Marc D Kohli, Marc B Rosenman, Sonya E Shooshan, Rodriguez Laritza, Antani Sameer, George R Thoma, and Clement J Mcdonald. 2015. Preparing a collection of radiology examinations for distribution and retrieval. Journal of the American Medical Informatics Association Jamia, (2):2. find
  8. Wenjun Hou, Yi Cheng, Kaishuai Xu, Heng Li, Yan Hu, Wenjie Li, and Jiang Liu. 2025. Radar: Enhancing radiology report generation with supplementary knowledge injection. Preprint, arXiv:2505.14318. find
  9. Saahil Jain, Ashwin Agrawal, Adriel Saporta, Steven QH Truong, Du Nguyen Duong, Tan Bui, Pierre Chambon, Yuhao Zhang, Matthew P. Lungren, Andrew Y. Ng, Curtis P. Langlotz, and Pranav Rajpurkar. 2021. Radgraph: Extracting clinical entities and relations from radiology reports. Preprint, arXiv:2106.14… — [arXiv:2106.14463](https://arxiv.org/abs/2106.14463) find
  10. Haibo Jin, Haoxuan Che, Yi Lin, and Hao Chen. 2024. Promptmrg: Diagnosis-driven prompts for medical report generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 2607–2615. find
  11. Alistair E. W. Johnson, Tom J. Pollard, Nathaniel R. Greenbaum, Matthew P. Lungren, Chih ying Deng, Yifan Peng, Zhiyong Lu, Roger G. Mark, Seth J. Berkowitz, and Steven Horng. 2019. Mimic-cxr-jpg, a large publicly available database of labeled chest radiographs. Preprint, arXiv:1901.07042. — [arXiv:1901.07042](https://arxiv.org/abs/1901.07042) find
  12. Elia Kaufmann, Leonard Bauersfeld, Antonio Loquercio, Matthias Müller, Vladlen Koltun, and Davide Scaramuzza. 2023. Champion-level drone racing using deep reinforcement learning. Nature, 620(7976):982–987. find
  13. Mingjie Li, Haokun Lin, Liang Qiu, Xiaodan Liang, Ling Chen, Abdulmotaleb Elsaddik, and Xiaojun Chang. 2024. Contrastive learning with counterfactual explanations for radiology report generation. Preprint, arXiv:2407.14474. — [arXiv:2407.14474](https://arxiv.org/abs/2407.14474) find
  14. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  15. Kang Liu, Zhuoqi Ma, Xiaolu Kang, Yunan Li, Kun Xie, Zhicheng Jiao, and Qiguang Miao. 2025. Enhanced contrastive learning with multi-view longitudinal data for chest x-ray report generation. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 10348–10359. IEEE. find
  16. Sophie Ostmeier, Justin Xu, Zhihong Chen, Maya Varma, Louis Blankemeier, Christian Bluethgen, Arne Edward Michalson Md, Michael Moseley, Curtis Langlotz, Akshay S Chaudhari, and Jean-Benoit Delbrouck. 2024. Green: Generative radiology report evaluation and error notation. In Findings of the Associat… find
  17. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
  18. Vu Minh Hieu Phan, Yutong Xie, Yuankai Qi, Lingqiao Liu, Liyang Liu, Bowen Zhang, Zhibin Liao, Qi Wu, Minh-Son To, and Johan W Verjans. 2024. Decomposing disease descriptions for enhanced pathology detection: A multi-aspect vision-language pre-training framework. In Proceedings of the IEEE/CVF Confe… find
  19. Han Qin and Yan Song. 2022. Reinforced cross-modal alignment for radiology report generation. In Findings of the Association for Computational Linguistics: ACL 2022, pages 448–458, Dublin, Ireland. Association for Computational Linguistics. find
  20. Akshay Smit, Saahil Jain, Pranav Rajpurkar, Anuj Pareek, Andrew Y. Ng, and Matthew P. Lungren. 2020. Chexbert: Combining automatic labelers and expert annotations for accurate radiology report labeling using bert. Preprint, arXiv:2004.09167.
  21. Tim Tanida, Philip Müller, Georgios Kaissis, and Daniel Rueckert. 2023. Interactive and explainable region-guided radiology report generation. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 7433–7442. IEEE. find
  22. Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. 2023. Medklip: Medical knowledge enhanced language-image pre-training. Proceedings of the IEEE/CVF International Conference on Computer Vision. find
  23. Ting Xiao, Lei Shi, Peng Liu, Zhe Wang, and Chenjia Bai. 2024. Radiology report generation via multi-objective preference optimization. Preprint, arXiv:2412.08901. — [arXiv:2412.08901](https://arxiv.org/abs/2412.08901) find
  24. Ting Xiao, Lei Shi, Yang Zhang, HaoFeng Yang, Zhe Wang, and Chenjia Bai. 2025. Online iterative self-alignment for radiology report generation. Preprint, arXiv:2505.11983. find
  25. Wei Xiong, Hanze Dong, Chenlu Ye, Ziqi Wang, Han Zhong, Heng Ji, Nan Jiang, and Tong Zhang. 2024. Iterative preference learning from human feedback: Bridging theory and practice for rlhf under kl-constraint. Preprint, arXiv:2312.11456.
  26. Heng Yin, Shanlin Zhou, Pandong Wang, Zirui Wu, and Yongtao Hao. 2025. KIA: Knowledge-guided implicit vision-language alignment for chest X-ray report generation. In Proceedings of the 31st International Conference on Computational Linguistics, pages 4096–4108, Abu Dhabi, UAE. Association for Comput… find
  27. Feiyang Yu, Mark Endo, Rayan Krishnan, Ian Pan, Andy Tsai, Eduardo Pontes Reis, Eduardo Kaiser Ururahy Nunes Fonseca, Henrique Min Ho Lee, Zahra Shakeri Hossein Abad, Andrew Y. Ng, Curtis P. Langlotz, Vasantha Kumar Venugopal, and Pranav Rajpurkar. 2022. Evaluating progress in automatic chest x-ray… find
  28. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. Preprint, arXiv:1904.09675. find
  29. Xi Zhang, Zaiqiao Meng, Jake Lever, and Edmond S. L. Ho. 2025. Libra: Leveraging temporal images for biomedical radiology analysis. Preprint, arXiv:2411.19378. find
  30. Hong-Yu Zhou, Julián Nicolás Acosta, Subathra Adithan, Suvrankar Datta, Eric J. Topol, and Pranav Rajpurkar. 2025a. Medversa: A generalist foundation model for medical image interpretation. Preprint, arXiv:2405.07988. — [arXiv:2405.07988](https://arxiv.org/abs/2405.07988) find
  31. Qin Zhou, Guoyan Liang, Xindi Li, Jingyuan Chen, Wang Zhe, Chang Yao, and Sai Wu. 2025b. Learnable retrieval enhanced visual-text alignment and fusion for radiology report generation. Preprint, arXiv:2507.07568. — [arXiv:2507.07568](https://arxiv.org/abs/2507.07568) find
  32. Zijian Zhou, Miaojing Shi, Meng Wei, Oluwatosin Alabi, Zijie Yue, and Tom Vercauteren. 2024. Large model driven radiology report generation with clinical quality reinforcement learning. Preprint, arXiv:2403.06728. — [arXiv:2403.06728](https://arxiv.org/abs/2403.06728) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhou-2026-enhancing-reinforcement-learning-radiology-report/