Elamur Research⌕
☰
home / papers / 2026 / ProMed: Shapley Information Gain Guided Reinforcement Learning for Proactive Medical LLMs

ProMed: Shapley Information Gain Guided Reinforcement Learning for Proactive Medical LLMs

Paper · 2026 · Hongxin Ding, Baixiang Huang, Yue Fang et al.

Summary

Most medical LLMs answer straight away from the patient's first description. When that description is incomplete, this can produce misdiagnoses. ProMed is a reinforcement learning framework that trains models to ask follow-up questions and collect missing clinical details before giving an answer. Its core is the Shapley Information Gain (SIG) reward. Each patient case is split into atomic facts, and an LLM fact-checker records which facts appear in the model's stated understanding after every turn. Shapley values then weight each fact by its importance in combination with other facts, rather than treating facts as independent. Training has two stages. First, Monte Carlo Tree Search guided by SIG builds high-reward dialogue trajectories for a supervised warm-up. Second, GRPO is extended so that reward is shared among questions according to their SIG utility, rather than spread evenly over tokens. On partial-information medical benchmarks, ProMed clearly beats earlier methods and the reactive baseline, and it transfers to out-of-domain data. This gives a principled way to reward good questions, a step toward safer interactive clinical assistants.

Key points

  • On average, ProMed outperforms state-of-the-art methods by 6.29% and the reactive (answer-immediately) paradigm by 54.45%.
  • The setup includes in-domain MedQA and out-of-domain CMB. The datasets also include CMB-Clin and Abg-CoQA, which were used for further checks on open-ended and general-domain tasks.
  • Results stay stable across different patient-simulator variants, so the gains do not depend on one particular simulator.
  • Both LLM evaluators and clinicians judged the generated questions against five criteria (Safety and Appropriateness, Clarity and Specificity, Logical Flow, Medical Relevance, and Helpfulness), and the full two-stage model achieved the top scores.
  • The authors release code (github.com/hxxding/ProMed) and two benchmarks for interactive medical questioning with standardized splits. They tested only models of up to 8B parameters, in simulated dialogues with text-only data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China315 works · 43,268 citations🎓University of Hong KongPok Fu Lam · China118 works · 13,496 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksMedicine · 275 worksHealth Informatics · 214 works

Cite

Cite as Hongxin Ding, Baixiang Huang, Yue Fang, Weibin Liao, Xinke Jiang, Jinyang Zhang, Yinghao Zhu, Zheng Li, Liantao Ma, Junfeng Zhao and Yasha Wang. 2026. ProMed: Shapley Information Gain Guided Reinforcement Learning for Proactive Medical LLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1500

@inproceedings{ding2026promed,
  title = {{ProMed: Shapley Information Gain Guided Reinforcement Learning for Proactive Medical LLMs}},
  author = {Hongxin Ding, Baixiang Huang, Yue Fang, Weibin Liao, Xinke Jiang, Jinyang Zhang, Yinghao Zhu, Zheng Li, Liantao Ma, Junfeng Zhao and Yasha Wang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1500},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1500}
}

References

53 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Moloud Balafar, Mahboub Pouraghaei, Mahnaz Ran-jkesh, Mahshid Dehghan, Ali Delkhorrami, and Samad Shams Vahdati. 2024. Comparison of the diagnostic value of ultrasound with chest ct scan in patients with unspecified pulmonary pneumonia in the emergency department. Journal of Emergency Practice and T… find
  2. Lynn Bickley and Peter G Szilagyi. 2012. Bates’ guide to physical examination and history-taking. Lippincott Williams & Wilkins. find
  3. Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, Jianye Hou, and Benyou Wang. 2024. Huatuogpt-o1, towards medical complex reasoning with llms. arXiv preprint arXiv:2412.18925.
  4. Rémi Coulom. 2006. Efficient selectivity and backup operators in monte-carlo tree search. In International conference on computers and games, pages 72–83. Springer. find
  5. DeepSeek-AI. 2025. Deepseek-v3.2: Pushing the frontier of open large language models.
  6. Hongxin Ding, Yue Fang, Runchuan Zhu, Xinke Jiang, Jinyang Zhang, Yongxin Xu, Xu Chu, Junfeng Zhao, and Yasha Wang. 2024. 3ds: Decomposed difficulty data selection’s case study on llm medical domain adaptation. arXiv preprint arXiv:2410.10901. — [arXiv:2410.10901](https://arxiv.org/abs/2410.10901) find
  7. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv e-prints, pages arXiv–2407.
  8. Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. 1996. Reinforcement learning: A survey. Journal of artificial intelligence research, 4:237–285. find
  9. Timo Kaufmann, Paul Weng, Viktor Bengs, and Eyke Hüllermeier. 2024. A survey of reinforcement learning from human feedback.
  10. CJ Eastmond and V Wright. 1979. The nail dystrophy of psoriatic arthritis. Annals of the Rheumatic Diseases, 38(3):226–228. find
  11. Levente Kocsis and Csaba Szepesvári. 2006. Bandit based monte-carlo planning. In European conference on machine learning, pages 282–293. Springer. find
  12. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  13. Suzanne Kurtz, Juliet Draper, and Jonathan Silverman. 2017. Teaching and learning communication skills in medicine. CRC press. find
  14. Meiqi Guo, Mingda Zhang, Siva Reddy, and Malihe Alikhani. 2021. Abg-coqa: Clarifying ambiguity in conversational question answering. In 3rd Conference on Automated Knowledge Base Construction. find
  15. Jiwoo Hong, Noah Lee, and James Thorne. 2024. Orpo: Monolithic preference optimization without reference model. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 11170–11189. find
  16. Zhiyuan Hu, Chumin Liu, Xidong Feng, Yilun Zhao, See-Kiong Ng, Anh Tuan Luu, Junxian He, Pang Wei W Koh, and Bryan Hooi. 2024. Uncertainty of thoughts: Uncertainty-aware planning enhances information seeking in llms. Advances in Neural Information Processing Systems, 37:24181–24215. find
  17. Saket Jha, Rajiv Ranjan Kumar, Aadhaar Dhooria, and Aman Sharma. 2019. Nail pitting: a key clinical sign of psoriatic arthritis. Rheumatology, 58(12):2250– 2250. find
  18. Xinke Jiang, Ruizhe Zhang, Yongxin Xu, Rihong Qiu, Yue Fang, Zhiyuan Wang, Jinyi Tang, Hongxin Ding, Xu Chu, Junfeng Zhao, and Yasha Wang. 2025. HyKGE: A hypothesis knowledge graph enhanced RAG framework for accurate and reliable medical LLMs responses. In Proceedings of the 63rd Annual Meeting of t… find
  19. Suzanne M Kurtz and Jonathan D Silverman. 1996. The calgary—cambridge referenced observation guides: an aid to defining the curriculum and organizing the teaching in communication training programmes. Medical education, 30(2):83–89. find
  20. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
  21. Dong-Ho Lee, Hyundong Cho, Jonathan May, and Jay Pujara. 2025. What is a good question? utility estimation with llm-based simulations. arXiv preprint arXiv:2502.17383. — [arXiv:2502.17383](https://arxiv.org/abs/2502.17383) find
  22. Stella Li, Vidhisha Balachandran, Shangbin Feng, Jonathan Ilgen, Emma Pierson, Pang Wei W Koh, and Yulia Tsvetkov. 2024. Mediq: Question-asking llms and a benchmark for reliable interactive clinical reasoning. Advances in Neural Information Processing Systems, 37:28858–28888.
  23. Weibin Liao, Xu Chu, and Yasha Wang. 2025a. Tpo: Aligning large language models with multi-branch & multi-step preference trees. In The Thirteenth International Conference on Learning Representations. find
  24. Weibin Liao, Xin Gao, Tianyu Jia, Rihong Qiu, Yifan Zhu, Yang Lin, Xu Chu, Junfeng Zhao, and Yasha Wang. 2025b. Learnat: Learning nl2sql with ast-guided task decomposition for large language models. arXiv preprint arXiv:2504.02327. — [arXiv:2504.02327](https://arxiv.org/abs/2504.02327) find
  25. Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2021. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):6421.
  26. Weibin Liao, Tianlong Wang, Yinghao Zhu, Yasha Wang, Junyi Gao, and Liantao Ma. Magical: Medical lay language generation via semantic invariance and layperson-tailored adaptation. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. Alistair EW Johnson, Lucas Bulgarelli, L… — MIMIC-IV, a freely accessible electronic health record dataset
  27. Yusheng Liao, Yutong Meng, Hongcheng Liu, Yanfeng Wang, and Yu Wang. 2023. An automatic evaluation framework for multi-turn medical consultations capabilities of large language models. arXiv preprint arXiv:2309.02077. — [arXiv:2309.02077](https://arxiv.org/abs/2309.02077) find
  28. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81.
  29. Ruoyu Liu, Kui Xue, Xiaofan Zhang, and Shaoting Zhang. 2025a. Interactive evaluation for medical llms via task-oriented dialogue system. In Proceedings of the 31st International Conference on Computational Linguistics, pages 4871–4896. find
  30. Zijie Liu, Xinyu Zhao, Jie Peng, Zhuangdi Zhu, Qingyu Chen, Xia Hu, and Tianlong Chen. 2025b. Dialogue is better than monologue: Instructing medical llms via strategical conversations. arXiv preprint arXiv:2501.17860. — [arXiv:2501.17860](https://arxiv.org/abs/2501.17860) find
  31. Davide Mazzaccara, Alberto Testoni, and Raffaella Bernardi. 2024. Learning to ask informative questions: Enhancing llms with preference optimization and expected information gain. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 5064–5074. find
  32. Daniel McDuff, Mike Schaekermann, Tao Tu, Anil Palepu, Amy Wang, Jake Garrison, Karan Singhal, Yash Sharma, Shekoofeh Azizi, Kavita Kulkarni, and 1 others. 2025. Towards accurate differential diagnosis with large language models. Nature, pages 1–7. find
  33. OpenAI. 2023. Gpt-4 technical report. abs/2303.08774.
  34. ArXiv, Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
  35. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728–53741.
  36. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. find
  37. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  38. Jinsheng Shi, Yuyu Yuan, Ao Wang, and Meng Nie. 2025. Fine-tuning a personalized openbiollm using offline reinforcement learning. Applied Sciences (2076-3417), 15(5). find
  39. Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R Pfohl, Heather Cole-Lewis, and 1 others. 2025. Toward expert-level medical question answering with large language models. Nature Medicine, 31(3):943–950. find
  40. Matthew J Snyder, Marjorie Guthrie, and Stephen Cagle. 2018. Acute appendicitis: efficient diagnosis and management. American family physician, 98(1):25– 33. find
  41. Richard S Sutton, Andrew G Barto, and 1 others. 1999. Reinforcement learning. Journal of Cognitive Neuroscience, 11(1):126–134. find
  42. Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, and 1 others. 2025a. Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for llm reasoning. arXiv preprint arXiv:2506.01939.
  43. Xidong Wang, Guiming Chen, Song Dingjie, Zhang Zhiyi, Zhihong Chen, Qingying Xiao, Junying Chen, Feng Jiang, Jianquan Li, Xiang Wan, and 1 others. 2024. Cmb: A comprehensive medical benchmark in chinese. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Compu…
  44. Ziyu Wang, Hao Li, Di Huang, Hye-Sung Kim, Chae-Won Shin, and Amir M Rahmani. 2025b. Healthq: Unveiling questioning capabilities of llm chains in healthcare conversations. Smart Health, page 100570. find
  45. Eyal Winter. 2002. The shapley value. Handbook of game theory with economic applications, 3:2025– 2054. find
  46. Chaoyi Wu, Weixiong Lin, Xiaoman Zhang, Ya Zhang, Weidi Xie, and Yanfeng Wang. 2024. Pmc-llama: toward building open-source language models for medicine. Journal of the American Medical Informatics Association, 31(9):1833–1843. find
  47. Hongling Xu, Qi Zhu, Heyuan Deng, Jinpeng Li, Lu Hou, Yasheng Wang, Lifeng Shang, Ruifeng Xu, and Fei Mi. 2025a. Kdrl: Post-training reasoning llms via unified knowledge distillation and reinforcement learning. arXiv preprint arXiv:2506.02208. — [arXiv:2506.02208](https://arxiv.org/abs/2506.02208) find
  48. Yongxin Xu, Xinke Jiang, Xu Chu, Rihong Qiu, Yujie Feng, Hongxin Ding, Junfeng Zhao, Yasha Wang, and Bing Xie. 2025b. Dearllm: Enhancing personalized healthcare via large language models-deduced feature correlations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages… find
  49. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  50. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, and 1 others. 2025. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476.
  51. Hongbo Zhang, Junying Chen, Feng Jiang, Fei Yu, Zhihong Chen, Jianquan Li, Guiming Chen, Xiangbo Wu, Zhiyi Zhang, Qingying Xiao, and 1 others. 2023. Huatuogpt, towards taming language model to be a doctor. arXiv preprint arXiv:2305.15075.
  52. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, and Zheyan Luo. 2024. LlamaFactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 400–410, Bangkok, Thail…
  53. Jiayuan Zhu and Junde Wu. 2025. Ask patients with patience: Enabling llms for human-centric medical dialogue with grounded reasoning. arXiv preprint arXiv:2502.07143.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ding-2026-promed-shapley-information-gain-guided/