Elamur Research⌕
☰
home / papers / 2026 / Evo-PI: Aligning Medical Reasoning via Evolving Principle-Guided Supervision

Evo-PI: Aligning Medical Reasoning via Evolving Principle-Guided Supervision

Paper · 2026 · Xianda Zheng, Huan Gao, Meng-Fen Chiang et al.

Summary

Multimodal LLMs trained for reasoning usually get static supervision. Fixed prompts, rules or scalar rewards check answer format and correctness but do little to shape how the model reasons, which leads to plateaus, fragile generalization and reward hacking. Evo-PI addresses this by turning reasoning principles into explicit natural-language supervision that changes during training. A frozen text LLM first drafts principles for each medical image type (e.g., CT, MR) and refines them using a few sample question–answer pairs. During RL, the MLLM produces reasoning chains and answers. A frozen judge LLM then scores how well these follow the principles (Principle Reward) and how complete the logic is (Thinking Point Reward), and these scores are added to the standard RL reward. After each round, failure cases are used to revise and extend the principle bank, so the model and its principles co-evolve until convergence. On medical VQA across eight benchmarks and several backbones, the approach raises accuracy by up to 24.6% and gives more clinically coherent explanations. This points toward adaptive, interpretable supervision for expert domains.

Key points

  • The framework has three stages: Principle Bank Initialization, Principle-Guided Reinforcement Learning, and Principle Evolution.
  • Two frozen models support training. A "knowledgeable" text-only LLM creates and updates principles, which keeps this step independent of the backbone architecture and modality. A judge LLM converts principle adherence into dense rewards.
  • The rewards derived from principles are merged with the underlying environment reward used for RL (GRPO-style optimization). The authors report stable gains across different RL algorithms and backbones.
  • Accuracy improves by up to 24.6% over static-supervision training across eight medical VQA benchmarks. A case study shows Evo-PI correcting a wrong Med-R1 answer on a CT scan through step-by-step, CT-specific reasoning.
  • Stated limitations:
    • Principle quality depends on the judge model's knowledge.
    • Training costs more than SFT.
    • The method has only been tested on medical VQA.
  • Code is available at github.com/zhengxianda/Evo_PI.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Harbin Institute of TechnologyHarbin · China185 works · 29,140 citations🎓Beijing University of Posts and TelecommunicationsBeijing · China78 works · 33,365 citations🎓University of AucklandAuckland · New Zealand15 works · 888 citations🎓National Yang Ming Chiao Tung UniversityHsinchu · Taiwan7 works · 85 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksComputer Vision and Pattern Recognition · 3,486 worksMedicine · 275 worksHealth Informatics · 214 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,685 works

Cite

Cite as Xianda Zheng, Huan Gao, Meng-Fen Chiang, Michael J. Witbrock, Kaiqi Zhao and Shangyang Li. 2026. Evo-PI: Aligning Medical Reasoning via Evolving Principle-Guided Supervision. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1469

@inproceedings{zheng2026evo,
  title = {{Evo-PI: Aligning Medical Reasoning via Evolving Principle-Guided Supervision}},
  author = {Xianda Zheng, Huan Gao, Meng-Fen Chiang, Michael J. Witbrock, Kaiqi Zhao and Shangyang Li},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1469},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1469}
}

References

28 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Ming-Hsuan Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, and 8 others. 2025. Qwen2.5-vl technical report. CoRR, abs/2502.13923…
  2. Junying Chen, Ruyi Ouyang, Anningzhe Gao, Shunian Chen, Guiming Hardy Chen, Xidong Wang, Ruifei Zhang, Zhenyang Cai, Ke Ji, Guangjun Yu, Xiang Wan, and Benyou Wang. 2024. Huatuogpt-vision, towards injecting medical visual knowledge into multimodal llms at scale. CoRR, abs/2406.19280. find
  3. Zhipeng Chen, Xiaobo Qin, Youbin Wu, Yue Ling, Qinghao Ye, Wayne Xin Zhao, and Guang Shi. 2025. Pass@ k training for adaptively balancing exploration and exploitation of large reasoning models. arXiv preprint arXiv:2508.10751. — [arXiv:2508.10751](https://arxiv.org/abs/2508.10751) find
  4. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit S. Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan-Jiang Jiang, and 81 othe… — Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
  5. Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, Zhiyuan Liu, Hao Peng, Lei Bai, Wanli Ouyang, Yu Cheng, Bowen Zhou, and Ning Ding. 2025. The entropy mechanism of reinforcement learning for reasoning language models. CoRR, abs/2…
  6. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, and 175 others. 2025. Deepseek-r1 incentivizes reasoning in llms through rein… find
  7. Yutao Hu, Tianbin Li, Quanfeng Lu, Wenqi Shao, Junjun He, Yu Qiao, and Ping Luo. 2024. Omnimedvqa: A new large-scale comprehensive evaluation benchmark for medical LVLM. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 22170–2218… find
  8. Nabil Issa, Mary Schuller, Susan Santacaterina, Michael Shapiro, Edward Wang, Richard E Mayer, and Debra A DaRosa. 2011. Applying multimedia design principles enhances learning in medical education. Medical education, 45(8):818–826. find
  9. Datasets, methodologies, and evaluations. CoRR, abs/2406.10303. — [arXiv:2406.10303](https://arxiv.org/abs/2406.10303) find
  10. Miao Jing, Mengting Jia, Junling Lin, Zhongxia Shen, Huan Gao, Mingkun Xu, and Shangyang Li. 2026. Beyond classification accuracy: Neural-medbench and the need for deeper reasoning benchmarks. In The Fourteenth International Conference on Learning Representations. find
  11. Xumeng Wen, Zihan Liu, Shun Zheng, Zhijian Xu, Shengyu Ye, Zhirong Wu, Xiao Liang, Yang Wang, Junjie Li, Ziming Miao, Jiang Bian, and Mao Yang. 2025. Reinforcement learning with verifiable rewards implicitly incentivizes correct reasoning in base llms. CoRR, abs/2506.14245. find
  12. Yuxiang Lai, Jike Zhong, Ming Li, Shitian Zhao, and Xiaofeng Yang. 2025. Med-r1: Reinforcement learning for generalizable medical reasoning in vision-language models. CoRR, abs/2503.13939. — [arXiv:2503.13939](https://arxiv.org/abs/2503.13939) find
  13. Jianyu Wu, Hao Yang, Xinhua Zeng, Guibing He, Zhiyu Chen, Zihui Li, Xiaochuan Zhang, Yangyang Ma, Run Fang, and Yang Liu. 2025. Pathvlm-r1: A reinforcement learning-driven reasoning model for pathology visual-language tasks. CoRR, abs/2504.09258. — [arXiv:2504.09258](https://arxiv.org/abs/2504.09258) find
  14. Lei Liu, Xiaoyan Yang, Junchi Lei, Xiaoyang Liu, Yue Shen, Zhiqiang Zhang, Peng Wei, Jinjie Gu, Zhixuan Chu, Zhan Qin, and Kui Ren. 2024. A survey on medical large language models: Technology, application, trustworthiness, and future directions. CoRR, abs/2406.03712. — [arXiv:2406.03712](https://arxiv.org/abs/2406.03712) find
  15. Jocelyn Lockyer, Carol Carraccio, Ming-Ka Chan, Danielle Hart, Sydney Smee, Claire Touchie, Eric S Holmboe, Jason R Frank, and ICBME Collaborators. 2017. Core principles of assessment in competency-based medical education. Medical teacher, 39(6):609–616. find
  16. Tao Lu and Shangyang Li. 2025. Harnessing pre-trained language models for eeg-based epilepsy detection. In 2025 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE. find
  17. Jiazhen Pan, Che Liu, Junde Wu, Fenglin Liu, Jiayuan Zhu, Hongwei Bran Li, Chen Chen, Cheng Ouyang, and Daniel Rueckert. 2025. Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning. CoRR, abs/2502.19634. — [arXiv:2502.19634](https://arxiv.org/abs/2502.19634) find
  18. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. CoRR, abs/1707.06347. find
  19. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. CoRR, abs/2402.03300.
  20. Hanguang Xiao, Feizhong Zhou, Xingyue Liu, Tianqi Liu, Zhipeng Li, Xin Liu, and Xiaoxuan Huang. 2025. A comprehensive survey of large language models and multimodal large language models in medicine. Inf. Fusion, 117:102888. find
  21. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, and 16 others. 2025. DAPO: an open-source LLM reinforcement learning sys…
  22. Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin. 2025. Group sequence policy optimization. CoRR, abs/2507.18071.
  23. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. In NeurIPS.
  24. Weihai Zhi, Jiayan Guo, and Shangyang Li. 2026. Medgr²: Breaking the data barrier for medical reasoning via generative reward learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 28901–28909. find
  25. Joar Skalse, Nikolaus H. R. Howe, Dmitrii Krasheninnikov, and David Krueger. 2022. Defining and characterizing reward hacking. CoRR, abs/2209.13085. — [arXiv:2209.13085](https://arxiv.org/abs/2209.13085) find
  26. Richard S. Sutton. 1988. Learning to predict by the methods of temporal differences. Mach. Learn., 3:9– 44. find
  27. OpenAI Team. 2023. GPT-4 technical report. CoRR, abs/2303.08774.
  28. Jinqiang Wang, Huansheng Ning, Yi Peng, Qikai Wei, Daniel Tesfai, Wenwei Mao, Tao Zhu, and Runhe Huang. 2024. A survey on large language models from general purpose to medical applications: find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zheng-2026-evo-pi-aligning-medical-reasoning/