Elamur Research⌕
☰
home / papers / 2026 / ClinAlign: Scaling Healthcare Alignment from Clinician Preference

ClinAlign: Scaling Healthcare Alignment from Clinician Preference

Paper · 2026 · Shiwei Lyu, Xidong Wang, Hao Zhu et al.

Summary

Medical LLMs score well on exam-style tests, but their free-form answers often miss what clinicians actually expect. Typical alignment signals, such as generic helpfulness or loosely grounded LLM judges, are too coarse for this. The authors first show that plain SFT on HealthBench rubric-aware responses saturates fast and fails on held-out questions. They then build a two-stage, rubric-based RL framework. In the first stage, GPT-5.1 drafts per-query rubrics for real medical prompts drawn from public preference data, and physicians select, correct and extend them to produce HealthRubrics. In the second stage, recurring rubric patterns are condensed into HealthPrinciples, a reusable library. It is used in two ways: to synthesize rubrics for unlabeled queries, which scales supervision, and as an inference-time tool that guides self-revision. Physician-edited rubrics beat automatic drafts, and principle-generated rubrics match or slightly exceed them. The gains also carry over to general open-ended tasks. The work matters because it gives small, efficient models a practical way to reach frontier-level clinical alignment without heavy per-example expert annotation.

Key points

  • HealthRubrics: 7,034 physician-verified preference examples. They were selected from about 103,575 queries pooled from Chatbot Arena releases (human-140k, human-55k, expert-5k) and HelpSteer3-Preference.
  • HealthPrinciples: 119 clinician-curated principles, organized according to how urgent and how uncertain a case is, how much expertise the user has, and what kind of task is being requested. They were used to generate rubrics for 16,872 additional real user medical queries.
  • Small-model gains: Training Qwen3-4B-Instruct on HealthRubrics raised HealthBench-Hard from 5.2% to 22.9%, above GPT-5.1-Instant at 20.8%.
  • Headline result: A 30B mixture model with 3B active parameters reached 33.4% on HealthBench-Hard, ahead of DeepSeek-R1 and o3. It also performed strongly on Arena-Hard-v2, while LLMEval-Med reasoning scores barely moved.
  • Scorer and scaling findings:
    • Qwen3-32B was chosen as the reward scorer because its agreement with the GPT-4.1 reference (87.6%) is nearly the same as Qwen3-235B's (87.9%).
    • Under a fixed compute budget, adding more distinct questions helped more than adding more epochs.
    • Repeated calls to the inference-time tool kept improving scores, but the gains levelled off after a few calls.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China244 works · 39,359 citations🎓Chinese University of Hong Kong, ShenzhenShenzhen · China47 works · 2,596 citations🏢Ant GroupHangzhou · China42 works · 1,131 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksMedicine · 241 worksHealth Informatics · 184 works

Cite

Cite as Shiwei Lyu, Xidong Wang, Hao Zhu, Lei Liu, Chaohe Zhang, Jian Wang, Jinjie Gu, Benyou Wang and Yue Shen. 2026. ClinAlign: Scaling Healthcare Alignment from Clinician Preference. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.825

@inproceedings{lyu2026clinalign,
  title = {{ClinAlign: Scaling Healthcare Alignment from Clinician Preference}},
  author = {Shiwei Lyu, Xidong Wang, Hao Zhu, Lei Liu, Chaohe Zhang, Jian Wang, Jinjie Gu, Benyou Wang and Yue Shen},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.825},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.825}
}

References

40 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Asma Ben Abacha and Dina Demner Fushman. 2019. A question-entailment approach to question answering. BMC Bioinform., 20(1):511:1–511:23. find
  2. Rahul K Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin Quiñonero-Candela, Foivos Tsimpourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, and 1 others. 2025. Healthbench: Evaluating large language models towards improved human health. arXiv preprint arXiv:2505.08775.
  3. Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, Jianye Hou, and Benyou Wang. 2024. Huatuogpt-o1, towards medical complex reasoning with llms. arXiv preprint arXiv:2412.18925. find
  4. Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E. Gonzalez, and Ion Stoica. 2024. Chatbot arena: An open platform for evaluating llms by human preference. Preprint, arXiv:2403.04132.
  5. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan-Jiang Jiang, and 3416 other…
  6. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2026. Deepseek-r1: Incentivizing reasoning capability in ll…
  7. Chengfeng Dou, Chong Liu, Fan Yang, Fei Li, Jiyuan Jia, Mingyang Chen, Qiang Ju, Shuai Wang, Shunya Dang, Tianpeng Li, and 1 others. 2025. Baichuanm2: Scaling medical capability with large verifier system. arXiv preprint arXiv:2509.02208.
  8. Jacob Eisenstein, Chirag Nagpal, Alekh Agarwal, Ahmad Beirami, Alex D’Amour, DJ Dvijotham, Adam Fisch, Katherine Heller, Stephen Pfohl, Deepak Ramachandran, and 1 others. 2023. Helping or herding? reward model ensembles mitigate but do not eliminate reward hacking. arXiv preprint arXiv:2312.09244. — [arXiv:2312.09244](https://arxiv.org/abs/2312.09244) find
  9. Dennis Fast, Lisa C Adams, Felix Busch, Conor Fallon, Marc Huppertz, Robert Siepmann, Philipp Prucker, Nadine Bayerl, Daniel Truhn, Marcus Makowski, and 1 others. 2024. Autonomous medical evaluation for guideline adherence of large language models. NPJ Digital Medicine, 7(1):358. find
  10. Jiayi Fu, Xuandong Zhao, Chengyuan Yao, Heng Wang, Qi Han, and Yanghua Xiao. 2025. Reward shaping to mitigate reward hacking in rlhf. arXiv preprint arXiv:2502.18770. — [arXiv:2502.18770](https://arxiv.org/abs/2502.18770) find
  11. Anisha Gunjal, Anthony Wang, Elaine Lau, Vaskar Nath, Yunzhong He, Bing Liu, and Sean Hendryx. 2025. Rubrics as rewards: Reinforcement learning beyond verifiable domains. arXiv preprint arXiv:2507.17746. find
  12. Dulhan Jayalath, Shashwat Goel, Thomas Foster, Parag Jain, Suchin Gururangan, Cheng Zhang, Anirudh Goyal, and Alan Schelten. 2025. Compute as teacher: Turning inference compute into reference-free supervision. arXiv preprint arXiv:2509.14234. — [arXiv:2509.14234](https://arxiv.org/abs/2509.14234) find
  13. Shuai Zhen, Jian Wang, and 1 others. 2025. Evolving diagnostic agents in a virtual clinical environment. arXiv preprint arXiv:2510.24654. — [arXiv:2510.24654](https://arxiv.org/abs/2510.24654) find
  14. AI Scale. 2025. Vista: Visual-language understanding leaderboard, 2025. URL https://scale.com/leaderboard/visual_language_understanding. Accessed, 3. ↗
  15. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. Preprint, arXiv:2402.03300.
  16. Yongnan Jin, Xurui Li, Feng Cao, Liucun Gao, and Juanjuan Yao. 2025. Multidimensional rubric-oriented reward model learning via geometric projection reference constraints. arXiv preprint arXiv:2511.16139. — [arXiv:2511.16139](https://arxiv.org/abs/2511.16139) find
  17. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2025. Hybridflow: A flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, EuroSys ’25, page 1279–1297. ACM.
  18. Shreya Johri, Jaehwan Jeong, Benjamin A Tran, Daniel I Schlessinger, Shannon Wongvibulsin, Leandra A Barnes, Hong-Yu Zhou, Zhuo Ran Cai, Eliezer M Van Allen, David Kim, and 1 others. 2025. An evaluation framework for clinical use of large language models in patient interaction tasks. Nature medicine… find
  19. Giulio Starace, Oliver Jaffe, Dane Sherburn, James Aung, Jun Shern Chan, Leon Maksin, Rachel Dias, Evan Mays, Benjamin Kinsella, Wyatt Thompson, and 1 others. 2025. Paperbench: Evaluating ai’s ability to replicate ai research. arXiv preprint arXiv:2504.01848. find
  20. Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E. Gonzalez, and Ion Stoica. 2024. From crowdsourced data to high-quality benchmarks: Arena-hard and benchbuilder pipeline. Preprint, arXiv:2406.11939.
  21. Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, and 1 others. 2025. Kimi k2: Open agentic intelligence. arXiv preprint arXiv:2507.20534.
  22. Yunxiang Li, Zihan Li, Kai Zhang, Ruilong Dan, Steve Jiang, and You Zhang. 2023. Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge. Preprint, arXiv:2303.14070.
  23. Pengkai Wang, Pengwei Liu, Zhijie Sang, Congkai Xie, Hongxia Yang, and 1 others. 2025a. Infimed-orbit: Aligning llms on open-ended complex tasks via rubric-based incremental training. arXiv preprint arXiv:2510.15859. — [arXiv:2510.15859](https://arxiv.org/abs/2510.15859) find
  24. Bill Yuchen Lin, Yuntian Deng, Khyathi Chandu, Faeze Brahman, Abhilasha Ravichander, Valentina Pyatkin, Nouha Dziri, Ronan Le Bras, and Yejin Choi. 2024. Wildbench: Benchmarking llms with challenging tasks from real users in the wild. arXiv preprint arXiv:2406.04770.
  25. Xidong Wang, Guiming Chen, Song Dingjie, Zhang Zhiyi, Zhihong Chen, Qingying Xiao, Junying Chen, Feng Jiang, Jianquan Li, Xiang Wan, and 1 others. 2024a. Cmb: A comprehensive medical benchmark in chinese. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Comp…
  26. Yujin Oh, Sangjoon Park, Hwa Kyung Byun, Yeona Cho, Ik Jae Lee, Jin Sung Kim, and Jong Chul Ye. 2024. Llm-driven multimodal target volume contouring in radiation oncology. Nature Communications, 15(1):9186. find
  27. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, and 262 others.… — GPT-4 Technical Report
  28. Pengcheng Qiu, Chaoyi Wu, Junwei Liu, Qiaoyu Zheng, Yusheng Liao, Haowen Wang, Yun Yue, Qianrui Fan, Xidong Wang, Nuo Chen, Junyin Chen, Yidong Wang, Guorui Zhen, Chunxian Zhang, Xiangbo Wu, Yan Hu, Anningzhe Gao, Xiang Wan, and 1 others. 2024b. Apollo: A lightweight multilingual medical llm towards… — [arXiv:2403.03640](https://arxiv.org/abs/2403.03640) find
  29. Xidong Wang, Jianquan Li, Shunian Chen, Yuxuan Zhu, Xiangbo Wu, Zhiyi Zhang, Xiaolong Xu, Junying Chen, Jie Fu, Xiang Wan, and 1 others. 2025b. Huatuo-26m, a large-scale chinese medical qa dataset. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 3828–3848. find
  30. Zhilin Wang, Jiaqi Zeng, Olivier Delalleau, Hoo-Chang Shin, Felipe Soares, Alexander Bukharin, Ellie Evans, Yi Dong, and Oleksii Kuchaiev. 2025c. Helpsteer3-preference: Open human-annotated preference data across diverse tasks and languages. Preprint, arXiv:2505.11475. — [arXiv:2505.11475](https://arxiv.org/abs/2505.11475) find
  31. Yuxin Zuo, Shang Qu, Yifei Li, Zhangren Chen, Xuekai Zhu, Ermo Hua, Kaiyan Zhang, Ning Ding, and Bowen Zhou. 2025. Medxpertqa: Benchmarking expert-level medical reasoning and understanding. arXiv preprint arXiv:2501.18362. find
  32. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  33. Xi Yang, Aokun Chen, Nima PourNejatian, Hoo Chang Shin, Kaleb E Smith, Christopher Parisien, Colin Compas, Cheryl Martin, Mona G Flores, Ying Zhang, and 1 others. 2022. Gatortron: A large clinical language model to unlock patient information from unstructured electronic health records. arXiv preprin… — [arXiv:2203.03540](https://arxiv.org/abs/2203.03540) find
  34. Kaiyan Zhang, Sihang Zeng, Ermo Hua, Ning Ding, Zhang-Ren Chen, Zhiyuan Ma, Haoxin Li, Ganqu Cui, Biqing Qi, Xuekai Zhu, Xingtai Lv, Hu Jinfang, Zhiyuan Liu, and Bowen Zhou. 2024. Ultramedical: Building specialized generalists in biomedicine. Preprint, arXiv:2406.03949. — [arXiv:2406.03949](https://arxiv.org/abs/2406.03949) find
  35. Ming Zhang, Yujiong Shen, Zelin Li, Huayu Sha, Binze Hu, Yuhui Wang, Chenhao Huang, Shichun Liu, Jingqi Tong, Changhao Jiang, and 1 others. 2025. Llmeval-med: A real-world clinical benchmark for medical llms with physician validation. arXiv preprint arXiv:2506.04078. find
  36. Weike Zhao, Chaoyi Wu, Yanjie Fan, Xiaoman Zhang, Pengcheng Qiu, Yuze Sun, Xiao Zhou, Yanfeng Wang, Xin Sun, Ya Zhang, and 1 others. 2025. An agentic system for rare disease diagnosis with traceable reasoning. arXiv preprint arXiv:2506.20430. — [arXiv:2506.20430](https://arxiv.org/abs/2506.20430) find
  37. Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024. Wildchat: 1m chatgpt interaction logs in the wild. Preprint, arXiv:2405.01470.
  38. Juexiao Zhou, Haoyang Li, Siyuan Chen, Zhangtianyi Chen, Zhongyi Han, and Xin Gao. 2025a. Large language models in biomedicine and healthcare. npj Artificial Intelligence, 1(1):44. find
  39. Yang Zhou, Sunzhu Li, Shunyu Liu, Wenkai Fang, Kongcheng Zhang, Jiale Zhao, Jingwen Yang, Yihe Zhou, Jianwei Lv, Tongya Zheng, and 1 others. 2025b. Breaking the exploration bottleneck: Rubric-scaffolded reinforcement learning for general llm reasoning. arXiv preprint arXiv:2508.16949. — [arXiv:2508.16949](https://arxiv.org/abs/2508.16949) find
  40. Yuxuan Zhou, Yubin Wang, Bin Wang, Chen Ning, Xien Liu, Ji Wu, and Jianye Hao. 2025c. Enhancing the medical context-awareness ability of llms via multifaceted self-refinement learning. arXiv preprint arXiv:2511.10067. — [arXiv:2511.10067](https://arxiv.org/abs/2511.10067) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lyu-2026-clinalign-scaling-healthcare-alignment-clinician/