Elamur Research⌕
☰
home / papers / 2026 / Experience is the Teacher: Reusing Atomic Thoughts from LLMs to Improve Medical Dialogue

Experience is the Teacher: Reusing Atomic Thoughts from LLMs to Improve Medical Dialogue

Paper · 2026 · Guangya Yu, Hui Luo, Qi Ye et al.

Summary

LLMs used as doctor simulators tend to reason in one of two weak ways. Fixed, expert-written reasoning templates are rigid. Long test-time-scaling chains, as in R1-style models, are wasteful. This paper adapts the idea of reusable "atomic thoughts" (small, composable reasoning units) to medical dialogue.

The distillation framework has three stages:

  • clinical experience is collected from human experts as patient intents with static CoT templates;
  • teacher LLMs annotate each dialogue turn with an intent and a sequence of atomic thoughts, and an update mechanism adds new thoughts;
  • filtered and sampled turns are rewritten by the teacher into experience-guided reasoning and revised doctor replies, which are used to fine-tune student models (Qwen3-ReMeDi).

Evaluation covers static response generation and interactive multi-agent consultation, both judged by LLM-as-judge. The tuned students reason with fewer steps and tokens while scoring higher. The transferred experience also lets weak or small teachers produce useful training data, which lowers annotation cost.

Key points

  • Data: training data is built from ReMeDi. Evaluation uses ReMeDi, MVME (interactive setting, following its framework) and an anonymized Private dataset from a real online hospital.
  • Update mechanism: embeddings and an LLM are combined to merge or refine new teacher-generated atomic thoughts, balancing cost against quality.
  • Efficiency on ReMeDi diagnosis (Table 7): the standard thinking prompt (Qwen3-4B) scores 74.09 using about 9.15 steps and 1049 tokens. Tuning on human plus LLM experience scores 75.42 using about 5.82 steps and 451 tokens.
  • Student and teacher scale (Qwen3-4B/8B on MVME):
    • Tuning without teacher annotations falls below the backbone model.
    • Qwen3-4B improves more when distilling from itself than from Qwen3-8B.
  • Limitations: medical knowledge is bounded by what the teacher model already knows. Full-parameter fine-tuning and RL are left for future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Shanghai Jiao Tong UniversityShanghai · China212 works · 19,544 citations🎓East China University of Science and TechnologyShanghai · China14 works · 87 citations🔬Renji HospitalShanghai · China3 works

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 worksMedicine · 275 worksHealth Informatics · 214 works

🧭 Topics 1Research subjects and languages the work is about.Dialogue systems · 654 works

Cite

Cite as Guangya Yu, Hui Luo, Qi Ye, Ruihui Hou, Weiyan Zhang, Mingxi Shang, Xuanwu Li, ChunMing Wang and Tong Ruan. 2026. Experience is the Teacher: Reusing Atomic Thoughts from LLMs to Improve Medical Dialogue. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.957

@inproceedings{yu2026experience,
  title = {{Experience is the Teacher: Reusing Atomic Thoughts from LLMs to Improve Medical Dialogue}},
  author = {Guangya Yu, Hui Luo, Qi Ye, Ruihui Hou, Weiyan Zhang, Mingxi Shang, Xuanwu Li, ChunMing Wang and Tong Ruan},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.957},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.957}
}

References

46 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Sanjeev Arora and Anirudh Goyal. 2023. A theory for emergence of complex skills in language models. arXiv preprint arXiv:2307.15936. — [arXiv:2307.15936](https://arxiv.org/abs/2307.15936) find
  3. Baichuan. 2023. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309.10305.
  4. Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, and Benyou Wang. 2025a. Towards medical complex reasoning with llms through medical verifiable problems. In Findings of the Association for Computational Linguistics: ACL 2025, pages 14552–14573. find
  5. Junying Chen, Chi Gui, Anningzhe Gao, Ke Ji, Xidong Wang, Xiang Wan, and Benyou Wang. 2025b. CoD, towards an interpretable medical agent using chain of diagnosis. In Findings of the Association for Computational Linguistics: ACL 2025, pages 14345–14368, Vienna, Austria. Association for Computational… find
  6. Junying Chen, Xidong Wang, Ke Ji, Anningzhe Gao, Feng Jiang, Shunian Chen, Hongbo Zhang, Song Dingjie, Wenya Xie, Chuyi Kong, Jianquan Li, Xiang Wan, Haizhou Li, and Benyou Wang. 2024. HuatuoGPT-II, one-stage training for medical adaption of LLMs. In First Conference on Language Modeling. find
  7. Aniket Didolkar, Nicolas Ballas, Sanjeev Arora, and Anirudh Goyal. 2025. Metacognitive reuse: Turning recurring llm reasoning into concise behaviors. arXiv preprint arXiv:2509.13237. find
  8. Aniket Didolkar, Anirudh Goyal, Nan Rosemary Ke, Siyuan Guo, Michal Valko, Timothy Lillicrap, Danilo Jimenez Rezende, Yoshua Bengio, Michael C Mozer, and Sanjeev Arora. 2024. Metacognitive capabilities of llms: An exploration in mathematical problem solving. Advances in Neural Information Processing… find
  9. Hongxin Ding, Baixiang Huang, Yue Fang, Weibin Liao, Xinke Jiang, Zheng Li, Junfeng Zhao, and Yasha Wang. 2025. Promed: Shapley information gain guided reinforcement learning for proactive medical llms. arXiv preprint arXiv:2508.13514.
  10. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  11. Yongqi Fan, Xiaoyang Chen, Dezhi Ye, Jie Liu, Haijin Liang, Jin Ma, Ben He, Yingfei Sun, and Tong Ruan. 2026. Tfrank: Think-free reasoning enables practical pointwise llm ranking. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 21020–21028. find
  12. Zhihao Fan, Lai Wei, Jialong Tang, Wei Chen, Wang Siyuan, Zhongyu Wei, and Fei Huang. 2025. Ai hospital: Benchmarking large language models in a multi-agent medical interaction simulator. In Proceedings of the 31st International Conference on Computational Linguistics, pages 10183–10213.
  13. Yichun Feng, Jiawei Wang, Lu Zhou, Zhen Lei, and Yixue Li. 2025. Doctoragent-rl: A multi-agent collaborative reinforcement learning system for multi-turn clinical dialogue. arXiv preprint arXiv:2505.19630. — [arXiv:2505.19630](https://arxiv.org/abs/2505.19630) find
  14. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, and 1 others. 2025a. Deepseekr1 incentivizes reasoning in llms through reinforcement learning. Nature, 645(8081):633–638. find
  15. Guangfu Guo, Kai Zhang, Bryan Hoo, Yujun Cai, Xiaoqian Lu, Nanyun Peng, and Yiwei Wang. 2025b. Structured outputs enable general-purpose llms to be medical experts. arXiv preprint arXiv:2503.03194. — [arXiv:2503.03194](https://arxiv.org/abs/2503.03194) find
  16. Ruihui Hou, Shencheng Chen, Yongqi Fan, Guangya Yu, Lifeng Zhu, Jing Sun, Jingping Liu, and Tong Ruan. 2025. Msdiagnosis: A benchmark and framework for evaluating large language models in multi-step clinical diagnosis. Knowledge-Based Systems, page 114524. find
  17. Minhao Jiang, Ken Ziyu Liu, Ming Zhong, Rylan Schaeffer, Siru Ouyang, Jiawei Han, and Sanmi Koyejo. 2024. Investigating data contamination for pre-training language models. arXiv preprint arXiv:2401.06059. — [arXiv:2401.06059](https://arxiv.org/abs/2401.06059) find
  18. Chengfeng Dou, Chong Liu, Fan Yang, Fei Li, Jiyuan Jia, Mingyang Chen, Qiang Ju, Shuai Wang, Shunya Dang, Tianpeng Li, and 1 others. 2025. Baichuanm2: Scaling medical capability with large verifier system. arXiv preprint arXiv:2509.02208.
  19. Nikhil Khandekar, Qiao Jin, Guangzhi Xiong, Soren Dunn, Serina Applebaum, Zain Anwar, Maame Sarfo-Gyamfi, Conrad Safranek, Abid Anwar, Andrew Zhang, and 1 others. 2025. Medcalc-bench: Evaluating large language models for medical calculations. Advances in Neural Information Processing Systems, 37:847… find
  20. Chengfeng Dou, Ying Zhang, Zhi Jin, Wenpin Jiao, Haiyan Zhao, Yongqiang Zhao, and Zhengwei Tao. 2024. Integrating physician diagnostic logic into large language models: Preference learning from process feedback. In Findings of the Association for Computational Linguistics ACL 2024, pages 2453– 2473. find
  21. Dongdong Li, Zhaochun Ren, Pengjie Ren, Zhumin Chen, Miao Fan, Jun Ma, and Maarten De Rijke. 2021. Semi-supervised variational reasoning for medical dialogue generation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 544–554. find
  22. Shuyue Stella Li, Vidhisha Balachandran, Shangbin Feng, Jonathan S. Ilgen, Emma Pierson, Pang Wei Koh, and Yulia Tsvetkov. 2024. Mediq: Question-asking LLMs and a benchmark for reliable interactive clinical reasoning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.
  23. Yupian Lin, Tong Ruan, Jingping Liu, and Haofen Wang. 2023. A survey on neural data-to-text generation. IEEE Transactions on Knowledge and Data Engineering, 36(4):1431–1449. find
  24. Yupian Lin, Guangya Yu, Cheng Yuan, Huan Du, Hui Luo, Yuang Bian, Jingping Liu, Zhidong He, Wen Du, and Tong Ruan. 2026. Logtop: Logic tree-of-program with table instruction-tuned llms for controlled logical table-to-text generation. In Findings of the Association for Computational Linguistics: EACL… find
  25. Jiaxiang Liu, Yuan Wang, Jiawei Du, Joey Zhou, and Zuozhu Liu. 2024. Medcot: Medical chain of thought via hierarchical expert. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17371–17389. find
  26. Ruoyu Liu, Kui Xue, Xiaofan Zhang, and Shaoting Zhang. 2025. Interactive evaluation for medical llms via task-oriented dialogue system. In Proceedings of the 31st International Conference on Computational Linguistics, pages 4871–4896. find
  27. Wenge Liu, Jianheng Tang, Yi Cheng, Wenjie Li, Yefeng Zheng, and Xiaodan Liang. 2022. Meddg: an entity-centric medical consultation dataset for entity-aware medical dialogue generation. In CCF International Conference on Natural Language Processing and Chinese Computing, pages 447–459. Springer. find
  28. Xiaoming Shi, Zeming Liu, Li Du, Yuxuan Wang, Hongru Wang, Yuhang Guo, Tong Ruan, Jie Xu, Xiaofan Zhang, and Shaoting Zhang. 2024. Medical dialogue system: A survey of categories, methods, evaluation and challenges. In Findings of the Association for Computational Linguistics: ACL 2024, pages 2840–… find
  29. Xiaoming Shi, Zeming Liu, Chuan Wang, Haitao Leng, Kui Xue, Xiaofan Zhang, and Shaoting Zhang. 2023. MidMed: Towards mixed-type dialogues for medical consultation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8145–8157, Tor… find
  30. Nan Wang, Yongqi Fan, ZongYu Wang, Xuezhi Cao, Xinyan He, Haiyun Jiang, Tong Ruan, Jingping Liu, and 1 others. 2025. Kg-o1: enhancing multi-hop question answering in large language models via knowledge graph integration. arXiv preprint arXiv:2508.15790. — [arXiv:2508.15790](https://arxiv.org/abs/2508.15790) find
  31. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  32. Noam Wies, Yoav Levine, and Amnon Shashua. 2023. The learnability of in-context learning. Advances in Neural Information Processing Systems, 36:36637– 36651.
  33. Jiageng Wu, Xian Wu, Yefeng Zheng, and Jie Yang. 2024. Medkp: Medical dialogue with knowledge enhancement and clinical pathway encoding. arXiv preprint arXiv:2403.06611. — [arXiv:2403.06611](https://arxiv.org/abs/2403.06611) find
  34. Siye Wu, Jian Xie, Yikai Zhang, Aili Chen, Kai Zhang, Yu Su, and Yanghua Xiao. 2025. ARM: Adaptive reasoning model. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  35. Guojun Yan, Jiahuan Pei, Pengjie Ren, Zhaochun Ren, Xin Xin, Huasheng Liang, Maarten De Rijke, and Zhumin Chen. 2022. Remedi: Resources for multi-domain, multi-service, medical dialogues. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retriev… find
  36. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025a. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  37. Dingkang Yang, Jinjie Wei, Mingcheng Li, Jiyao Liu, Lihao Liu, Ming Hu, Junjun He, Yakun Ju, Wei Zhou, Yang Liu, and 1 others. 2025b. Medaide: Information fusion and anatomy of medical intents via llm-based agent collaboration. Information Fusion, page 103743. find
  38. Ling Yang, Zhaochen Yu, Bin Cui, and Mengdi Wang. 2025c. Reasonflux: Hierarchical llm reasoning via scaling thought templates. arXiv preprint arXiv:2502.06772. find
  39. Ling Yang, Zhaochen Yu, Tianjun Zhang, Shiyi Cao, Minkai Xu, Wentao Zhang, Joseph E. Gonzalez, and Bin CUI. 2024. Buffer of thoughts: Thought-augmented reasoning with large language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.
  40. Ling Yang, Zhaochen Yu, Tianjun Zhang, Minkai Xu, Joseph E. Gonzalez, Bin CUI, and Shuicheng YAN. 2025d. Supercorrect: Advancing small LLM reasoning with thought template distillation and self-correction. In The Thirteenth International Conference on Learning Representations. find
  41. Qi Ye, Guangya Yu, Jingping Liu, Erzhen Chen, Chenjie Dong, Xiaosheng Lin, Zelei Liu, Han Yu, and Tong Ruan. 2025. Imqc: A large language model platform for medical quality control. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 28810–28818. find
  42. Kaiyan Zhang, Sihang Zeng, Ermo Hua, Ning Ding, Zhang-Ren Chen, Zhiyuan Ma, Haoxin Li, Ganqu Cui, Biqing Qi, Xuekai Zhu, and 1 others. 2024a. Ultramedical: Building specialized generalists in biomedicine. Advances in Neural Information Processing Systems, 37:26045–26081. find
  43. Weiyan Zhang, Jiacheng Wang, Chuang Chen, Wanpeng Lu, Wen Du, Haofen Wang, Jingping Liu, and Tong Ruan. 2024b. A bidirectional extraction-then-evaluation framework for complex relation extraction. IEEE Transactions on Knowledge and Data Engineering, 36(12):7442–7454. find
  44. Guorui Zheng, Xidong Wang, Juhao Liang, Nuo Chen, Yuping Zheng, and Benyou Wang. 2025. Efficiently democratizing medical LLMs for 50 languages via a mixture of language family experts. In The Thirteenth International Conference on Learning Representations. find
  45. Xiaoquan Zhi, Hongke Zhao, Likang Wu, Chuang Zhao, and Hengshu Zhu. 2025. Reinventing clinical dialogue: Agentic paradigms for llm enabled healthcare communication. Preprint, arXiv:2512.01453. — [arXiv:2512.01453](https://arxiv.org/abs/2512.01453) find
  46. Jiayuan Zhu, Jiazhen Pan, Yuyuan Liu, Fenglin Liu, and Junde Wu. 2025. Ask patients with patience: Enabling llms for human-centric medical dialogue with grounded reasoning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 2846–2857.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yu-2026-experience-is-teacher-reusing-atomic/