Elamur Research⌕
☰
home / papers / 2026 / Linking Knowledge to Care: Knowledge Graph-Augmented Medical Follow-Up Question Generation

Linking Knowledge to Care: Knowledge Graph-Augmented Medical Follow-Up Question Generation

Paper · 2026 · Liwen Sun, Xiang Yu, Ming Tan et al.

Summary

Before a diagnosis, clinicians spend much of their time collecting patient history, and gaps in that history are a known source of diagnostic error. LLMs could help by asking follow-up questions, but they often overlook symptoms because their medical knowledge is limited. The authors propose KG-Followup, which connects LLM question generation to a biomedical knowledge graph. The pipeline first lets the LLM ask its own clarification questions. It then extracts clinical entities as a stand-in for an EHR and adds questions from three KG-based steps: retrieval of shared neighbouring concepts, reasoning paths from symptoms to differential diagnoses (DDX), and in-context examples drawn from cases the graph cannot map ("hard" cases). Finally, overlapping questions are clustered and merged. The authors also build ClinicalInquiryBench, a benchmark with importance weights derived from HealthBench conversations. On it and on FollowupBench, KG-Followup reaches about 70–80% weighted recall and beats the multi-agent FollowupQ baseline by roughly 5–8%. It matches the coverage of much longer zero-shot question lists with fewer questions, which matters for keeping patient interactions short.

Key points

  • ClinicalInquiryBench: 5,000 HealthBench cases were filtered to 1,498 context-seeking instances with weighted reference questions (about 5 per case), split into 250 dev and 1,248 test. FollowupBench has 250 expert-curated cases with about 9 unweighted questions each.
  • Evaluation: weighted recall judged by an LLM. Backbones are Claude Haiku and Claude Sonnet, with GPT-4o and MedGemma-27b tested only as Zero-Shot-k baselines.
  • Best results (Claude Sonnet with KG-informed active ICL): 0.75/0.77 recall on ClinicalInquiryBench dev/test with 20 questions, versus 0.65/0.66 for FollowupQ. On FollowupBench it reaches 0.82 with 40 questions, versus 0.74.
  • Ablations:
    • The DDX-guided module contributes more than the EHR-guided one, and the choice among KG retrieval variants matters little.
    • Clustering-based consolidation cuts the question pool from 51 to 40 with only a small loss (0.79 → 0.77). Having an LLM select the questions drops recall to 0.71.
    • Unsupervised selection of KG-based hard examples performs on par with supervised selection of hard cases.
  • Limitations: the method depends on a comprehensive, current knowledge graph, and the evaluation is offline only.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations🏢AmazonSeattle · United States112 works · 40,565 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksMedicine · 241 worksHealth Informatics · 184 works

🧭 Topics 2Research subjects and languages the work is about.Question answering · 441 worksKnowledge graphs & ontologies · 192 works

Cite

Cite as Liwen Sun, Xiang Yu, Ming Tan, Zhuohao Chen, Anqi Cheng, Ashutosh Joshi and Chenyan Xiong. 2026. Linking Knowledge to Care: Knowledge Graph-Augmented Medical Follow-Up Question Generation. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-eacl.43

@inproceedings{sun2026linking,
  title = {{Linking Knowledge to Care: Knowledge Graph-Augmented Medical Follow-Up Question Generation}},
  author = {Liwen Sun, Xiang Yu, Ming Tan, Zhuohao Chen, Anqi Cheng, Ashutosh Joshi and Chenyan Xiong},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.43},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.43}
}

References

26 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bohingamu Mudiyanselage S. Catchpool M. et al. Abimanyi-Ochom, J. 2019. Strategies to reduce diagnostic errors: a systematic review. find
  2. Rahul K. Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin Quiñonero-Candela, Foivos Tsimpourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, Johannes Heidecke, and Karan Singhal. 2025. Healthbench: Evaluating large language models towards improved human health. Preprint…
  3. Abhinand Balachandran. 2024. Medembed: Medical-focused embedding models. find
  4. Payal Chandak, Kexin Huang, and Marinka Zitnik. 2023. Building a knowledge graph to enable precision medicine. Nature Scientific Data.
  5. John W Ely, Jerome A Osheroff, Melisa L Chambliss, Mark H Ebell, and Mark E Rosenbaum. 2011. The value of clinical questions: identifying research priorities in primary care. BMJ Quality & Safety, 20(9):787–792. find
  6. Bryan T. Miller Erin P. Balogh and John R. Ball. 2015. Improving Diagnosis in Health Care, chapter 3, Overview of Diagnostic Error in Health Care. The National Academies Press. find
  7. OpenAI et al. 2024. Gpt-4o system card. Preprint, arXiv:2410.21276.
  8. Joseph Gatto, Parker Seegmiller, Timothy Burdick, Inas S. Khayal, Sarah DeLozier, and Sarah M. Preum. 2025. Follow-up question generation for enhanced patient-provider conversations. Preprint, arXiv:2503.17509. find
  9. Mark L Graber, Robert M Wachter, and Christine K Cassel. 2013. Diagnostic error in internal medicine. BMJ Quality & Safety, 22(Suppl 2):ii21–ii27. find
  10. Pengcheng Jiang, Cao Xiao, Minhao Jiang, Parminder Bhatia, Taha Kass-Hout, Jimeng Sun, and Jiawei Han. 2025. Reasoning-enhanced healthcare predictions with knowledge graph community retrieval. Preprint, arXiv:2410.04585. — [arXiv:2410.04585](https://arxiv.org/abs/2410.04585) find
  11. Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2020. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Preprint, arXiv:2009.13081.
  12. Schaekermann M. Palepu A. et al. Tu, T. 2025. Towards conversational diagnostic artificial intelligence. Nature, pages 642, 442–450. find
  13. Juncheng Wu, Wenlong Deng, Xingxuan Li, Sheng Liu, Taomian Mi, Yifan Peng, Ziyang Xu, Yi Liu, Hyunjin Cho, Chang-In Choi, Yihan Cao, Hui Ren, Xiang Li, Xiaoxiao Li, and Yuyin Zhou. 2025. Medreason: Eliciting factual medical reasoning steps in llms via knowledge graphs. Preprint, arXiv:2504.00993. — [arXiv:2504.00993](https://arxiv.org/abs/2504.00993) find
  14. Jerome P. Kassirer and Richard I. Kopelman. 2010. Learning Clinical Reasoning, 2 edition. Lippincott Williams & Wilkins. find
  15. Junde Wu, Jiayuan Zhu, Yunli Qi, Jingkun Chen, Min Xu, Filippo Menolascina, and Vicente Grau. 2024. Medical graph rag: Towards safe medical large language model via graph retrieval-augmented generation. Preprint, arXiv:2408.04187. — [arXiv:2408.04187](https://arxiv.org/abs/2408.04187) find
  16. Shuyue Stella Li, Vidhisha Balachandran, Shangbin Feng, Jonathan S. Ilgen, Emma Pierson, Pang Wei Koh, and Yulia Tsvetkov. 2024a. Mediq: Question-asking llms and a benchmark for reliable interactive clinical reasoning. Preprint, arXiv:2406.00922.
  17. Guangzhi Xiong, Qiao Jin, Zhiyong Lu, and Aidong Zhang. 2024. Benchmarking retrieval-augmented generation for medicine. Preprint, arXiv:2402.13178.
  18. Shuyue Stella Li, Vidhisha Balachandran, Shangbin Feng, Jonathan S. Ilgen, Emma Pierson, Pang Wei Koh, and Yulia Tsvetkov. 2024b. Mediq: Question-asking llms and a benchmark for reliable interactive clinical reasoning. In Advances in Neural Information Processing Systems, volume 37, pages 28858– 288…
  19. Shuyue Stella Li, Jimin Mun, Faeze Brahman, Pedram Hosseini, Bryceton G. Thomas, Jessica M. Sin, Bing Ren, Jonathan S. Ilgen, Yulia Tsvetkov, and Maarten Sap. 2025. Alfa: Aligning llms to ask good questions a case study in clinical reasoning. Preprint, arXiv:2502.14860. — [arXiv:2502.14860](https://arxiv.org/abs/2502.14860) find
  20. Gordon D Schiff, Osman Hasan, Sujan Kim, Ramona Abrams, Karen Cosby, Bruce Lambert, Arthur S Elstein, Sarah Hasler, Ngoy Kabongo, Nada Krosnjar, and 1 others. 2009. Diagnostic error in medicine: analysis of 583 physician-reported errors. Archives of Internal Medicine, 169(20):1881–1887. find
  21. Henk G. Schmidt and Remy M. Rikers. 2007. How expertise develops in medicine: knowledge encapsulation and illness script formation. Medical Education, 41(12):1133–1139. find
  22. Andrew et al. Sellergren. 2025. Medgemma technical report. Preprint, arXiv:2507.05201. find
  23. Hardeep Singh, Andrew N D Meyer, and Eric J Thomas. 2013. The frequency of diagnostic errors in outpatient care: estimations from three large observational studies. JAMA Internal Medicine, 173(6):418–425. find
  24. Hardeep Singh and Dean F Sittig. 2020. Advancing the science of measurement of diagnostic errors in healthcare: the safer dx framework. BMJ Quality & Safety, 29(10):874–880. find
  25. Robert L Trowbridge, Jason J Rencic, and Steven J Durning. 2013. Teaching clinical reasoning: a practical framework for classroom and bedside instruction. Academic Medicine, 88(2):182–188. find
  26. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Preprint, arXiv:2306.05685.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sun-2026-linking-knowledge-care-knowledge-graph/