Elamur Research⌕
☰
home / papers / 2026 / Faithful Medical Dialogue Generation Using Homo-Heterogeneous Exemplar-based In-Context Knowledge Grounding

Faithful Medical Dialogue Generation Using Homo-Heterogeneous Exemplar-based In-Context Knowledge Grounding

Summary

LLMs used for medical dialogue generation (MDG) often hallucinate. They can contradict even correct knowledge they are given, which is dangerous in clinical advice. The authors present IMKDG (In-context Medical Knowledge-grounded Dialogue Generator), a plug-and-play retrieval framework that needs no extra knowledge-encoding modules. It combines external medical knowledge with in-context learning (ICL) and has three parts:

  • Medical Knowledge Enrichment (MKE): finds medical concepts in the patient–doctor history with QuickUMLS and builds a small knowledge graph of triplets using relations from the UMLS Semantic Network.
  • Homo-Heterogeneous Exemplar Selection (H2ES): picks demonstration examples that are both relevant and varied, judged by dialogue context and attached knowledge.
  • Prompt Construction (PC): assembles the final prompt.

The framework was tested with seven LLMs on the MedDialog-EN and CDialog datasets. Automatic and human evaluations indicate fewer hallucinations and more faithful responses. The method is model-agnostic and does not require fine-tuning, so it can be applied to other LLMs and medical corpora.

Key points

  • Task setup: MDG is cast as ICL. The model generates the next doctor turn from the alternating patient/doctor history plus retrieved knowledge K.
  • Knowledge graph construction: each UMLS CUI becomes a node, and edges come from the UMLS Semantic Network. An example triplet is (coronavirus, affects, respiratory system).
  • Models evaluated: BioGPT, ChatGPT, MedAlpaca, ChatDoctor, PMC-LLaMA, MedPaLM 2 and GPT-4.
  • Datasets: MedDialog-EN and CDialog (a multi-turn COVID-19 conversation corpus).
  • Code and resources: released at github.com/priyanshu-profile/LREC26_IMKDG. The excerpt provided did not include the numeric result tables, so specific scores are not reported here.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/43dzmqsdheef

@inproceedings{priya2026faithful,
  title = {{Faithful Medical Dialogue Generation Using Homo-Heterogeneous Exemplar-based In-Context Knowledge Grounding}},
  author = {Priya, Priyanshu and Goyal, Hardik and Ekbal, Asif},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/43dzmqsdheef},
  url = {https://doi.org/10.63317/43dzmqsdheef}
}

References

73 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. find our card
  2. Monica Agrawal, Stefan Hegselmann, Hunter Lang, Yoon Kim, and David Sontag. 2022. Large language models are few-shot clinical information extractors. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1998–2022. find our card
  3. Explosion AI. 2025. Pipeline · spacy universe. https://spacy.io/universe/category/pipeline. Accessed: 2025-10-25. ↗
  4. Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72. find our card
  5. Olivier Bodenreider. 2004. The unified medical language system (umls): integrating biomedical terminology. Nucleic acids research, 32(suppl_1):D267–D270. find
  6. Peter F Brown, Stephen A Della Pietra, Vincent J Della Pietra, Jennifer C Lai, and Robert L Mercer. 1992. An estimate of an upper bound for the entropy of english. Computational Linguistics, 18(1):31–40. find our card
  7. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901. find
  8. BSI. 1973a. Natural Fibre Twines, 3rd edition. British Standards Institution, London. BS 2570. find
  9. BSI. 1973b. Natural fibre twines. BS 2570, British Standards Institution, London. 3rd. edn. find
  10. A. Castor and L. E. Pollux. 1992. The use of user modelling to guide inference and learning. Applied Intelligence, 2(1):37–53. find
  11. J.L. Chercheur. 1994. Case-Based Reasoning, 2nd edition. Morgan Kaufman Publishers, San Mateo, CA. find
  12. N. Chomsky. 1973. Conditions on transformations. In A festschrift for Morris Halle, New York. Holt, Rinehart & Winston. find
  13. Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and play language models: A simple approach to controlled text generation. find our card
  14. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. find
  15. Chengfeng Dou, Zhi Jin, Wenpin Jiao, Haiyan Zhao, Yongqiang Zhao, and Zhengwei Tao. 2023. Plugmed: Improving specificity in patient-centered medical dialogue generation using in-context learning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5050–5066. find
  16. Esin Durmus, He He, and Mona Diab. 2020. Feqa: A question answering evaluation framework for faithfulness assessment in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5055–5070. find our card
  17. Umberto Eco. 1990. The Limits of Interpretation. Indian University Press. find
  18. Global Market Estimates. 2020. Global online doctor consultation market. https://www.globalmarketestimates.com/market-report/ global-online-doctor-consultation-market-2172. Accessed: 2025-10-25. ↗
  19. Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378. find
  20. Yanjun Gao, Ruizhe Li, John Caskey, Dmitriy Dligach, Timothy Miller, Matthew M Churpek, and Majid Afshar. 2023. Leveraging a medical knowledge graph into large language models for diagnosis prediction. arXiv preprint arXiv:2308.14321. — [arXiv:2308.14321](https://arxiv.org/abs/2308.14321) find
  21. Tianyu Han, Lisa C Adams, Jens-Michalis Papaioannou, Paul Grundmann, Tom Oberhauser, Alexander Löser, Daniel Truhn, and Keno K Bressem. 2023. Medalpaca–an open-source collection of medical conversational ai models and training data. arXiv:2304.08247. arXiv preprint Paul Gerhard Hoel. 1971a. Elementa… find our card
  22. Paul Gerhard Hoel. 1971b. Elementary Statistics, 3rd edition. Wiley series in probability and mathematical statistics. Wiley, New York, Chichester. ISBN 0 471 40300. find
  23. Abid Hossain, Priyanshu Priya, Armita Mani Tripathi, Pradeepika Verma, and Asif Ekbal. 2025. Mender: Multi-hop commonsense and domain-specific cot reasoning for knowledge-grounded empathetic counseling of crime victims. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of… find
  24. Otto Jespersen. 1922. Language: Its Nature, Development, and Origin. Allen and Unwin. find
  25. Ziwei Ji, Zihan Liu, Nayeon Lee, Tiezheng Yu, Bryan Wilie, Min Zeng, and Pascale Fung. 2023. Rho: Reducing hallucination in open-domain dialogues with knowledge grounding. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4504–4522. find
  26. Haoqiang Kang, Juntong Ni, and Huaxiu Yao. 2023. Ever: Mitigating hallucination in large language models through real-time verification and rectification. arXiv preprint arXiv:2311.09114. — [arXiv:2311.09114](https://arxiv.org/abs/2311.09114) find
  27. Minki Kang, Jin Myung Kwak, Jinheon Baek, and Sung Ju Hwang. 2022. Knowledge-consistent dialogue generation with knowledge graphs. In ICML 2022 Workshop on Knowledge Retrieval and Language Models. find
  28. Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2019. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234–1240. find our card
  29. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. find our card
  30. Dongdong Li, Zhaochun Ren, Pengjie Ren, Zhumin Chen, Miao Fan, Jun Ma, and Maarten de Rijke. 2021. Semi-supervised variational reasoning for medical dialogue generation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 544–554. find
  31. Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023. Helma: A large-scale hallucination evaluation benchmark for large language models. arXiv preprint arXiv:2305.11747. find our card
  32. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81. find
  33. Chia-Wei Liu, Ryan Lowe, Iulian V Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. arXiv preprint arXiv:1603.08023. find
  34. Tianyu Liu, Yizhe Zhang, Chris Brockett, Yi Mao, Zhifang Sui, Weizhu Chen, and William B Dolan. 2022a. A token-level reference-free hallucination detection benchmark for free-form text generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: L… find our card
  35. Wenge Liu, Jianheng Tang, Yi Cheng, Wenjie Li, Yefeng Zheng, and Xiaodan Liang. 2022b. Meddg: an entity-centric medical consultation dataset for entity-aware medical dialogue generation. In CCF International Conference on Natural Language Processing and Chinese Computing, pages 447–459. Springer. find
  36. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. find
  37. Yongkang Liu, Shi Feng, Daling Wang, Yifei Zhang, and Hinrich Schütze. 2023. Pvgru: Generating diverse and relevant dialogue responses via pseudo-variational mechanism. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3295–3310… find
  38. Zihan Liu, Mostofa Patwary, Ryan Prenger, Shrimai Prabhumoye, Wei Ping, Mohammad Shoeybi, and Bryan Catanzaro. 2022c. Multi-stage prompting for knowledgeable dialogue generation. arXiv preprint arXiv:2203.08745. find
  39. Zihan Liu, Mostofa Patwary, Ryan Prenger, Shrimai Prabhumoye, Wei Ping, Mohammad Shoeybi, and Bryan Catanzaro. 2022d. Multi-stage prompting for knowledgeable dialogue generation. find
  40. Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie-Yan Liu. 2022a. Biogpt: generative pre-trained transformer for biomedical text generation and mining. Briefings in bioinformatics, 23(6):bbac409. find
  41. Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie-Yan Liu. 2022b. Biogpt: generative pre-trained transformer for biomedical text generation and mining. Briefings in Bioinformatics, 23(6). find
  42. Kshitij Mishra, Priyanshu Priya, Manisha Burja, and Asif Ekbal. 2023a. e-therapist: I suggest you to cultivate a mindset of positivity and nurture uplifting thoughts. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13952–13967. find
  43. Kshitij Mishra, Priyanshu Priya, and Asif Ekbal. 2023b. Help me heal: A reinforced polite and empathetic mental health and legal counseling dialogue system for crime victims. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 14408–14416. find
  44. Kshitij Mishra, Priyanshu Priya, and Asif Ekbal. 2023c. Pal to lend a helping hand: Towards building an emotion adaptive polite and empathetic counseling conversational agent. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12… find
  45. Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, and Xindong Wu. 2024. Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering. find our card
  46. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318. find our card
  47. Priyanshu Priya, Kshitij Mishra, Palak Totala, and Asif Ekbal. 2023. Partner: A persuasive mental health and legal counselling dialogue system for women and children crime victims. In IJCAI, pages 6183–6191. find
  48. Priyanshu Priya, Armita Mani Tripathi, Deeksha Varshney, Mauajama Firdaus, and Asif Ekbal. 2025. Empower-kare: Deep prompt learning for knowledge-aware response generation in clinical counseling and legal support conversations. IEEE Transactions on Artificial Intelligence. find
  49. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. find
  50. Vipula Rawte, Amit Sheth, and Amitava Das. 2023. A survey of hallucination in large foundation models. arXiv preprint arXiv:2309.05922. find our card
  51. Stephen E Robertson and Steve Walker. 1994. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In SIGIR’94: Proceedings of the Seventeenth Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval, organised by Dubli… find
  52. João Sá, Vanessa Queiroz Marinho, Ana Rita Magalhães, Tiago Lacerda, and Diogo Goncalves. 2022. Diversity vs relevance: A practical multi-objective study in luxury fashion recommendations. In Proceedings of the 45th International ACM SIGIR Conference on research and development in information retrie… find
  53. Thomas Scialom, Paul-Alexis Dray, Patrick Gallinari, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano, and Alex Wang. 2021. Questeval: Summarization asks for fact-based evaluation. arXiv preprint arXiv:2103.12693. find our card
  54. Xiaoming Shi, Zeming Liu, Li Du, Yuxuan Wang, Hongru Wang, Yuhang Guo, Tong Ruan, Jie Xu, and Shaoting Zhang. 2024. Medical dialogue: A survey of categories, methods, evaluation and challenges. arXiv preprint arXiv:2405.10630. — [arXiv:2405.10630](https://arxiv.org/abs/2405.10630) find
  55. Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. Retrieval augmentation reduces hallucination in conversation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3784–3803. find our card
  56. Charles Joseph Singer, E. J. Holmyard, and A. R. Hall, editors. 1954–58. A history of technology. Oxford University Press, London. 5 vol. find
  57. Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. 2022. Large language models encode clinical knowledge. arXiv preprint arXiv:2212.13138. find our card
  58. Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Le Hou, Kevin Clark, Stephen Pfohl, Heather Cole-Lewis, Darlene Neal, et al. 2023. Towards expert-level medical question answering with large language models. arXiv preprint arXiv:2305.09617. find our card
  59. Jannik Strötgen and Michael Gertz. 2012. Temporal tagging on different domains: Challenges, strategies, and gold standards. In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12), pages 3746–3753, Istanbul, Turkey. European Language Resource Association (… find our card
  60. S. Superman, B. Batman, C. Catwoman, and S. Spiderman. 2000. Superheroes experiences with books, 20th edition. The Phantom Editors Associates, Gotham City. find
  61. Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. 2023. Large language models in medicine. Nature medicine, 29(8):1930–1940. find
  62. Deeksha Varshney, Aizan Zafar, Niranshu Behera, and Asif Ekbal. 2022a. CDialog: A multi-turn covid-19 conversation dataset for entity-aware dialog generation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11373–11385, Abu Dhabi, United Arab Emirates… find
  63. Deeksha Varshney, Aizan Zafar, Niranshu Kumar Behra, and Asif Ekbal. 2022b. Cdialog: A multi-turn covid-19 conversation dataset for entity-aware dialog generation. arXiv preprint arXiv:2212.06049. find
  64. Bernard L Welch. 1947. The generalization of ‘student’s’problem when several different population varlances are involved. Biometrika, 34(1-2):28–35. find
  65. Chaoyi Wu, Weixiong Lin, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. 2023. Pmc-llama: Towards building open-source language models for medicine. arXiv preprint arXiv:2305.10415, 6. find our card
  66. Jiageng Wu, Xian Wu, Zhaopeng Qiu, Minghui Li, Shixu Lin, Yingying Zhang, Yefeng Zheng, Changzheng Yuan, and Jie Yang. 2024. Large language models leverage external knowledge to extend clinical insight beyond language boundaries. Journal of the American Medical Informatics Association, page ocae079. find
  67. Zeqiu Wu, Michel Galley, Chris Brockett, Yizhe Zhang, Xiang Gao, Chris Quirk, Rik Koncel-Kedziorski, Jianfeng Gao, Hannaneh Hajishirzi, Mari Ostendorf, et al. 2021. A controllable model of grounded response generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, page… find
  68. Yan Xu, Etsuko Ishii, Samuel Cahyawijaya, Zihan Liu, Genta Indra Winata, Andrea Madotto, Dan Su, and Pascale Fung. 2022. Retrieval-free knowledge-grounded dialogue response generation with adapters. In Proceedings of the Second DialDoc Workshop on Document-grounded Dialogue and Conversational Questi… find
  69. Li Yunxiang, Li Zihan, Zhang Kai, Dan Ruilong, and Zhang You. 2023. Chatdoctor: A medical chat model fine-tuned on llama model using medical domain knowledge. arXiv preprint arXiv:2303.14070. find our card
  70. Guangtao Zeng, Wenmian Yang, Zeqian Ju, Yue Yang, Sicheng Wang, Ruisi Zhang, Meng Zhou, Jiaqi Zeng, Xiangyu Dong, Ruoyu Zhang, Hongchao Fang, Penghui Zhu, Shu Chen, and Pengtao Xie. 2020. MedDialog: Large-scale medical dialogue datasets. In Proceedings of the 2020 Conference on Empirical Methods in… find
  71. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. find
  72. Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. DIALOGPT : Large-scale generative pre-training for conversational response generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Ling… find our card
  73. Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. 2023. Analyzing and mitigating object hallucination in large vision-language models. arXiv preprint arXiv:2310.00754. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/priya-2026-faithful-medical-dialogue-generation-homo/