Elamur Research⌕
☰
home / papers / 2026 / Dialectic-Med: Mitigating Diagnostic Hallucinations via Counterfactual Adversarial Multi-Agent Debate

Dialectic-Med: Mitigating Diagnostic Hallucinations via Counterfactual Adversarial Multi-Agent Debate

Paper · 2026 · Zhixiang Lu and Jionglong Su

Summary

Medical multimodal LLMs often commit to an early diagnostic guess and then invent image findings to justify it. This confirmation-bias failure is something linear Chain-of-Thought prompting cannot fix, because it only looks for supporting evidence. Lu and Su address this with Dialectic-Med, a debate framework inspired by Popper's notion of falsification. Three agents interact:

  • a Proponent proposes a diagnosis and rationale;
  • an Opponent tries to refute it with image evidence;
  • a Mediator weighs the exchange and guides revisions.

The Opponent's Visual Falsification Module (VFM) works in two steps. It first writes a counterfactual probe describing what the image should show if the hypothesis were wrong. It then localizes matching regions through PubMedCLIP patch–text similarity. Weak attacks end the debate. Strong attacks add weighted edges to a dynamic consensus graph, from which the final answer is derived. On MIMIC-CXR-VQA, VQA-RAD and PathVQA, the authors report state-of-the-art accuracy, fewer hallucinations and more faithful explanations than single-agent baselines. The approach also yields an auditable reasoning trail suited to clinical decision support.

Key points

  • Debate loop. Attack strength is the mean VFM attention over the top-k retrieved regions. If it falls below a threshold θ, consensus is declared; otherwise a falsification edge with that weight is added to the graph. Revised hypotheses get rectification edges weighted by the Proponent's self-reported confidence in [0, 1]. The loop runs for at most T_max turns.

  • Benchmarks and headline result. Evaluation covers MIMIC-CXR-VQA, VQA-RAD and PathVQA. The authors claim a 12.5% gain in explanation faithfulness over single-agent baselines.

  • Hallucination metrics. Across the compared systems, CHAIR_S falls from 18.4% to 10.7%, CHAIR_I from 12.1% to 6.5%, and expert-rated faithfulness rises from 3.42 to 4.33. The best row is presumably Dialectic-Med, but the row labels were lost in PDF extraction.

  • Case study. An initial "Pneumonia" answer with invented consolidations is corrected to "Atelectasis." The correction relies on high-attention evidence (α > 0.8) at an elevated left hemidiaphragm.

  • Stated limitations. The authors note four constraints:

    • higher latency than single-pass inference;
    • dependence on the VLM backbone, which struggles with nodules under 3 mm and out-of-distribution rare conditions;
    • thresholds that may need recalibration per sub-specialty;
    • possible Mediator adjudication errors.

    They position the system as decision support only, tested on de-identified public data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksMedicine · 241 worksHealth Informatics · 184 works

Cite

Cite as Zhixiang Lu and Jionglong Su. 2026. Dialectic-Med: Mitigating Diagnostic Hallucinations via Counterfactual Adversarial Multi-Agent Debate. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1837

@inproceedings{lu2026dialectic,
  title = {{Dialectic-Med: Mitigating Diagnostic Hallucinations via Counterfactual Adversarial Multi-Agent Debate}},
  author = {Zhixiang Lu and Jionglong Su},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1837},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1837}
}

References

40 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Seongsu Bae, Daeun Kyung, Jaehee Ryu, Eunbyeol Cho, Gyubok Lee, Sunjun Kweon, Jungwoo Oh, Lei Ji, Eric Chang, Tackeun Kim, and Edward Choi. 2023. Ehrxqa: A multi-modal question answering dataset for electronic health records with chest x-ray images. In Advances in Neural Information Processing Syste… find
  2. Yakoub Bazi, Mohamad Mahmoud Al Rahhal, Laila Bashmal, and Mansour Zuair. 2023. Vision–language model for visual question answering in medical imagery. Bioengineering, 10(3). find
  3. Benedikt Boecking, Naoto Usuyama, Shruthi Bannur, Daniel C. Castro, Anton Schwaighofer, Stephanie Hyland, Maria Wetscherek, Tristan Naumann, Aditya Nori, Javier Alvarez-Valle, Hoifung Poon, and Ozan Oktay. 2022. Making the most of text semantics to improve biomedical vision–language processing. In C… find
  4. Justin Chen, Swarnadeep Saha, and Mohit Bansal. 2024. ReConcile: Round-table conference improves reasoning via consensus among diverse LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7066–7085, Bangkok, Thailand. Associa…
  5. Alexander Ding, Jonathan Joshi, and Emily Tiwana. 2023. Patient safety in radiology and medical imaging. In Patient Safety: A Case-based Innovative Playbook for Safer Care, pages 261–277. Springer. find
  6. Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. 2024. Improving factuality and reasoning in language models through multiagent debate. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org.
  7. Sedigheh Eslami, Christoph Meinel, and Gerard de Melo. 2023. PubMedCLIP: How much does CLIP benefit visual question answering in the medical domain? In Findings of the Association for Computational Linguistics: EACL 2023, pages 1181–1193, Dubrovnik, Croatia. Association for Computational Linguistics… find
  8. Xuehai He, Yichen Zhang, Luntian Mou, Eric Xing, and Pengtao Xie. 2020. Pathvqa: 30000+ questions for medical visual question answering. Preprint, arXiv:2003.10286. — [arXiv:2003.10286](https://arxiv.org/abs/2003.10286) find
  9. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM Comput. Surv., 55(12).
  10. Alistair EW Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Roger G Mark, and Steven Horng. 2019. Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports. Scientific data, 6(1):317. find
  11. Yubin Kim, Hyewon Jeong, Shan Chen, Shuyue Stella Li, Chanwoo Park, Mingyu Lu, Kumail Alhamoud, Jimin Mun, Cristina Grau, Minseok Jung, Rodrigo Gameiro, Chunjong Park, Hyeonhoon Lee, Hae Won Park, Daniel McDuff, Samir Tulebaev, and Cynthia Breazeal. 2025. Medical hallucinations in foundation models… — [arXiv:2503.05777](https://arxiv.org/abs/2503.05777) find
  12. Jason J Lau, Soumya Gayen, Asma Ben Abacha, and Dina Demner-Fushman. 2018. A dataset of clinically generated visual questions and answers about radiology images. Scientific data, 5(1):180251. find
  13. Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. 2023a. Llava-med: training a large language-and-vision assistant for biomedicine in one day. In Proceedings of the 37th International Conference on Neural Information Proc…
  14. Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023b. Camel: communicative agents for "mind" exploration of large language model society. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY,…
  15. Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. 2024. Encouraging divergent thinking in large language models through multi-agent debate. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17…
  16. Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. 2024a. A survey on hallucination in large vision-language models. Preprint, arXiv:2402.00253. find
  17. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024b. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173.
  18. Valentin Liévin, Christoffer Egeberg Hother, Andreas Geert Motzfeldt, and Ole Winther. 2024. Can large language models reason about medical questions? Patterns, 5(3):100943. find
  19. Zhixiang Lu, Yulong Li, Feilong Tang, Zhengyong Jiang, Chong Li, Mian Zhou, Tenglong Li, and Jionglong Su. 2026a. Deepgb-tb: A risk-balanced cross-attention gradient-boosted convolutional network for rapid, interpretable tuberculosis screening. Proceedings of the AAAI Conference on Artificial Intell… find
  20. Zhixiang Lu, Shijie Xu, Kaicheng Yan, Xuyue Cai, Chong Zhang, Yulong Li, Angelos Stefanidis, Anh Nguyen, and Jionglong Su. 2026b. Skinclip-vl: Consistency-aware vision-language learning for multimodal skin cancer diagnosis. Preprint, arXiv:2603.21010. — [arXiv:2603.21010](https://arxiv.org/abs/2603.21010) find
  21. Jie Ma, Zhitao Gao, Qi Chai, Wangchun Sun, Pinghui Wang, Hongbin Pei, Jing Tao, Lingyun Song, Jun Liu, Chen Zhang, and Lizhen Cui. 2025. Debate on graph: A flexible and reliable reasoning framework for large language models. Proceedings of the AAAI Conference on Artificial Intelligence, 39(23):24768… find
  22. Zahra Mahdavi, Zahra Khodakaramimaghsoud, Hooman Khaloo, Sina Bakhshandeh Taleshani, Erfan Hashemi, Javad Mirzapour Kaleybar, and Omid Nejati Manzari. 2026. Med-vcd: Mitigating hallucination for medical large vision language models through visual contrastive decoding. Computers in Biology and Medici… find
  23. Jing Miao, Charat Thongprayoon, Supawadee Sup-padungsuk, Pajaree Krisanapan, Yeshwanter Radhakrishnan, and Wisit Cheungpasitporn. 2024. Chain of thought utilization in large language models and application in nephrology. Medicina, 60(1). find
  24. Michael Moor, Oishi Banerjee, Zahra Shakeri Hossein Abad, Harlan M Krumholz, Jure Leskovec, Eric J Topol, and Pranav Rajpurkar. 2023. Foundation models for generalist medical artificial intelligence. Nature, 616(7956):259–265. find
  25. Yoojin Nam, Dong Yeong Kim, Sunggu Kyung, Jinyoung Seo, Jeong Min Song, Jimin Kwon, Jihyun Kim, Wooyoung Jo, Hyungbin Park, Jimin Sung, Sangah Park, Heeyeon Kwon, Taehee Kwon, Kanghyun Kim, and Namkug Kim. 2025. Multimodal large language models in medical imaging: Current state and future directions… find
  26. Raymond S Nickerson. 1998. Confirmation bias: A ubiquitous phenomenon in many guises. Review of general psychology, 2(2):175–220. find
  27. Yulei Niu, Kaihua Tang, Hanwang Zhang, Zhiwu Lu, Xian-Sheng Hua, and Ji-Rong Wen. 2021. Counterfactual vqa: A cause-effect look at language bias. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12695–12705. find
  28. Shrey Pandit, Jiawei Xu, Junyuan Hong, Zhangyang Wang, Tianlong Chen, Kaidi Xu, and Ying Ding. 2025. MedHallu: A comprehensive benchmark for detecting medical hallucinations in large language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 285… find
  29. Karl Popper. 2005. The logic of scientific discovery. Routledge. find
  30. Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. 2018. Object hallucination in image captioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4035–4045, Brussels, Belgium. Association for Computational Linguistics.
  31. Johannes Rückert, Louise Bloch, Raphael Brüngel, Ahmad Idrissi-Yaghir, Henning Schäfer, Cynthia S. Schmidt, Sven Koitka, Obioma Pelka, Asma Ben Abacha, Alba G. Seco de Herrera, Henning Müller, Peter A. Horn, Felix Nensa, and Christoph M. Friedrich. 2024. Rocov2: Radiology objects in context version… find
  32. Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul Gamble, Chris Kelly, Alvin Babiker, Joelle Barral, Christopher Semturs, Alan Karthikesalingam, and Vivek Nataraja… — Large Language Models Encode Clinical Knowledge
  33. Tao Tang, Shijie Xu, Jionglong Su, and Zhixiang Lu. 2026. Causal-sam-llm: Large language models as causal reasoners for robust medical segmentation. Preprint, arXiv:2507.03585. — [arXiv:2507.03585](https://arxiv.org/abs/2507.03585) find
  34. Xiangru Tang, Anni Zou, Zhuosheng Zhang, Ziming Li, Yilun Zhao, Xingyao Zhang, Arman Cohan, and Mark Gerstein. 2024. MedAgents: Large language models as collaborators for zero-shot medical reasoning. In Findings of the Association for Computational Linguistics: ACL 2024, pages 599–621, Bangkok, Thai…
  35. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
  36. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations.
  37. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, R…
  38. Zhu Zhang, Zhou Zhao, Zhijie Lin, jieming zhu, and Xiuqiang He. 2020. Counterfactual contrastive learning for weakly-supervised vision-language grounding. In Advances in Neural Information Processing Systems, volume 33, pages 18123–18134. Curran Associates, Inc. find
  39. Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chunyuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, and Jianfeng Gao. 2022. Regionclip: Region-based language-image pretraining. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16772–1… find
  40. Zhihong Zhu, Yunyan Zhang, Xianwei Zhuang, Fan Zhang, Zhongwei Wan, Yuyan Chen, Qingqing Long, Yefeng Zheng, and Xian Wu. 2025. Can we trust AI doctors? a survey of medical hallucination in large language and large vision-language models. In Findings of the Association for Computational Linguistics:… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lu-2026-dialectic-med-mitigating-diagnostic-hallucinations/