Elamur Research⌕
☰
home / papers / 2026 / When Correct Beliefs Collapse: Epistemic Resilience of LLMs under Clinical Pressure

When Correct Beliefs Collapse: Epistemic Resilience of LLMs under Clinical Pressure

Paper · 2026 · Boyu Xiao, Xiuqi Tian, Xuwen Song et al.

Summary

Medical LLMs score highly on static exam-style benchmarks, but this work asks whether they hold onto a correct diagnosis when a user keeps pushing back over several turns. The authors build MED-STRESS, a multi-turn stress test that uses an Anchor-Attack protocol. Only cases the model answers correctly without pressure are kept. These cases then go through three rounds of escalating, non-evidential pushback drawn from four pressure types. Testing nine frontier LLMs reveals a Knowledge-Robustness Gap: strong initial diagnostic ability does not predict stable beliefs under pressure.

Two remedies are proposed:

  • RBED is a system-prompt defense that casts the model as an assertive expert who ranks evidence above authority.
  • R-FT fine-tunes a model on pressure-resistant reasoning trajectories produced by a teacher model.

RBED helps but has limited effect. R-FT almost completely removes belief flips, keeps the model's general skills intact, and is linked through representation-level probing to consistent latent shifts. The study argues that clinical AI evaluation should go beyond single-turn accuracy and explicitly test and train for resilience in dialogue.

Key points

  • MED-STRESS uses four pressure strategies over T = 3 adversarial turns:
    • Baseline: repeated "are you sure?" doubt.
    • Authority: a senior doctor disagrees.
    • Logical Trap: plausible but wrong physiological arguments.
    • Safety Pressure: the wrong answer is framed as the safer choice.
  • Metrics:
    • ACC@i: accuracy at turn i.
    • IDC: initial diagnostic capability, equal to ACC@0.
    • MR, BSP and BRS: belief preservation among initially correct cases.
    • VCR: verbal compliance under pressure.
  • In an ablation, Llama-3.1-8B was fine-tuned only on correct single-turn DeepSeek-R1 chains of thought.
    • Robustness rose from 1.55% to just 6.38%.
    • Full R-FT reached 99.84%.
    • This shows that training on the adversarial turns is what drives the gain.
  • On MMLU subsets, R-FT showed:
    • +14.44 pp on High School Math.
    • +15.11 pp on Philosophy.
    • No change on Computer Security.
    • −1.17 pp on World Religions.
  • Further analyses and data:
    • Corrigibility was checked with a rational-updating test on 800 questions, where valid corrective evidence was supplied.
    • RepE probing found a resilience-related direction at Layer 12.
    • The data come from public sets such as MedQA and PubMedQA, plus synthetic material.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Harbin Institute of TechnologyHarbin · China185 works · 29,140 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksMedicine · 275 worksHealth Informatics · 214 works

Cite

Cite as Boyu Xiao, Xiuqi Tian, Xuwen Song, Haochun Wang, Guanchun Song, Sendong Zhao and Bing Qin. 2026. When Correct Beliefs Collapse: Epistemic Resilience of LLMs under Clinical Pressure. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.395

@inproceedings{xiao2026when,
  title = {{When Correct Beliefs Collapse: Epistemic Resilience of LLMs under Clinical Pressure}},
  author = {Boyu Xiao, Xiuqi Tian, Xuwen Song, Haochun Wang, Guanchun Song, Sendong Zhao and Bing Qin},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.395},
  url = {https://doi.org/10.18653/v1/2026.acl-long.395}
}

References

44 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Manar Aljohani, Jun Hou, Sindhura Kommu, and Xuan Wang. 2025. A comprehensive survey on the trustworthiness of large language models in healthcare. arXiv preprint arXiv:2502.15871. find
  2. Anthropic. 2025. Introducing claude 4: Next-generation models for coding and reasoning. https://www.anthropic.com/news/claude-4. ↗
  3. Configurations that act only at inference time (Warning Prompt and RBED) leave Turn-0 accuracy unchanged, and therefore share the same IDC as vanilla in this matched evaluation setting. find
  4. Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. 2024. Refusal in language models is mediated by a single direction. Advances in Neural Information Processing Systems, 37:136037–136083.
  5. Hanjie Chen, Zhouxiang Fang, Yash Singla, and Mark Dredze. 2025a. Benchmarking large language models on answering and explaining challenging medical questions. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Languag…
  6. Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, Jianye Hou, and Benyou Wang. 2024a. Huatuogpt-o1, towards medical complex reasoning with llms. Preprint, arXiv:2412.18925.
  7. Shan Chen, Mingye Gao, Kuleen Sasse, Thomas Hartvigsen, Brian Anthony, Lizhou Fan, Hugo Aerts, Jack Gallifant, and Danielle S Bitterman. 2025b. When helpfulness backfires: Llms and the risk of false medical information due to sycophantic behavior. npj Digital Medicine, 8(1):605. find
  8. Wei Chen, Zhen Huang, Liang Xie, Binbin Lin, Houqiang Li, Le Lu, Xinmei Tian, Deng Cai, Yonggang Zhang, Wenxiao Wang, and 1 others. 2024b. From yes-men to truth-tellers: Addressing sycophancy in large language models with pinpoint tuning. In International Conference on Machine Learning, pages 6950–6… find
  9. Xiaolan Chen, Jiayang Xiang, Shanfu Lu, Yexin Liu, Mingguang He, and Danli Shi. 2025c. Evaluating large language models and agents in healthcare: key challenges in clinical applications. Intelligent Medicine. find
  10. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan-Jiang Jiang, and 3416 other…
  11. Tri Dao. 2024. Flashattention-2: Faster attention with better parallelism and work partitioning. In The Twelfth International Conference on Learning Representations.
  12. Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2019. Plug and play language models: A simple approach to controlled text generation. arXiv preprint arXiv:1912.02164.
  13. DeepSeek-AI, Aixin Liu, Bei Feng, and 1 others. 2025a. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. Preprint, arXiv:2501.12948.
  14. DeepSeek-AI, Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenhao Xu, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, and 245 others. 2025b. Deepseek-v3.2: Pushing the frontier of ope…
  15. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — The Llama 3 Herd of Models
  16. Zikun Guo, Jingwei Lv, Xinyue Xu, Shu Yang, Jun Wen, Di Wang, and Lijie Hu. 2025. Benchmarking and mitigating sycophancy in medical vision language models. arXiv preprint arXiv:2509.21979. — [arXiv:2509.21979](https://arxiv.org/abs/2509.21979) find
  17. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR).
  18. Evan Hernandez, Belinda Z Li, and Jacob Andreas. 2023. Inspecting and editing knowledge representations in language models. arXiv preprint arXiv:2304.00740.
  19. Jiseung Hong, Grace Byun, Seungone Kim, and Kai Shu. 2025. Measuring sycophancy of language models in multi-turn dialogues. arXiv preprint arXiv:2505.23840. find
  20. Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2021. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):6421.
  21. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019. PubMedQA: A dataset for biomedical research question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Proce…
  22. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles, SOSP ’23, page 611…
  23. Kenneth Li, Tianle Liu, Naomi Bashkansky, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2024. Measuring and controlling instruction (in) stability in language model dialogs. arXiv preprint arXiv:2402.10962. — [arXiv:2402.10962](https://arxiv.org/abs/2402.10962) find
  24. Joshua Liu, Aarav Jain, Soham Takuri, Srihan Vege, Aslihan Akalin, Kevin Zhu, Sean O’Brien, and Vasu Sharma. 2025. Truth decay: Quantifying multi-turn sycophancy in language models. CoRR. find
  25. Lars Malmqvist. 2025. Sycophancy in large language models: Causes and mitigations. In Intelligent Computing-Proceedings of the Computing Conference, pages 61–74. Springer.
  26. OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander M adry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, and 401 others. 2024. Gpt-4o system card.…
  27. Yujin Potter, Shiyang Lai, Junsol Kim, James Evans, and Dawn Song. 2024. Hidden persuaders: Llms’ political leaning and their influence on voters. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 4244– 4275. find
  28. Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the parameters of a language model? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5418–5426, Online. Association for Computational Linguistics. find
  29. Kyra L Rosen, Margaret Sui, Kimia Heydari, Elizabeth J Enichen, and Joseph C Kvedar. 2025. The perils of politeness: how large language models may amplify medical misinformation. NPJ Digital Medicine, 8(1):644. find
  30. Sarah Sandmann, Stefan Hegselmann, Michael Fujarski, Lucas Bickmann, Benjamin Wild, Roland Eils, and Julian Varghese. 2025. Benchmark evaluation of deepseek large language models in clinical decision-making. Nature Medicine, pages 1–1. find
  31. Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Esin DURMUS, Zac Hatfield-Dodds, Scott R Johnston, Shauna M Kravec, Timothy Maxwell, Sam Mc-Candlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Perez. 2024. Towards under…
  32. Elias Stengel-Eskin, Peter Hase, and Mohit Bansal. 2025. Teaching models to balance resisting and accepting persuasion. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers),… find
  33. Alessandro Stolfo, Vidhisha Balachandran, Safoora Yousefi, Eric Horvitz, and Besmira Nushi. 2024. Improving instruction-following in language models through activation steering. arXiv preprint arXiv:2410.12877. — [arXiv:2410.12877](https://arxiv.org/abs/2410.12877) find
  34. Bryan Chen Zhengyu Tan, Daniel Wai Kit Chin, Zhengyuan Liu, Nancy Chen, and Roy Ka-Wei Lee. 2025. Persuasion dynamics in llms: Investigating robustness and adaptability in knowledge and safety with duet-pd. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pa… find
  35. Xinyu Tang, Zhihao Lv, Xiaoxue Cheng, Junyi Li, Wayne Xin Zhao, Zujie Wen, Zhiqiang Zhang, and Jun Zhou. 2025. Enhancing cross-task transfer of large language models via activation steering. arXiv preprint arXiv:2507.13236. — [arXiv:2507.13236](https://arxiv.org/abs/2507.13236) find
  36. Tao Tu, Anil Palepu, Mike Schaekermann, Khaled Saab, Jan Freyberg, Ryutaro Tanno, Amy Wang, Brenna Li, Mohamed Amin, Nenad Tomasev, and 1 others. 2024. Towards conversational diagnostic ai. arXiv preprint arXiv:2401.05654. — [arXiv:2401.05654](https://arxiv.org/abs/2401.05654) find
  37. Keyu Wang, Jin Li, Shu Yang, Zhuoran Zhang, and Di Wang. 2025a. When truth is overridden: Uncovering the internal origins of sycophancy in large language models. arXiv preprint arXiv:2508.02087. — [arXiv:2508.02087](https://arxiv.org/abs/2508.02087) find
  38. Wenxuan Wang, Zizhan Ma, Zheng Wang, Chenghan Wu, Jiaming Ji, Wenting Chen, Xiang Li, and Yixuan Yuan. 2025b. A survey of llm-based agents in medicine: How far are we from baymax? arXiv preprint arXiv:2502.11211. find
  39. Rongwu Xu, Brian Lin, Shujian Yang, Tianqi Zhang, Weiyan Shi, Tianwei Zhang, Zhixuan Fang, Wei Xu, and Han Qiu. 2024. The earth is flat because...: Investigating LLMs’ belief towards misinformation via persuasive conversation. In Proceedings of the 62nd Annual Meeting of the Association for Computat… find
  40. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  41. Binwei Yao, Chao Shang, Wanyu Du, Jianfeng He, Ruixue Lian, Yi Zhang, Hang Su, Sandesh Swamy, and Yanjun Qi. 2025. Peacemaker or troublemaker: How sycophancy shapes multi-agent debate. arXiv preprint arXiv:2509.23055. — [arXiv:2509.23055](https://arxiv.org/abs/2509.23055) find
  42. Botai Yuan, Yutian Zhou, Yingjie Wang, Fushuo Huo, Yongcheng Jing, Li Shen, Ying Wei, Zhiqi Shen, Ziwei Liu, Tianwei Zhang, and 1 others. 2025. Echobench: Benchmarking sycophancy in medical large vision-language models. arXiv preprint arXiv:2509.20146. — [arXiv:2509.20146](https://arxiv.org/abs/2509.20146) find
  43. Kaiwei Zhang, Qi Jia, Zijian Chen, Wei Sun, Xiangyang Zhu, Chunyi Li, Dandan Zhu, and Guangtao Zhai. 2025. Sycophancy under pressure: Evaluating and mitigating sycophantic bias via adversarial dialogues in scientific qa. arXiv preprint arXiv:2508.13743. — [arXiv:2508.13743](https://arxiv.org/abs/2508.13743) find
  44. Andy Zou, Long Phan, Sarah Chen, and 1 others. 2023. Representation engineering: A top-down approach to ai transparency. arXiv preprint arXiv:2310.01405. *Figure 5: Aggregated Baseline Performance. The average BRS and VCR for all models across the complete test suite.*

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xiao-2026-when-correct-beliefs-collapse-epistemic/