Elamur Research⌕
☰
home / papers / 2026 / Truth or Dare: Analyzing LLM Susceptibility to External Evidence of Varying Factuality

Truth or Dare: Analyzing LLM Susceptibility to External Evidence of Varying Factuality

Paper · 2026 · Han-Yu Su, Kuan-Yu Chu, Yung-Hui Li et al.

Summary

This paper asks how the specific properties of false retrieved evidence affect how easily LLMs are misled in retrieval-augmented question answering. Prior work mostly showed that misinformation causes harm. Here the authors vary three factors under controlled conditions: the rhetorical style of incorrect evidence, the number of false passages, and how semantically close a false passage is to the model's own evidence-free answer. They sampled 1,000 questions each from ASQA and ConflictQA and tested GPT-5-mini, Claude-sonnet-4.5, Llama-3-8B-Instruct and Qwen-3-8B. ConflictQA was scored with exact match. ASQA was scored with a GPT-4o factuality score (FS).

Evidence that explicitly tells the model to trust it over its own knowledge was by far the most damaging. Adding more false passages steadily lowered quality, with each extra passage hurting less than the last. A single correct passage partly offset this damage. False passages rewritten to resemble the model's parametric answer overrode correct internal knowledge more often. The work shows that RAG safeguards need to consider how evidence is phrased, not only whether it is accurate.

Key points

  • The paper compares four styles of false evidence: Plain, Instructive, Authoritative and Emotional. On ConflictQA, Instructive evidence lowered exact-match accuracy below 17% for every model, down from about 92–96% with real evidence. Emotional framing was the weakest of the content-only styles.
  • Without any evidence, ConflictQA accuracy ranged from 54.3% (GPT-5-mini) down to 16.4% (Qwen-3-8B).
  • In the quantity experiments, replacing the correct passage with one false passage cut average ConflictQA accuracy from 93.4% to 12.6%. Adding one false passage while keeping the correct one cost only about 19% on average.
  • The similarity experiments used all-MiniLM-L6-v2 cosine similarity to measure closeness to the model's answer. DeBERTa-v3 NLI checked that each pair of passages asserted the same false claim. The high-similarity versions raised the share of incorrect responses (FS ≤ 2) for all four models.
  • Stated limitations: the evidence was injected artificially, the study is English-only and open-domain, and it proposes no defenses.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓National Taiwan UniversityTaipei · Taiwan29 works · 2,116 citations🔬Academia SinicaTaipei · Taiwan10 works · 760 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Han-Yu Su, Kuan-Yu Chu, Yung-Hui Li and Lun-Wei Ku. 2026. Truth or Dare: Analyzing LLM Susceptibility to External Evidence of Varying Factuality. Proceedings of the 6th Workshop on Trustworthy NLP (TrustNLP 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.trustnlp-main.39

@inproceedings{su2026truth,
  title = {{Truth or Dare: Analyzing LLM Susceptibility to External Evidence of Varying Factuality}},
  author = {Han-Yu Su, Kuan-Yu Chu, Yung-Hui Li and Lun-Wei Ku},
  booktitle = {Proceedings of the 6th Workshop on Trustworthy NLP (TrustNLP 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.trustnlp-main.39},
  url = {https://doi.org/10.18653/v1/2026.trustnlp-main.39}
}

References

17 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yizhong Wang Avirup Sil Hannaneh Hajishirzi Akari Asai, Zeqiu Wu. 2023. Self-rag: Learning to retrieve, generate, and critique through self-reflection. arXiv:2310.11511.
  2. Victor Zhong Rajarshi Das Daniel Khashabi Hannaneh Hajishirzi Alex Mallen, Akari Asai. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. arXiv:2212.10511. find
  3. Sheng Shen Dan Klein Alexander Wan, Eric Wallace. 2023. Poisoning language models during instruction tuning. Proceedings of the 40th International Conference on Machine Learning. find
  4. Patrick Lewis Anton Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel Fabio Petroni, Tim Rocktäschel. 2019. Language models as knowledge bases? arXiv:1909.01066.
  5. Alberto Barrón-Cedeño Rostislav Petrov Preslav Nakov Giovanni Da San Martino, Seunghak Yu. 2019. Fine-grained analysis of propaganda in news articles. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language…
  6. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. International Conference on Learning Representation.
  7. Bhuwan Dhingra Ming-Wei Chang Ivan Stelmakh, Yi Luan. 2022. Asqa: Factoid questions meet long-form answers. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.
  8. Ana Marasović William Agnew Gabriel Ilharco Dirk Groeneveld Margaret Mitchell Matt Gardner Jesse Dodge, Maarten Sap. 2021. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. find
  9. Jiangjie Chen Renze Lou Yu Su Jian Xie, Kai Zhang. 2024. Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts. The Twelfth International Conference on Learning Representations.
  10. Xianpei Han Le Sun Jiawei Chen, Hongyu Lin. 2023. Benchmarking large language models in retrieval-augmented generation. The Thirty-Eighth AAAI Conference on Artificial Intelligence.
  11. Zhichao Shi Hexiang Tan Xuehao Zhai Chengjin Xu Wei Li Yinghan Shen-Shengjie Ma Honghao Liu Saizhuo Wang Kun Zhang Yuanzhuo Wang-Wen Gao Lionel Ni Jian Guo Jiawei Gu, Xuhui Jiang. 2024. A survey on llm-as-a-judge. arXiv:2411.15594. find
  12. Zhuoran Jin, Pengfei Cao, Hongbang Yuan, Yubo Chen, Jiexin Li, Qianghua Cheng, Carong Liu, Luewei Li, Jun Zhao, and Kang Liu. 2024. Cutting off the head ends the conflict: A mechanism for interpreting and mitigating knowledge conflicts in language models. In Findings of the Association for Computati… find
  13. Preslav Nakov Yejin Choi Heng Ji Kung-Hsiang Huang, Kathleen McKeown. 2023. Faking fake news for real fake news detection: Propaganda-loaded training data generation. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). find
  14. Sid Black Laurence Golding Travis Hoppe Charles Foster Jason Phang Horace He-Anish Thite Noa Nabeshima Shawn Presser Connor Leahy Leo Gao, Stella Biderman. 2020. The pile: An 800gb dataset of diverse text for language modeling. arXiv:2101.00027. find
  15. Jianheng Tang Jia Li Miao Peng, Nuo Chen. 2025. How does misinformation affect large language model behaviors and preferences? Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). find
  16. Jordan Hoffmann Cyprien de Masson d’Autume Phil Blunsom Aida Nematzadeh Xiang Lorraine Li, Adhiguna Kuncoro. 2022. A systematic investigation of commonsense knowledge in large language models. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. find
  17. Xinyu Gaob Kangxiang Jiab Jinliu Panb Yuxi Bic Yi Daia Jiawei Suna Meng Wangc Yunfan Gaoa, Yun Xiongb and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/su-2026-truth-or-dare-analyzing-llm/