Elamur Research⌕
☰
home / papers / 2026 / FaithLM: Towards Faithful Explanations for Large Language Models

FaithLM: Towards Faithful Explanations for Large Language Models

Paper · 2026 · Yu-Neng Chuang, Guanchu Wang, Chia-Yuan Chang et al.

Summary

LLMs often explain their answers in natural language, but these explanations frequently do not reflect what actually drove the prediction. Classic fidelity tests, which delete important tokens, also fit poorly with free-text rationales whose content is semantic rather than token-level.

FaithLM is a model-agnostic, post-hoc framework for black-box or API-only models that treats faithfulness as a causal intervention:

  • An explainer LLM writes an explanation for the target model's answer.
  • An agent then generates a "contrary hint" expressing the opposite meaning, which is appended to the target model's input.
  • How strongly the prediction shifts becomes the fidelity score. The authors prove (Theorem 1) that this score validly measures faithfulness and can tell faithful explanations apart from unfaithful ones.

Using the score as feedback, FaithLM iteratively improves both the explanation and the prompt that elicits it, through in-context learning. Across three datasets and several LLM backbones, it achieves higher fidelity than self-explanation baselines. Its explanations also align more closely with human rationales.

This matters because it offers a practical, internals-free way to audit and strengthen explanation faithfulness, which is valuable in high-stakes areas such as healthcare.

Key points

  • Evaluation datasets are ECQA, TriviaQA-Long and COPA; the main setup uses Claude-2 as explainer and Vicuna-7B as the target LLM, and the paper reports four LLMs overall.
  • FaithLM explicitly separates post-hoc explanation from CoT reasoning and self-refinement, arguing the latter two lack proper fidelity checks.
  • Optimized trigger prompts beat human-crafted ones on all three datasets. They also transfer without further tuning from ECQA to Social-IQA and from COPA to XCOPA.
  • On 100 sampled instances, NLI classifiers (Roberta-Large, XLNet-Large) labeled up to 86% (ECQA) and 82% (COPA) of contrary hints as contradictions of the original explanation. A GPT-4o classifier was also used in this check.
  • The main limitation is extra compute and carbon cost from the iterative optimization; code is released at github.com/ynchuang/FaithLM.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢GoogleMountain View · United States430 works · 424,363 citations🏢MetaMenlo Park · United States183 works · 228,558 citations🎓Johns Hopkins UniversityBaltimore · United States240 works · 96,487 citations🎓Rutgers, The State University of New JerseyNew Brunswick · United States36 works · 12,026 citations🎓Texas A&M UniversityCollege Station · United States26 works · 1,871 citations🎓Rice UniversityHouston · United States11 works · 1,850 citations🎓New Jersey Institute of TechnologyNewark · United States15 works · 1,333 citations🎓University of North Carolina at CharlotteCharlotte · United States6 works · 125 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Yu-Neng Chuang, Guanchu Wang, Chia-Yuan Chang, Ruixiang Tang, Shaochen Zhong, Fan Yang, Andrew Wen, Mengnan Du, Xuanting Cai, Vladimir Braverman and Xia Hu. 2026. FaithLM: Towards Faithful Explanations for Large Language Models. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-long.177

@inproceedings{chuang2026faithlm,
  title = {{FaithLM: Towards Faithful Explanations for Large Language Models}},
  author = {Yu-Neng Chuang, Guanchu Wang, Chia-Yuan Chang, Ruixiang Tang, Shaochen Zhong, Fan Yang, Andrew Wen, Mengnan Du, Xuanting Cai, Vladimir Braverman and Xia Hu},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.177},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.177}
}

References

52 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Shourya Aggarwal, Divyanshu Mandowara, Vishwajeet Agrawal, Dinesh Khandelwal, Parag Singla, and Dinesh Garg. 2021. Explanations for CommonsenseQA: New Dataset and Models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confe…
  3. Anthropic. 2023. Claude 2. find
  4. AnthropicAI. 2023. Introducing claude. See https://www.anthropic.com/index/introducing-claude. ↗
  5. Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2023. Longbench: A bilingual, multitask benchmark for long context understanding. Preprint, arXiv:2308.14508.
  6. Hugh Chen, Ian C Covert, Scott M Lundberg, and Su-In Lee. 2023a. Algorithms to estimate shapley value feature attributions. Nature Machine Intelligence, pages 1–12. find
  7. Qianglong Chen, Feng Ji, Xiangji Zeng, Feng-Lin Li, Ji Zhang, Haiqing Chen, and Yin Zhang. 2021. Kace: Generating knowledge aware contrastive explanations for natural language inference. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Internati…
  8. Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, et al. 2025. Reasoning models don’t always say what they think. arXiv preprint arXiv:2505.05410. find
  9. Yanda Chen, Ruiqi Zhong, Narutatsu Ri, Chen Zhao, He He, Jacob Steinhardt, Zhou Yu, and Kathleen McKeown. 2023b. Do models explain themselves? counterfactual simulatability of natural language explanations. arXiv preprint arXiv:2307.08678. find
  10. Zichen Chen, Ambuj K Singh, and Misha Sra. 2023c. Lmexplainer: a knowledge-enhanced explainer for language models. arXiv preprint arXiv:2303.16537. — [arXiv:2303.16537](https://arxiv.org/abs/2303.16537) find
  11. Jiale Cheng, Xiao Liu, Kehan Zheng, Pei Ke, Hongning Wang, Yuxiao Dong, Jie Tang, and Minlie Huang. 2023. Black-box prompt optimization: Aligning large language models without model training. arXiv preprint arXiv:2311.04155.
  12. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. find
  13. Yu-Neng Chuang, Guanchu Wang, Fan Yang, Zirui Liu, Xuanting Cai, Mengnan Du, and Xia Hu. 2023. Efficient xai techniques: A taxonomic survey. arXiv preprint arXiv:2302.03225. — [arXiv:2302.03225](https://arxiv.org/abs/2302.03225) find
  14. Mengnan Du, Ninghao Liu, and Xia Hu. 2019. Techniques for interpretable machine learning. Communications of the ACM, 63(1):68–77. find
  15. Luciano Floridi. 2019. Establishing the rules for building trustworthy ai. Nature Machine Intelligence, 1(6):261–262. find
  16. Bryce Goodman, Seth Flaxman, and Y X. 2017. European union regulations on algorithmic decision-making and a “right to explanation”. AI magazine, 38(3):50–57. find
  17. Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang. 2023. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. arXiv preprint arXiv:2309.08532. find
  18. Shiyuan Huang, Siddarth Mamidanna, Shreedhar Jangam, Yilun Zhou, and Leilani H Gilpin. 2023. Can large language models explain themselves? a study of llm-generated self-explanations. arXiv preprint arXiv:2310.11207. — [arXiv:2310.11207](https://arxiv.org/abs/2310.11207) find
  19. Mojan Javaheripi and Sébastien Bubeck. 2023. Phi-2: The surprising power of small language models. find
  20. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. triviaqa: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension. arXiv e-prints, arXiv:1705.03551.
  21. Pride Kavumba, Naoya Inoue, Benjamin Heinzerling, Keshav Singh, Paul Reisert, and Kentaro Inui. 2019. When choosing plausible alternatives, clever hans can be clever. In Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing, pages 33–42, Hong Kong, China. Associat… find
  22. Sawan Kumar and Partha Talukdar. 2020. Nile: Natural language inference with faithful natural language explanations. arXiv preprint arXiv:2005.12116.
  23. Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, et al. 2023. Measuring faithfulness in chain-of-thought reasoning. arXiv preprint arXiv:2307.13702.
  24. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. arXiv preprint arXiv:2305.20050.
  25. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. Gpteval: Nlg evaluation using gpt-4 with better human alignment. arXiv preprint arXiv:2303.16634.
  26. Rakesh R Menon, Kerem Zaman, and Shashank Srivastava. 2023. Mantle: Model-agnostic natural language explainer. arXiv preprint arXiv:2305.12995. find
  27. Miquel Miró-Nicolau, Antoni Jaume-i Capó, and Gabriel Moyà-Alcover. 2024. A comprehensive study on fidelity metrics for xai. arXiv preprint arXiv:2401.10640. — [arXiv:2401.10640](https://arxiv.org/abs/2401.10640) find
  28. Christoph Molnar. 2022. Interpretable machine learning: A guide for making black box models explainable. find
  29. Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020. Adversarial NLI: A new benchmark for natural language understanding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics.
  30. Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. 2022. Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. In Proceedings of the Conference on Health, Inference, and Learning, volume 174 of Proceedings of Machine Learning Research, pages 24…
  31. Gianluigi Lopardo, Frederic Precioso, and Damien Garreau. 2023. Faithful and robust local interpretability for textual predictions. arXiv preprint arXiv:2311.01605. — [arXiv:2311.01605](https://arxiv.org/abs/2311.01605) find
  32. Ansh Radhakrishnan, Karina Nguyen, Anna Chen, Carol Chen, Carson Denison, Danny Hernandez, Esin Durmus, Evan Hubinger, Jackson Kernion, Kamilė Lukošiūtė, et al. 2023. Question decomposition improves the faithfulness of model-generated reasoning. arXiv preprint arXiv:2307.11768. find
  33. Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, and Chris Callison-Burch. 2023. Faithful chain-of-thought reasoning. arXiv preprint arXiv:2301.13379.
  34. Melissa Roemmele, Cosmin Adrian Bejan, and Andrew S Gordon. 2011. Choice of plausible alternatives: An evaluation of commonsense causal reasoning. In 2011 AAAI Spring Symposium Series. find
  35. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2024. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36. find
  36. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik R Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. In Thirty-seventh Conference on Neural Information Processing Systems.
  37. Andreas Madsen, Sarath Chandar, and Siva Reddy. 2024. Are self-explanations from large language models faithful? In Findings of the Association for Computational Linguistics: ACL 2024, pages 295–337, Bangkok, Thailand. Association for Computational Linguistics. find
  38. Bodhisattwa Prasad Majumder, Oana-Maria Camburu, Thomas Lukasiewicz, and Julian McAuley. 2021. Knowledge-grounded self-rationalization via extractive and natural language explanations. arXiv preprint arXiv:2106.13876. — [arXiv:2106.13876](https://arxiv.org/abs/2106.13876) find
  39. Ramesh Manuvinakurike, Emanuel Moss, Elizabeth Anne Watkins, Saurav Sahay, Giuseppe Raffa, and Lama Nachman. 2025. Thoughts without thinking: Reconsidering the explanatory value of chain-of-thought reasoning in llms through agentic pipelines. arXiv preprint arXiv:2505.00875. — [arXiv:2505.00875](https://arxiv.org/abs/2505.00875) find
  40. Noah Y Siegel, Oana-Maria Camburu, Nicolas Heess, and Maria Perez-Ortiz. 2024. The probabilities also matter: A more faithful metric for faithfulness of free-text explanations in large language models. arXiv preprint arXiv:2404.03189. find
  41. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  42. Sree Harsha Tanneru, Dan Ley, Chirag Agarwal, and Himabindu Lakkaraju. 2024. On the hardness of faithful chain-of-thought reasoning in large language models. arXiv preprint arXiv:2406.10625. — [arXiv:2406.10625](https://arxiv.org/abs/2406.10625) find
  43. Paul Thomas, Seth Spielman, Nick Craswell, and Bhaskar Mitra. 2023. Large language models can accurately predict searcher preferences. arXiv preprint arXiv:2309.10621. — [arXiv:2309.10621](https://arxiv.org/abs/2309.10621) find
  44. Ye Tian, Baolin Peng, Linfeng Song, Lifeng Jin, Dian Yu, Haitao Mi, and Dong Yu. 2024. Toward self-improvement of llms via imagination, searching, and criticizing. arXiv preprint arXiv:2404.12253. find
  45. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  46. Miles Turpin, Julian Michael, Ethan Perez, and Samuel R Bowman. 2023. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. arXiv preprint arXiv:2305.04388.
  47. Guanchu Wang, Yu-Neng Chuang, Fan Yang, Mengnan Du, Chia-Yuan Chang, Shaochen Zhong, Zirui Liu, Zhaozhuo Xu, Kaixiong Zhou, Xuanting Cai, et al. 2023. Leta: Learning transferable attribution for generic vision explainer. arXiv preprint arXiv:2312.15359. — [arXiv:2312.15359](https://arxiv.org/abs/2312.15359) find
  48. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171.
  49. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2019. Huggingface’s transformers: State-ofthe-art natural language processing. arXiv preprint arXiv:1910.03771.
  50. Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. 2023. Large language models as optimizers. arXiv preprint arXiv:2309.03409.
  51. Xi Ye and Greg Durrett. 2022. The unreliability of explanations in few-shot prompting for textual reasoning. Advances in neural information processing systems, 35:30378–30392.
  52. Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2023. Explainability for large language models: A survey. arXiv preprint arXiv:2309.01029.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/chuang-2026-faithlm-faithful-explanations-large-language/