Elamur Research⌕
☰
home / papers / 2024 / Towards Faithful Model Explanation in NLP: A Survey

Towards Faithful Model Explanation in NLP: A Survey

Summary

Neural NLP models are hard to interpret, and many explanation techniques have been proposed. The authors argue that one property, faithfulness, is often neglected or confused with plausibility. Faithfulness means an explanation truly reflects how the model reached its prediction. Unfaithful but convincing explanations can mask reliance on sensitive attributes such as race or gender.

This survey examines more than 110 explanation methods with faithfulness as the central criterion. It clarifies the concept, critically compares the existing ways of evaluating it, and introduces a five-family taxonomy. For each family it weighs representative work, strengths, and weaknesses. The authors run no new experiments and do not rank families overall.

Their main finding is that no single standard for measuring faithfulness exists. They recommend choosing methods that are designed for faithfulness and empirically tested for it, while staying aware of known pitfalls. The survey serves as both a practical guide for practitioners and a research agenda for the field.

Key points

  • The taxonomy has five families: similarity-based methods, analysis of model-internal structures (neurons, attention), backpropagation-based methods, counterfactual intervention, and self-explanatory models.
  • Evaluation approaches reviewed include axiomatic, predictive power, robustness, perturbation-based, white-box, and human perception evaluation.
  • Gradient and propagation methods (Integrated Gradients, SmoothGrad, LRP, DeepLift, etc.) show problems like saturation and input sensitivity. In counterfactual intervention, perturbation tends to be more faithful than erasure because erasure creates out-of-distribution inputs.
  • Self-explanatory models often claim to be faithful by design, but issues such as label leakage weaken these claims.
  • The appendix lists tools: BertViz, LIT, LSTMVis, Seq2Seq-Vis, Captum, AllenNLP Interpret, LIME, SHAP, Anchors, and the What-if Tool.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

76 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 73 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli_a_00511

@article{lyu2024faithful,
  title = {{Towards Faithful Model Explanation in NLP: A Survey}},
  author = {Lyu, Qing and Apidianaki, Marianna and Callison-Burch, Chris},
  journal = {Computational Linguistics},
  year = {2024},
  doi = {10.1162/coli\_a\_00511},
  url = {https://doi.org/10.1162/coli_a_00511}
}

References

97 references, 35 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abnar, Samira and Willem Zuidema. 2020. Quantifying attention flow in transformers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4190–4197. https://doi.org/10.18653/v1/2020 .acl-main.385 Abraham, Eldar David, Karel D’Oosterlinck, Amir Feder, Yair… ↗ our card
  2. Adebayo, Julius, Justin Gilmer, Michael Muelly, Ian J. Goodfellow, Moritz Hardt, and Been Kim. 2018. Sanity checks for saliency maps. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, pages 9525–9536. find
  3. Adebayo, Julius, Michael Muelly, Ilaria Liccardi, and Been Kim. 2020. Debugging tests for model explanations. In Advances in Neural Information Processing Systems, 33:700–712. find
  4. Adi, Yossi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2017. Fine-grained analysis of sentence embeddings using auxiliary prediction tasks. In 5th International Conference on Learning Representations, ICLR 2017. find our card
  5. Alvarez-Melis, David and Tommi S. Jaakkola. 2018. On the robustness of interpretability methods. ArXiv preprint, abs/1806.08049. — [arXiv:1806.08049](https://arxiv.org/abs/1806.08049) find
  6. Alvarez-Melis, David and Tommi S. Jaakkola. 2018. Towards robust interpretability with self-explaining neural networks. In Advances in Neural Information Processing Systems 31: Annual Conference on 36 https://allenai.github.io/allennlp-website/interpret. 37 https://captum.ai/. 38 https://www.eecs.tu… ↗
  7. Neural Information Processing Systems 2018, NeurIPS 2018, pages 7786–7795. find
  8. Amini, Afra, Tiago Pimentel, Clara Meister, and Ryan Cotterell. 2022. Naturalistic causal probing for morpho-syntax. In Transactions of the Association for Computational Linguistics, 11:384–403. https://doi.org/10.1162/tacl_a_00554 ↗
  9. Andreas, Jacob, Marcus Rohrbach, Trevor Darrell, and Dan Klein. 2016a. Learning to compose neural networks for question answering. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1545–1554. https… find our card
  10. Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Bengio, Yoshua and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015. find our card
  11. Bansal, Gagan, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the whole exceed its parts? The effect of AI explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Syste… find our card
  12. Bau, Anthony, Yonatan Belinkov, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and James R. Glass. 2019. Identifying and controlling important neurons in neural machine translation. In 7th International Conference on Learning Representations, ICLR 2019. find
  13. Belinkov, Yonatan, Nadir Durrani, Fahim Dalvi, Hassan Sajjad, and James Glass. 2020. On the linguistic representational power of neural machine translation models. Computational Linguistics, 46(1):1–52. https://doi.org/10.1162/coli a 00367 Belinkov, Yonatan and James Glass. 2019. Analysis methods i… ↗ our card
  14. Bowman, Samuel R., Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642. https://doi s.org/10.18653/v1/D15-1075 Bro… find
  15. Brunner, Gino, Yuyi Wang, Roger Wattenhofer, and Michael Weigelt. 2018. Natural language multitasking: Analyzing and improving syntactic saliency of hidden representations. arXiv preprint arXiv:1801.06024. — [arXiv:1801.06024](https://arxiv.org/abs/1801.06024) find
  16. Calderon, Nitay, Eyal Ben-David, Amir Feder, and Roi Reichart. 2022. DoCoGen: Domain counterfactual generation for low resource domain adaptation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7727–7746. https://doi.org/10.… find
  17. Camburu, Oana Maria, Brendan Shillingford, Pasquale Minervini, Thomas Lukasiewicz, and Phil Blunsom. 2020. Make up your mind! Adversarial generation of inconsistent natural language explanations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4157–4… find
  18. Chen, Wenhu, Xueguang Ma, Xinyi Wang, and William W. Cohen. 2022b. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. Transactions on Machine Learning Research. find
  19. Choenni, Rochelle, Ekaterina Shutova, and Robert van Rooij. 2021. Stepmothers are mean and academics are pretentious: What do pretrained language models learn about you? In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1477–1491. https://doi.org/10.186… find
  20. Clark, Peter, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? Try ARC, the AI2 Reasoning Challenge. abs/1803.05457. find
  21. Clinciu, Miruna Adriana, Arash Eshghi, and Helen Hastie. 2021. A study of automatic metrics for the evaluation of natural language explanations. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2376–2387. https://doi.o… ↗ our card
  22. Dalvi, Bhavana, Peter Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighanna Pipatanangkura, and Peter Clark. 2021. Explaining answers with entailment trees. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7358–7370. https://doi.org/10.18653/v1… ↗ our card
  23. Danilevsky, Marina, Kun Qian, Ranit Aharonov, Yannis Katsis, Ban Kawas, and Prithviraj Sen. 2020. A survey of the state of explainable AI for natural language processing. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th In… find our card
  24. De Cao, Nicola, Michael Sejr Schlichtkrull, Wilker Aziz, and Ivan Titov. 2020. How do decisions emerge across layers in neural models? Interpretation with differentiable masking. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3243–3255. https… find
  25. Deutsch, Daniel, Shyam Upadhyay, and Dan Roth. 2019. A general-purpose algorithm for constrained sequential inference. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 482–492. https://doi.org/10.18653/v1/K19-1045 Devlin, Jacob, Ming-Wei Chang, Kenton… ↗
  26. DeYoung, Jay, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. 2020. ERASER: A benchmark to evaluate rationalized NLP models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4443–4458. https://doi… find our card
  27. Dua, Dheeru, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019. DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisti… find our card
  28. Eberle, Oliver, Stephanie Brandl, Jonas Pilot, and Anders Søgaard. 2022. Do transformer models show similar attention patterns to task-specific human gaze? In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4295–4309. https://doi… find
  29. Feder, Amir, Nadav Oved, Uri Shalit, and Roi Reichart. 2021. CausaLM: Causal model explanation through counterfactual language models. Computational Linguistics, 47(2):333–386. https://doi.org/10.1162/coli a 00404 Feng, Shi, Eric Wallace, Alvin Grissom II, Mohit Iyyer, Pedro Rodriguez, and Jordan B… ↗
  30. Gardner, Matt, Yoav Artzi, Victoria Basmov, et al. 2020. Evaluating models’ local decision boundaries via contrast sets. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1307–1323. https://doi.org/10.18653/v1/2020 .findings-emnlp.117 Geva, Mor, Yoav Goldberg, and Jonat… ↗
  31. Haghighatkhah, Pantea, Antske Fokkens, Pia Sommerauer, Bettina Speckmann, and Kevin Verbeek. 2022. Better hit the nail on the head than beat around the bush: Removing protected attributes with a single projection. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Process… find our card
  32. Elsevier. Hase, Peter and Mohit Bansal. 2020. Evaluating explainable AI: Which algorithmic explanations help users predict model behavior? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5540–5552. https://doi.org/10.18653/v1/2020 .acl-main.491 Hase,… ↗
  33. Herman, Bernease. 2017. The promise and peril of human evaluation for model interpretability. ArXiv preprint, abs/1711.07414. — [arXiv:1711.07414](https://arxiv.org/abs/1711.07414) find
  34. Hewitt, John and Percy Liang. 2019. Designing and interpreting probes with control tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2733–2743. https://doi.o… ↗
  35. Hooker, Sara, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. 2019. A benchmark for interpretability methods in deep neural networks. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, pages 9734–9745. find
  36. Hu, Ronghang, Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Kate Saenko. 2017. Learning to reason: End-to-end module networks for visual question answering. In IEEE International Conference on Computer Vision, ICCV 2017, pages 804–813. https://doi.org/10 .1109/ICCV.2017.93 Jacovi, Alon and Yoa… find
  37. Jain, Sarthak, Sarah Wiegreffe, Yuval Pinter, and Byron C. Wallace. 2020. Learning to faithfully rationalize by construction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4459–4473. https://doi.org/10 .18653/v1/2020.acl-main.409 Janizek, Joseph D.… find our card
  38. Jiang, Chengyue, Yinggong Zhao, Shanbo Chu, Libin Shen, and Kewei Tu. 2020. find
  39. Cold-start and interpretability: Turning regular expressions into trainable recurrent neural networks. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3193–3207. https://doi.org/10.18653/v1/2020.emnlp -main.258 Jiang, Yichen, Nitish Joshi, Ye… ↗
  40. Karpathy, Andrej, Justin Johnson, and Li Fei-Fei. 2015. Visualizing and understanding recurrent networks. ArXiv preprint, abs/1506.02078. Kassner, Nora, Oyvind Tafjord, Hinrich Schütze, and Peter Clark. 2021. BeliefBank: Adding memory to a pre-trained language model for a systematic notion of belie… find our card
  41. Kaushik, Divyansh and Zachary C. Lipton. 2018. How much reading does reading comprehension require? A critical investigation of popular benchmarks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 5010–5015. https://doi.org/10.18653/v1/D18-1546 Kim, B… ↗ our card
  42. Koh, Pang Wei and Percy Liang. 2017. Understanding black-box predictions via influence functions. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, pages 1885–1894. find our card
  43. Kojima, Takeshi, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in Neural Information Processing Systems, 35:22199–22213. find our card
  44. Kokhlikyan, Narine, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, and Orion Reblitz-Richardson. 2020. Captum: A unified and generic model interpretability library for PyTorch. ArXiv preprint, abs/2009.07… — [arXiv:2009.07896](https://arxiv.org/abs/2009.07896) find
  45. Krishnamurthy, Jayant and Thomas Kollar. 2013. Jointly learning to parse and perceive: Connecting natural language to the physical world. Transactions of the Association for Computational Linguistics, 1:193–206. https://doi.org/10 .1162/tacl a 00220 Kumar, Abhinav, Chenhao Tan, and Amit Sharma. 2022… find
  46. Kumar, Sawan and Partha Talukdar. 2020. NILE: Natural language inference with faithful natural language explanations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8730–8742. https://doi.org/10 .18653/v1/2020.acl-main.771 Kunkel, Johannes, Tim Donk… find
  47. Laugel, Thibault, Xavier Renard, Marie-Jeanne Lesot, Christophe Marsala, and Marcin Detyniecki. 2018. Defining locality for surrogates in post-hoc interpretablity. In Workshop on Human Interpretability for Machine Learning (WHI)-International Conference on Machine Learning (ICML). find
  48. Lei, Tao, Regina Barzilay, and Tommi Jaakkola. 2016. Rationalizing neural predictions. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 107–117. https://doi.org/10.18653/v1/D16-1011 Levesque, Hector, Ernest Davis, and Leora Morgenstern. 2012. The Wino… ↗ our card
  49. Lewkowycz, Aitor, Anders Andreassen, David Dohan, et al. 2022. Solving quantitative reasoning problems with language models. Advances in Neural Information Processing Systems, 35:3843–3857. find our card
  50. Li, Jierui, Lemao Liu, Huayang Li, Guanlin Li, Guoping Huang, and Shuming Shi. 2020. Evaluating explanation methods for neural machine translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 365–375. https://doi.org/10 .18653/v1/2020.acl-main.35… find our card
  51. Li, Yifei, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen. 2022. On the Advance of Making Language Models Better Reasoners. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5315–5333. https://doi.o… ↗
  52. Liu, Yibing, Haoliang Li, Yangyang Guo, Chenqi Kong, Jing Li, and Shiqi Wang. 2022. Rethinking attention-model explainability through faithfulness violation test. In International Conference on Machine Learning, ICML 2022, pages 13807–13824. find
  53. Liu, Yinhan, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. ArXiv preprint, abs/1907.11692. find
  54. Lovering, Charles, Rohan Jha, Tal Linzen, and Ellie Pavlick. 2020. Information-theoretic probing explains reliance on spurious features. In International Conference on Learning Representations. find
  55. Lu, Kaiji, Zifan Wang, Piotr Mardziel, and Anupam Datta. 2021. Influence patterns for explaining information flow in BERT. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, pages 4461–4474. find
  56. Lundberg, Scott M. and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, pages 4765–4774. find our card
  57. Lyu, Qing, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, and Chris Callison-Burch. 2023. Faithful chain-of-thought reasoning. In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapte… find our card
  58. Mao, Jiayuan, Chuang Gan, Pushmeet Kohli, Joshua B. Tenenbaum, and Jiajun Wu. 2019. The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. In 7th International Conference on Learning Representations, ICLR 2019. find
  59. Marasovic, Ana, Iz Beltagy, Doug Downey, and Matthew Peters. 2022. Few-shot self-rationalization with natural language prompts. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 410–424. https://doi.org/10 .18653/v1/2022.findings-naacl.31 Martins, André F. T. and Ramó… find our card
  60. McCoy, Tom, Ellie Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3428–3448. https://doi.org/10.18653/v1/P19-1334 ↗ our card
  61. Miller, Tim. 2017. Explanation in artificial intelligence: Insights from the social sciences. ArXiv preprint, abs/1706.07269. find our card
  62. Ming, Yao, Shaozu Cao, Ruixiang Zhang, Zhen Li, Yuanzhe Chen, Yangqiu Song, and Huamin Qu. 2017. Understanding hidden memories of recurrent neural networks. In 2017 IEEE Conference on Visual Analytics Science and Technology (VAST), pages 13–24. https://doi.org/10.1109/VAST.2017.8585721 Montavon, Gr… — [doi:10.1016/j.patcog.2016.11.008](https://doi.org/10.1016/j.patcog.2016.11.008) ↗
  63. Mosca, Edoardo, Ferenc Szigeti, Stella Tragianni, Daniel Gallagher, and Georg Groh. 2022. SHAP-based explanation methods: A review for NLP interpretability. In Proceedings of the 29th International Conference on Computational Linguistics, pages 4593–4603. find
  64. Mueller, Aaron, Yu Xia, and Tal Linzen. 2022. Causal analysis of syntactic agreement neurons in multilingual language models. In Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL), pages 95–109. https://doi.org/10.18653/v1/2022.conll-1.8 Mullenbach, James, Sarah W… ↗
  65. Murdoch, W. James, Chandan Singh, Karl Kumbier, Reza Abbasi-Asl, and Bin Yu. 2019. Definitions, methods, and applications in interpretable machine learning. Proceedings of the National Academy of Sciences, 116(44):22071–22080. https://doi.org/10.1073/pnas .1900654116, PubMed: 31619572 Mylonas, Nikol… ↗
  66. Nie, Weili, Yang Zhang, and Ankit Patel. 2018. A theoretical explanation for perplexing behaviors of backpropagation-based visualizations. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, pages 3806–3815. find
  67. Nye, Maxwell, Anders Johan Andreassen, Guy Gur-Ari, et al. 2021. Show your work: Scratchpads for intermediate computation with language models. Deep Learning for Code Workshop. find our card
  68. OpenAI. 2023. GPT-4 technical report. arXiv preprint arXiv:2303.08774. find our card
  69. Parcalabescu, Letitia and Anette Frank. 2024. On measuring faithfulness or self-consistency of natural language explanations. arXiv, cs.CL. find
  70. Park, Dong Huk, Lisa Anne Hendricks, Zeynep Akata, Anna Rohrbach, Bernt Schiele, Trevor Darrell, and Marcus Rohrbach. 2018. Multimodal explanations: Justifying decisions and pointing to the evidence. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, pages 8779–8788. http… find
  71. Raffel, Colin, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21:140:1–140:67. Raganato, Alessandro, Yves Sc… find
  72. Ravfogel, Shauli, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. 2020. Null it out: Guarding protected attributes by iterative nullspace projection. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7237–7256. https://doi.org/10 .18653/v1… find our card
  73. Sajjad, Hassan, Nadir Durrani, and Fahim Dalvi. 2022. Neuron-level interpretation of deep NLP models: A survey. Transactions of the Association for Computational Linguistics, 10:1285–1303. https://doi.org/10 .1162/tacl a 00519 Sakaguchi, Keisuke, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2… find our card
  74. Shapley, L. S. 1953. 17. A value for n-person games. In Harold William Kuhn and Albert William Tucker, editors, Contributions to the Theory of Games (AM-28), Volume II. Princeton University Press, pages 307–318. https://doi.org/10.1515/9781400881970-018 Shrikumar, Avanti, Peyton Greenside, and Ansh… — [doi:10.1515/9781400881970-018](https://doi.org/10.1515/9781400881970-018) ↗
  75. Shrikumar, Avanti, Peyton Greenside, Anna Shcherbina, and Anshul Kundaje. 2017. Not just a black box: Learning important features through propagating activation differences. ArXiv preprint, arXiv:1605.01713 [cs]. find our card
  76. Sia, Suzanna, Anton Belyy, Amjad Almahairi, Madian Khabsa, Luke Zettlemoyer, and Lambert Mathias. 2022. Logical satisfiability of counterfactuals for faithful explanations in NLI. Proceedings of the AAAI Conference on Artificial Intelligence, pages 9837–9845. https://doi.org/10 .1609/aaai.v37i8.2617… find
  77. Slack, Dylan, Sophie Hilgard, Emily Jia, Sameer Singh, and Himabindu Lakkaraju. 2020. Fooling LIME and SHAP: Adversarial attacks on post hoc explanation methods. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pages 180–186. https://doi.org/10.1145/3375627.3375830 Smilkov, Dan… — [doi:10.1145/3375627.3375830](https://doi.org/10.1145/3375627.3375830) ↗
  78. Springenberg, J., Alexey Dosovitskiy, Thomas Brox, and M. Riedmiller. 2015. In Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806. — [arXiv:1412.6806](https://arxiv.org/abs/1412.6806) find
  79. Strobelt, Hendrik, Sebastian Gehrmann, Michael Behrisch, Adam Perer, Hanspeter Pfister, and Alexander M. Rush. 2019. Seq2seq-Vis: A visual debugging tool for sequence-to-sequence models. IEEE Transactions on Visualization and Computer Graphics, 25(1):353–363. https://doi.org /10.1109/TVCG.2018.28650… ↗
  80. Tsang, Michael, Sirisha Rambhatla, and Yan Liu. 2020. How does this interaction affect me? Interpretable attribution for feature interactions. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, pages 6147–6159. find
  81. Tucker, Mycal, Peng Qian, and Roger Levy. 2021. What if this modified that? Syntactic interventions with counterfactual embeddings. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 862–875. https://doi.org/10.18653/v1/2021.findings -acl.76 Tutek, Martin and Jan S… ↗
  82. Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, pages 5998–6008. find
  83. Veldhoen, Sara, Dieuwke Hupkes, and Willem Zuidema. 2016. Diagnostic classifiers: Revealing how neural networks process hierarchical structure. In CoCo@ NIPS, pages 69–77. find
  84. Barcelona. Vig, Jesse. 2019. Visualizing attention in transformer-based language representation models. ArXiv preprint, abs/1904.02679. — [arXiv:1904.02679](https://arxiv.org/abs/1904.02679) find
  85. Vig, Jesse, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart M. Shieber. 2020. Investigating gender bias in language models using causal mediation analysis. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Proces… find
  86. Voita, Elena, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. 2019. Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5797–5808. https://do… find our card
  87. Voita, Elena and Ivan Titov. 2020. Information-theoretic probing with minimum description length. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 183–196. https://doi.org/10.18653/v1/2020 .emnlp-main.14 Wallace, Eric, Shi Feng, and Jordan Boyd… ↗
  88. Wang, Alex, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019b. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In 7th International Conference on Learning Representations, ICLR 2019. https://doi.org/10.18653/v1/W18-5446 Wang,… ↗ our card
  89. Wei, Jason, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. ArXiv preprint, abs/2201.11903. Wexler, James, Mahima Pushkarna, Tolga Bolukbasi, Martin Wattenberg, Fernanda… find our card
  90. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10266–10284. https://doi.org/10.18653/v1/2021 .emnlp-main.804 Wiegreffe, Sarah and Yuval Pinter. 2019. Attention is not not explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natu… ↗
  91. Yang, Zhilin, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 236… find our card
  92. Ye, Xi and Greg Durrett. 2023. Explanation Selection Using Unlabeled Data for In-Context Learning. Singapore. https://doi.org/10.18653/v1/2023.emnlp -main.41 Ye, Xi, Srinivasan Iyer, Asli Celikyilmaz, Ves Stoyanov, Greg Durrett, and Ramakanth Pasunuru. 2022. Complementary explanations for effective… ↗
  93. Yi, Kexin, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli, and Josh Tenenbaum. 2018. Neural-symbolic VQA: Disentangling reasoning from vision and language understanding. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018,… find
  94. Yin, Fan, Zhouxing Shi, Cho-Jui Hsieh, and Kai-Wei Chang. 2022. On the sensitivity and stability of model interpretations in NLP. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2631–2647. https://doi.org/10 .18653/v1/2022.acl… find
  95. Zaidan, Omar, Jason Eisner, and Christine Piatko. 2007. Using “Annotator Rationales” to Improve Machine Learning for Text Categorization. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Confer… find our card
  96. Zeiler, Matthew D. and Rob Fergus. 2014. Visualizing and understanding convolutional networks. In David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars, editors, Computer Vision – ECCV 2014, volume 8689. Springer International Publishing, Cham, pages 818–833. https://doi.org/10 .1007/978-3-… find
  97. Zheng, Yiming, Serena Booth, Julie Shah, and Yilun Zhou. 2022. The irrationality of neural rationale models. In Proceedings of the 2nd Workshop on Trustworthy Natural Language Processing (TrustNLP 2022), pages 64–73. https://doi.org/10.18653/v1/2022.trustnlp-1.6 Zhou, Denny, Nathanael Schärli, Le… ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lyu-2024-faithful-model-explanation-nlp-survey/