Elamur Research⌕
☰
home / papers / 2026 / Trustworthy and Explainable Causal Representation Learning in Transformers

Trustworthy and Explainable Causal Representation Learning in Transformers

Paper · 2026 · Yang Liu, Yinghao Zhang, Lin Liu et al.

Summary

Explanations for Transformers are often built by learning an importance mask over input features and applying it to the input representation. Raw attention weights are another common source of explanations. The authors argue that both approaches lack a guarantee of identifiability, meaning there is no assurance that the explanation is uniquely determined by the data distribution. Attention-based explanations also face doubts about faithfulness. They propose CIEM (Causal Identifiable Explanation Model), a pluggable module attached to a frozen Transformer and its frozen head. CIEM uses a structural causal model, nonlinear ICA and an identifiable VAE (iVAE). Instead of learning a mask, it learns latent attention-weight (Z_w) and value (Z_v) representations from a chosen layer's output. This turns the frozen model into a self-explaining system that can target specific layers. Experiments use BERT-base-uncased with baselines such as LIME, IG and RawAttention. CIEM agrees better with human-annotated rationales and stays competitive on faithfulness. The work matters because it gives explanations a theoretically grounded origin rather than an arbitrary one.

Key points

  • Trustworthiness is measured as token-level agreement with e-SNLI human rationales using Accuracy, Precision, Recall and F1 on the Top-40% features. CIEM beats all baselines on every metric across three manually labeled datasets.
  • Faithfulness is tested with COMP and SUFF on four real-world datasets. CIEM outperforms most methods on the majority of metrics, which the authors frame as a deliberate trade-off rather than winning everywhere.
  • The composite representation Z = Z_w(XZ_v), passed through the frozen head, gives accuracy very close to the original model's. Synthetic-data experiments support the identifiability claim.
  • Identifiability depends on three assumptions in Theorem 4.1. The auxiliary generative model adds training cost, and the experiments cover only BERT-base-uncased.
  • Code is available at github.com/LORD-ROY/CIEM. The paper appears in Findings of ACL 2026.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓The University of AdelaideAdelaide · Australia6 works · 1,417 citations🎓Huazhong Agricultural UniversityWuhan · China3 works · 22 citations🎓Hainan UniversityHaikou · China3 works · 2 citations🎓Adelaide UniversityAdelaide · Australia5 works · 3 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

Cite

Cite as Yang Liu, Yinghao Zhang, Lin Liu, Jiuyong Li, Debo Cheng and Zaiwen Feng. 2026. Trustworthy and Explainable Causal Representation Learning in Transformers. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1368

@inproceedings{liu2026trustworthy,
  title = {{Trustworthy and Explainable Causal Representation Learning in Transformers}},
  author = {Yang Liu, Yinghao Zhang, Lin Liu, Jiuyong Li, Debo Cheng and Zaiwen Feng},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1368},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1368}
}

References

55 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. 2015. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10(7):e0130140. find
  2. Bing Bai, Jian Liang, Guanhua Zhang, Hao Li, Kun Bai, and Fei Wang. 2021. Why attentions may not be interpretable? In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pages 25–34. find
  3. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal. Association for Computati…
  4. Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-snli: Natural language inference with natural language explanations. Advances in Neural Information Processing Systems, 31.
  5. Gianluca Carloni, Andrea Berti, and Sara Colantonio. 2025. The role of causality in explainable artificial intelligence. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 15(2):e70015. find
  6. Chun Sik Chan, Huanqi Kong, and Liang Guanqing. 2022. A comparative study of faithfulness metrics for model interpretability methods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5029–5038, Dublin, Ireland. Association for… find
  7. George Chrysostomou and Nikolaos Aletras. 2021. Improving the faithfulness of attention-based explanations with task-specific information for text classification. arXiv preprint arXiv:2105.02657. find
  8. Jacob Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
  9. Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. 2020. ERASER: A benchmark to evaluate rationalized NLP models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4443–4458, Online. Asso…
  10. Mengnan Du, Ninghao Liu, Fan Yang, Shuiwang Ji, and Xia Hu. 2019. On attribution of recurrent neural network predictions via additive decomposition. In The world wide web conference, pages 383–393. find
  11. Yaru Hao, Li Dong, Furu Wei, and Ke Xu. 2021. Self-attention attribution: Interpreting information interactions inside transformer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 12963–12971. find
  12. Aapo Hyvarinen, Hiroaki Sasaki, and Richard Turner. 2019. Nonlinear ica using auxiliary variables and generalized contrastive learning. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 859–868. PMLR. find
  13. Alon Jacovi and Yoav Goldberg. 2020. Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4198–4205, Online. Association for Computational Linguistics.
  14. Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. 2020a. Variational autoencoders and nonlinear ica: A unifying framework. In International Conference on Artificial Intelligence and Statistics, pages 2207–2217. PMLR. find
  15. Ilyes Khemakhem, Diederik P. Kingma, Ricardo Pio Monti, and Aapo Hyvärinen. 2020b. Variational autoencoders and nonlinear ica: A unifying framework. In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS), volume 108 of Proceedings of Machine Learning… find
  16. Diederik P Kingma. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114.
  17. Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016. Rationalizing neural predictions. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 107–117, Austin, Texas. Association for Computational Linguistics.
  18. Arthur Lewbel. 2019. The identification zoo: Meanings of identification in econometrics. Journal of Economic Literature, 57:835–903. find
  19. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. find
  20. Wei Liu, Zhiying Deng, Zhongyu Niu, Jun Wang, Haozhao Wang, and Ruixuan Li. 2025. Exploring practical gaps in using cross entropy to implement maximum mutual information criterion for rationalization. Transactions of the Association for Computational Linguistics, 13:577–594. find
  21. Wei Liu, Haozhao Wang, Jun Wang, Ruixuan Li, Chao Yue, and YuanKai Zhang. 2022. Fr: Folded rationalization with a unified encoder. Advances in Neural Information Processing Systems, 35:6954–6966. find
  22. Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30.
  23. Siwen Luo, Hamish Ivison, Soyeon Caren Han, and Josiah Poon. 2024. Local interpretations for explainable natural language processing: A survey. ACM Computing Surveys, 56(9):1–36. find
  24. Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142–150, Portland, Oregon…
  25. Qianren Mao, Jianxin Li, Senzhang Wang, Yuanning Zhang, Hao Peng, Min He, and Lihong Wang. 2019. Aspect-based sentiment classification with attentive neural turing machines. In IJCAI, pages 5139–5145. find
  26. Bjørn Leth Møller, Christian Igel, Kristoffer Knutsen Wickstrøm, Jon Sporring, Robert Jenssen, and Bulat Ibragimov. 2024. Finding NEM-u: Explaining unsupervised representation learning through neural network generated explanation masks. In Proceedings of the 41st International Conference on Machine… find
  27. Zhenyu Nie, Zheng Xiao, Tao Wang, Anthony Theodore Chronopoulos, Răzvan Andonie, and Amir Mosavi. 2025. Tips: A text interaction evaluation metric for learning model interpretation. Expert Systems with Applications, 287:128184. find
  28. Judea Pearl. 2009. Causality: Models, Reasoning and Inference, 2nd edition. Cambridge University Press, USA. find
  29. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "why should i trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, page 1135–1144, New York, NY, USA. Association for Co…
  30. Raanan Y Rohekar, Yaniv Gurwicz, and Shami Nisi-mov. 2024. Causal interpretation of self-attention in pre-trained transformers. Advances in Neural Information Processing Systems, 36. find
  31. Sofia Serrano and Noah A Smith. 2019. Is attention interpretable? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2931–2951.
  32. Ying Shen, Yang Deng, Min Yang, Yaliang Li, Nan Du, Wei Fan, and Kai Lei. 2018. Knowledge-aware attentive neural network for ranking question answer pairs. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR ’18, page 901–904, New York, NY, USA. A… find
  33. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631–…
  34. Kihyuk Sohn, Honglak Lee, and Xinchen Yan. 2015. Learning structured output representation using deep conditional generative models. Advances in neural information processing systems, 28. find
  35. Peter Sorrenson, Carsten Rother, and Ullrich Köthe. 2020. Disentanglement by nonlinear ica with general incompressible-flow networks (gin). arXiv preprint arXiv:2001.04872. — [arXiv:2001.04872](https://arxiv.org/abs/2001.04872) find
  36. Ying Sun, Hengshu Zhu, and Hui Xiong. 2025. Toward faithful neural network intrinsic interpretation with shapley additive self-attribution. IEEE Transactions on Neural Networks and Learning Systems, 36(9):16294–16308. find
  37. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In International conference on machine learning, pages 3319–3328. PMLR. find
  38. Khanh Hiep Tran, Azin Ghazimatin, and Rishiraj Saha Roy. 2021. Counterfactual explanations for neural recommenders. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’21, page 1627–1631, New York, NY, USA. Association for Comput… find
  39. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
  40. Jie Wang, Haidong Shao, Jing He, Le Liu, Jingqiang Ma, and Bin Liu. 2025. A novel interpretable fault diagnosis method using multi-image feature extraction and attention fusion. Pattern Recognition Letters, 189:38–47. find
  41. Jingjing Wang, Jie Li, Shoushan Li, Yangyang Kang, Min Zhang, Luo Si, and Guodong Zhou. 2018. Aspect sentiment classification with both word-level and clause-level attention networks. In IJCAI, volume 2018, pages 4439–4445. find
  42. Matthew D Zeiler and Rob Fergus. 2014. Visualizing and understanding convolutional networks. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13, pages 818–833. Springer.
  43. Wenbo Zhang, Tong Wu, Yunlong Wang, Yong Cai, and Hengrui Cai. 2023. Towards trustworthy explanation: On causal rationalization. In International Conference on Machine Learning, pages 41715– 41736. PMLR. find
  44. Chenwang Wu, Xiting Wang, Defu Lian, Xing Xie, and Enhong Chen. 2023. A causality inspired framework for model interpretation. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2731–2741. find
  45. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28.
  46. Pengzhou Wu and Kenji Fukumizu. 2021. β-intact-vae: Identifying and estimating causal effects under limited overlap. arXiv preprint arXiv:2110.05225. — [arXiv:2110.05225](https://arxiv.org/abs/2110.05225) find
  47. Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2024. Explainability for large language models: A survey. ACM Transactions on Intelligent Systems and Technology, 15(2):1–38.
  48. Biao Xu and Guanci Yang. 2025. Interpretability research of deep learning: A literature survey. Information Fusion, 115:102721. find
  49. Ziqi Xu, Debo Cheng, Jiuyong Li, Jixue Liu, Lin Liu, and Kui Yu. 2024. Causal inference with conditional front-door adjustment and identifiable variational autoencoder. In The Twelfth International Conference on Learning Representations, pages 1 – 22. find
  50. Fan Zhou, Yuzhou Mao, Liu Yu, Yi Yang, and Ting Zhong. 2023. Causal-debias: Unifying debiasing in pretrained language models and fine-tuning via causal invariant learning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4227–4… find
  51. Shuting Yang, Zehui Liu, Wolfgang Mayer, Ningpei Ding, Ying Wang, Yu Huang, Pengfei Wu, Wanli Li, Lin Li, Hong-Yu Zhang, and Zaiwen Feng. 2025. Shizishangpt: An agricultural large language model integrating tools and resources. In Web Information Systems Engineering – WISE 2024, pages 284–298, Singa… find
  52. Mo Yu, Shiyu Chang, Yang Zhang, and Tommi Jaakkola. 2019. Rethinking cooperative rationalization: Introspective extraction and complement control. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language…
  53. Mo Yu, Yang Zhang, Shiyu Chang, and Tommi Jaakkola. 2021. Understanding interlocking dynamics of cooperative rationalization. Advances in Neural Information Processing Systems, 34:12822–12835. find
  54. Libing Yuan, Shuaibo Hu, Kui Yu, and Le Wu. 2025. Boosting explainability through selective rationalization in pre-trained language models. arXiv preprint arXiv:2501.03182. — [arXiv:2501.03182](https://arxiv.org/abs/2501.03182) find
  55. Vladimir Zaigrajew, Hubert Baniecki, and Przemyslaw Biecek. 2025. Interpreting clip with hierarchical sparse autoencoders. arXiv preprint arXiv:2502.20578. — [arXiv:2502.20578](https://arxiv.org/abs/2502.20578) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2026-trustworthy-explainable-causal-representation-learning/