Elamur Research⌕
☰
home / papers / 2021 / CausaLM: Causal Model Explanation Through Counterfactual Language Models

CausaLM: Causal Model Explanation Through Counterfactual Language Models

Paper · 2021 · Amir Feder, Nadav Oved, Uri Shalit et al.

Summary

Explaining what drives a neural text classifier is hard. Standard attribution tools measure correlation, not causation, so they cannot separate the influence of two related high-level concepts, such as a politician's name and the adjectives around it. A proper causal estimate requires counterfactual texts, which current generators cannot reliably produce. CausaLM avoids this by building a counterfactual representation instead of a counterfactual text. An extra copy of BERT is fine-tuned with adversarial auxiliary tasks taken from the problem's causal graph. This copy learns to discard a chosen "treated" concept while still encoding designated control concepts. The causal effect of the concept is then the difference between test-set predictions of classifiers built on the original and on the counterfactual encoder. The authors evaluate this on sentiment and emotion (POMS) tasks, using data versions with controlled degrees of bias. They report that the method recovers concept effects better than correlation-based and representation-removal baselines, such as INLP. A useful side product is an encoder that ignores the concept, which can help reduce unwanted bias such as gender or race effects.

Key points

  • Three-stage pipeline: (1) start from pre-trained BERT-O, (2) adversarially fine-tune it into BERT-CF, with BERT-MLM as a control continuing standard MLM training, and (3) train downstream classifiers and compare their test-set outputs.
  • The authors introduce the Enriched EEC (EEEC) corpus, which extends EEC from 9,840 to 33,738 sentences, 11 to 42 templates, and 40 to 55 emotion words. It adds noise, prefix and suffix sentences.
  • Each data set comes in Balanced, Gentle and Aggressive versions, which vary the strength of the planted correlation between concept and label.
  • The tasks are Sentiment, POMS, TC and CC probing. The causal estimator (TReATE) is compared with an INLP-based baseline that feeds BERT-O's CLS vector into logistic regression.
  • Setup: case-sensitive BERT-base (12 layers, 110M parameters), with all layers fine-tuned in Stage 2. Training used Adam with learning rate 1e-3, and hyperparameters were not tuned. Code and data are available at amirfeder.github.io/CausaLM.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Technion – Israel Institute of TechnologyHaifa · Israel28 works · 4,941 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 1,343 works

Cite

Cite as Amir Feder, Nadav Oved, Uri Shalit and Roi Reichart. 2021. CausaLM: Causal Model Explanation Through Counterfactual Language Models. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli_a_00404

@article{feder2021causalm,
  title = {{CausaLM: Causal Model Explanation Through Counterfactual Language Models}},
  author = {Amir Feder, Nadav Oved, Uri Shalit and Roi Reichart},
  journal = {Computational Linguistics},
  year = {2021},
  doi = {10.1162/coli\_a\_00404},
  url = {https://doi.org/10.1162/coli_a_00404}
}

References

59 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Adi, Yossi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2017. Fine-grained analysis of sentence embeddings using auxiliary prediction tasks. In 5th International Conference on Learning Representations, ICLR 2017, pages 1–13.
  2. Toulon. Aharoni, Roee, Melvin Johnson, and Orhan Firat. 2019. Massively multilingual neural machine translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, pages 3874–3884. Minneap…
  3. Blei, David M., Andrew Y. Ng, and Michael I. Jordan. 2003. Latent dirichlet allocation. Journal of Machine Learning Research, 3:993–1022.
  4. Blitzer, John, Mark Dredze, and Fernando Pereira. 2007. Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, pages 440–447. Prague. Bottou, Léon, Jonas Peters, Joa…
  5. Boyd-Graber, Jordan L., Yuening Hu, and David M. Mimno. 2017. Applications of topic models. Foundations and Trends in Information Retrieval, 11(2–3):143–296. https://doi.org/10.1561/1500000030 — [doi:10.1561/1500000030](https://doi.org/10.1561/1500000030) ↗
  6. Caliskan, Aylin, Joanna J. Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. https://doi.org/10.1126/science.aal4230, PubMed: 28408601 Chang, Ming-Wei, Kristina Toutanova, Kenton Lee, and Jacob Devlin. 20… ↗
  7. Che, Tong, Yanran Li, Ruixiang Zhang, R. Devon Hjelm, Wenjie Li, Yangqiu Song, and Yoshua Bengio. 2017. Maximum-likelihood augmented discrete generative adversarial networks. arXiv preprint arXiv:1702.07983. Conneau, Alexis, Germán Kruszewski, Guillaume Lample, Loı̈c Barrault, and Marco Baroni. 201…
  8. D’Amour, Alexander, Peng Ding, Avi Feller, Lihua Lei, and Jasjeet Sekhon. 2017. Overlap in observational studies with high-dimensional covariates. arXiv preprint arXiv:1711.02582. — [arXiv:1711.02582](https://arxiv.org/abs/1711.02582) find
  9. Davidson, Thomas, Debasmita Bhattacharya, and Ingmar Weber. 2019. Racial bias in hate speech and abusive language detection data sets. In Proceedings of the Third Workshop on Abusive Language Online, pages 25–35. https://doi.org/10.18653/v1/W19-3504 De-Arteaga, Maria, Alexey Romanov, Hanna M. Walla… ↗
  10. Minneapolis, MN. Dong, Li, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. 2019. Unified language model pre-training for natural language understanding and generation. In Advances in Neural Information Processing Systems 32: Annual Conference on N…
  11. Egami, Naoki, Christian J. Fong, Justin Grimmer, Margaret E. Roberts, and Brandon M. Stewart. 2018. How to make causal inferences using texts. arXiv preprint arXiv:abs/1802.02163. — [arXiv:1802.02163](https://arxiv.org/abs/1802.02163) find
  12. Elazar, Yanai and Yoav Goldberg. 2018. Adversarial removal of demographic attributes from text data. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 11–21. Brussels. https://doi.org/10.18653/v1/D18-1002 Falcon, WA. 2019. Pytorch lightning. GitHub. ht… ↗
  13. Feder, Amir, Danny Vainstein, Roni Rosenfeld, Tzvika Hartman, Avinatan Hassidim, and Yossi Matias. 2020. Active deep learning to detect demographic traits in free-form clinical notes. Journal of Biomedical Informatics, vol. 107, 103436. https://doi.org/10.1016/j.jbi.2020 .103436, PubMed: 32428572 Fe… — [doi:10.1016/j.jbi.2020](https://doi.org/10.1016/j.jbi.2020) ↗
  14. Vancouver. Fong, Christian and Justin Grimmer. 2016. Discovery of treatments from text corpora. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1600–1609. Berlin. https://doi.org/10.18653/v1/P16-1151 Ganin, Yaroslav, Evgeniya Ustinova, Hana Ajakan,… ↗
  15. Gao, Jianfeng, Michel Galley, and Lihong Li. 2018. Neural approaches to conversational AI. In Proceedings of ACL 2018, pages 2–7. Melbourne. https://doi.org/10.18653/v1/P18-5002 Gardner, Matt, Yoav Artzi, Victoria Basmova, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai El… ↗
  16. Vancouver. Ghosal, Deepanway, Devamanyu Hazarika, Abhinaba Roy, Navonil Majumder, Rada Mihalcea, and Soujanya Poria. 2020. Kingdom: Knowledge-guided domain adaptation for sentiment analysis. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, pages 319… find
  17. Minneapolis, MN. Goyal, Yash, Amir Feder, Uri Shalit, and Been Kim. 2019a. Explaining classifiers with causal concept effect (cace). arXiv preprint arXiv:abs/1907.07165. — [arXiv:1907.07165](https://arxiv.org/abs/1907.07165) find
  18. Goyal, Yash, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019b. Counterfactual visual explanations. In Proceedings of the 36th International Conference on Machine Learning, pages 2376–2384. Long Beach, California. find
  19. Guo, Jiaxian, Sidi Lu, Han Cai, Weinan Zhang, Yong Yu, and Jun Wang. 2018. Long text generation via adversarial training with leaked information. In Thirty-Second AAAI Conference on Artificial Intelligence, pages 5141–5148. find
  20. New Orleans, LA. Gururangan, Suchin, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, p… find
  21. Isabelle, Pierre, Colin Cherry, and George F. Foster. 2017. A challenge set approach to evaluating machine translation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2486–2496. Copenhagen. https://doi.org/10.18653/v1/D17-1263 ↗
  22. Jain, Sarthak and Byron C. Wallace. 2019. Attention is not explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3543–3556.
  23. Minneapolis, MN. Jernite, Yacine, Samuel R. Bowman, and David A. Sontag. 2017. Discourse-based objectives for fast unsupervised sentence representation learning. arXiv preprint arXiv:abs/1705.00557. — [arXiv:1705.00557](https://arxiv.org/abs/1705.00557) find
  24. Johansson, Fredrik D., Uri Shalit, and David A. Sontag. 2016. Learning representations for counterfactual inference. In Proceedings of the 33nd International Conference on Machine Learning, pages 3020–3029. find
  25. New York, NY. Kaushik, Divyansh, Eduard H. Hovy, and Zachary Chase Lipton. 2020. Learning the difference that makes a difference with counterfactually-augmented data. In 8th International Conference on Learning Representations, pages 1–17. find
  26. Addis Ababa. Keith, Katherine A., David Jensen, and Brendan O’Connor. 2020. Text and causal inference: A review of using text to remove confounding from causal estimates. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, pages 5332–5344. https://doi… find
  27. Barcelona. Kim, Been, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. 2016b. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International Conference on Machine Learning, pages 2668–2677. find
  28. Barcelona. Kingma, Diederik P. and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, pages 1–15. find
  29. San Diego, CA. Kiritchenko, Svetlana and Saif Mohammad. 2018. Examining gender and race bias in two hundred sentiment analysis systems. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pages 43–53. New Orleans, LA. https://doi.org/10.18653/v1/S18-2005 Lau, Jey… ↗
  30. Linzen, Tal, Grzegorz Chrupała, Yonatan Belinkov, and Dieuwke Hupkes, editors. 2019. Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP. find
  31. Florence. Lipton, Zachary C. 2018. The mythos of model interpretability. Communications of ACM, 61(10):36–43. https://doi.org/10.1145/3233231 Liu, Yinhan, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A rob… ↗
  32. Logeswaran, Lajanugen and Honglak Lee. 2018. An efficient framework for learning sentence representations. In 6th International Conference on Learning Representations, pages 1–15.
  33. Vancouver. Lorberbom, Guy, Tommi S. Jaakkola, Andreea Gane, and Tamir Hazan. 2019. Direct optimization through arg max for discrete variational auto-encoder. In Advances in Neural Information Processing Systems, pages 6200–6211. find
  34. Vancouver. Lundberg, Scott M. and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, pages 4765–4774.
  35. Long Beach, CA. Maas, Andrew L., Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142–150. OR. Naik, Aaka…
  36. Florence. Oved, Nadav, Amir Feder, and Roi Reichart. 2020. Predicting in-game actions from interviews of NBA players. Computational Linguistics, 46(3):667–712. find
  37. Pang, Bo, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up? Sentiment classification using machine learning techniques. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing, pages 79–86.
  38. Philadelphia, PA. Paszke, Adam, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in PyTorch. In NIPS-W, pages 1–4. find
  39. Long Beach, CA. Pearl, Judea. 1995. Causal diagrams for empirical research. Biometrika, 82(4):669–688. https://doi.org/10 .1093/biomet/82.4.669 Pearl, Judea. 2009a. Causal inference in statistics: An overview. Statistics Surveys, 3:96–146. https://doi.org/10.1214/09-SS057 Pearl, Judea. 2009b. find
  40. Causality. Cambridge University Press. Pennebaker, James W., Martha E. Francis, and Roger J. Booth. 2001. Linguistic inquiry and word count: LIWC 2001. http://downloads.liwc.net.s3.amazonaws.com/LIWC2015 OperatorManual.pdf. ↗
  41. Peters, Matthew E., Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techn…
  42. Radford, Alec, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, 1(8):9. find
  43. Ramponi, Alan and Barbara Plank. 2020. Neural unsupervised domain adaptation in NLP - A survey. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6838–6855. Barcelona. https://doi.org/10.18653/v1/2020 .coling-main.603 Ravfogel, Shauli, Yanai Elazar, Hila Gonen,… ↗
  44. Roberts, Margaret E., Brandon M. Stewart, Dustin Tingley, Christopher Lucas, Jetson Leder-Luis, Shana Kushner Gadarian, Bethany Albertson, and David G. Rand. 2014. Structural topic models for open-ended survey responses. American Journal of Political Science, 58(4):1064–1082. https://doi.org/10 .111… find
  45. Rudinger, Rachel, Chandler May, and Benjamin Van Durme. 2017. Social bias in elicited natural language inferences. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, EthNLP@EACL, Valencia, Spain, April 4, 2017, pages 74–79. Valencia. https://doi.org/10.18653/v1/w17-1… ↗
  46. Saha, Koustuv, Benjamin Sugar, John Torous, Bruno D. Abrahao, Emre Kiciman, and Munmun De Choudhury. 2019. A social media study on the effects of psychiatric medication use. In Proceedings of the International AAAI Conference on Web and Social Media, pages 440–451. find
  47. Munich. Sari, Yunita, Mark Stevenson, and Andreas Vlachos. 2018. Topic or style? Exploring the most useful features for authorship attribution. In Proceedings of the 27th International Conference on Computational Linguistics, pages 343–353. find
  48. Santa Fe, New Mexico. Semeniuta, Stanislau, Aliaksei Severyn, and Erhardt Barth. 2017. A hybrid convolutional variational autoencoder for text generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 627–637. Copenhagen. https://doi.org/10.18653 /v… find
  49. Sennrich, Rico. 2017. How grammatical is character-level neural machine translation? Assessing MT quality with contrastive translation pairs. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, pages 376–382. Valencia. https://doi.org/10.18…
  50. Subramanian, Sandeep, Sai Rajeswar, Francis Dutil, Chris Pal, and Aaron C. Courville. 2017. Adversarial generation of natural language. In Proceedings of the 2nd Workshop on Representation Learning for NLP, 241–251. Vancouver. https://doi.org/10.18653/v1/W17-2629 Sun, Chi, Xipeng Qiu, Yige Xu, and… ↗
  51. Long Beach, CA. Veitch, Victor, Dhanya Sridhar, and David M. Blei. 2019. Using text embeddings for causal inference. arXiv preprint arXiv:abs/1905.12741. — [arXiv:1905.12741](https://arxiv.org/abs/1905.12741) find
  52. Vig, Jesse, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart M. Shieber. 2020. Investigating gender bias in language models using causal mediation analysis. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Proces… find
  53. Wiegreffe, Sarah and Yuval Pinter. 2019. Attention is not not explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 11–20, Hong Kong. https://doi .org/10.18653/v1/D19-100…
  54. Wood-Doughty, Zach, Ilya Shpitser, and Mark Dredze. 2018. Challenges of using text classifiers for causal inference. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4586–4598. Brussels. https://www.aclweb.org/anthology/D18-1488/. ↗
  55. Woodward, James. 2005. Making Things Happen: A Theory Of Causal Explanation. Oxford University Press. Wu, Yonghui, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu,… find
  56. Xie, Qizhe, Zihang Dai, Yulun Du, Eduard H. Hovy, and Graham Neubig. 2017. Controllable invariance through adversarial feature learning. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems, pages 585–596. find
  57. Long Beach, CA. Yang, Zhilin, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2018. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. In Advances in Neural Information Processing Systems… find
  58. Vancouver. Zech, John R., Marcus A. Badgeley, Manway Liu, Anthony B. Costa, Joseph J. Titano, and Eric Karl Oermann. 2018. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. find
  59. PLoS Medicine, 15(11):e1002683. https://doi.org/10.1371/journal.pmed.1002683, PubMed: 30399157 Zhu, Yukun, Ryan Kiros, Richard S. Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Aligning books and movies: Towards story-like visual explanations by watching movi… ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/feder-2021-causalm-causal-model-explanation-through/