Elamur Research⌕
☰
home / papers / 2018 / Comparing Automatic and Human Evaluation of Local Explanations for Text Classification

Comparing Automatic and Human Evaluation of Local Explanations for Text Classification

Paper · 2018 · Dong Nguyen

Summary

Explanations that point to the words driving a single prediction are a popular way to make text classifiers more transparent, but it is unclear how to judge their quality. Nguyen tests whether cheap automatic metrics agree with human judgement. Four word-level explanation methods (a random baseline, LIME, word omission, and first-derivative saliency) explain the outputs of a Logistic Regression model and an MLP. The models are trained on two binary tasks: 20news (Christianity vs. Atheism) and movie review sentiment.

Fidelity is scored automatically by deleting the highlighted words, using the switching point and AOPC. Human quality is measured in a crowdsourced forward-prediction task, where workers see an explanation and guess what the classifier predicted.

The automatic scores correlate only moderately with how often workers guess correctly. Classifier confidence and the type of prediction outcome (true or false positive or negative) also shape human accuracy. This matters because deletion-based metrics are useful but cannot stand in for user studies.

Key points

  • Data:
    • Movie reviews come from Pang and Lee (2004), using the Zaidan et al. (2007) version, split 60/20/20 into train, development and test.
    • 20news uses the 20news-bydate version, restricted to two classes.
  • Classifiers:
    • Logistic Regression (Scikit-learn, TF-IDF unigrams) reaches 0.797 accuracy on movies and 0.921 on 20news.
    • The Keras MLP (512 hidden units) reaches 0.832 and 0.939.
  • Omission as reference: for LR, word omission matches the known feature contributions, so it provides a ground-truth ranking.
  • Regression analysis: an OLS model on the movie data (n = 1,620, R² = 0.177) shows that a higher switching point significantly lowers crowd accuracy. Classifier confidence and prediction outcome also have significant effects.
  • Noise experiment: randomly replacing 0–80% of the top-10 saliency words lowers mean AOPC from 0.2627 to 0.0411. Crowd accuracy on true positives falls from 0.940 to 0.736.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of EdinburghEdinburgh · United Kingdom255 works · 68,771 citations🔬The Alan Turing InstituteLondon · United Kingdom19 works · 3,608 citations

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksSoftware · 25 works

🧭 Topics 1Research subjects and languages the work is about.Evaluation & benchmarks · 685 works

Cite

Cite as Dong Nguyen. 2018. Comparing Automatic and Human Evaluation of Local Explanations for Text Classification. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n18-1097

@inproceedings{nguyen2018comparing,
  title = {{Comparing Automatic and Human Evaluation of Local Explanations for Text Classification}},
  author = {Dong Nguyen},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-1097},
  url = {https://doi.org/10.18653/v1/n18-1097}
}

References

30 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Leila Arras, Franziska Horn, Grégoire Montavon, Klaus-Robert Muller, and Wojciech Samek. 2016. Explaining predictions of non-linear classifiers in NLP. In Proceedings of the 1st Workshop on Representation Learning for NLP. pages 1–7. find
  2. Leila Arras, Franziska Horn, Grégoire Montavon, Klaus-Robert Müller, and Wojciech Samek. 2017. ”What is relevant in a text document?”: An interpretable machine learning approach. PLOS ONE 12(8):e0181142. find
  3. Malika Aubakirova and Mohit Bansal. 2016. Interpreting neural networks to improve politeness comprehension. In Proceedings of EMNLP 2016. pages 2035–2041.
  4. Chollet et al. 2015. Keras. https://github.com/fchollet/keras. ↗
  5. Mark W. Craven. 1996. Extracting comprehensible models from trained neural networks. Ph.D. thesis, University of Wisconsin–Madison. find
  6. Yanzhuo Ding, Yang Liu, Huanbo Luan, and Maosong Sun. 2017. Visualizing and understanding neural machine translation. In Proceedings of ACL 2017. pages 1150–1159. find
  7. Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. In arXiv preprint arXiv:1702.08608.
  8. Alex A. Freitas. 2014. Comprehensible classification models: A position paper. SIGKDD Explorations Newsletter 15(1):1–10. find
  9. Yash Goyal, Akrit Mohapatra, Devi Parikh, and Dhruv Batra. 2016. Towards transparent AI systems: Interpreting visual question answering models. In International Conference on Machine Learning (ICML) Workshop on Visualization for Deep Learning. find
  10. Abram Handler, Su Lin Blodgett, and Brendan O’Connor. 2016. Visualizing textual models with in-text and word-as-pixel highlighting. In Proceedings of the 2016 Workshop on Human Interpretability in Machine Learning. find
  11. Jonathan L. Herlocker, Joseph A. Konstan, and John Riedl. 2000. Explaining collaborative filtering recommendations. In Proceedings of CSCW ’00. pages 241–250. find
  12. Ákos Kádár, Grzegorz Chrupala, and Afra Alishahi. 2016. Representation of linguistic form and function in recurrent neural networks. CoRR abs/1602.08952.
  13. Ashish Kapoor, Bongshin Lee, Desney Tan, and Eric Horvitz. 2010. Interactive optimization for steering machine classification. In Proceedings of CHI ’10. pages 1343–1352. find
  14. Pang Wei Koh and Percy Liang. 2017. Understanding black-box predictions via influence functions. In Proceedings of ICML 2017. pages 1885–1894. a deep neural network has learned. IEEE Transactions on Neural Networks and Learning Systems 28(11):2660 – 2673.
  15. Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015. Principles of explanatory debugging to personalize interactive machine learning. In Proceedings of IUI ’15. pages 126–137. find
  16. Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. CoRR abs/1312.6034.
  17. Jiwei Li, Xinlei Chen, Eduard Hovy, and Dan Jurafsky. 2016a. Visualizing and understanding neural models in NLP. In Proceedings of NAACL 2016. pages 681–691.
  18. Omar Zaidan, Jason Eisner, and Christine Piatko. 2007. Using “annotator rationales” to improve machine learning for text categorization. In Proceedings of NAACL 2007. pages 260–267.
  19. Jiwei Li, Will Monroe, and Dan Jurafsky. 2016b. Understanding neural networks through representation erasure. arXiv preprint arXiv:1612.08220 . find
  20. Matthew D. Zeiler and Rob Fergus. 2014. Visualizing and understanding convolutional networks. In ECCV 2014. pages 818–833.
  21. Brian Y. Lim, Anind K. Dey, and Daniel Avrahami. 2009. Why and why not explanations improve the intelligibility of context-aware intelligent systems. In Proceedings of CHI ’09. pages 2119–2128. find
  22. A Zachary C. Lipton. 2016. The mythos of model interpretability. In Proceedings of the 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016). pages 96–100. find
  23. Christopher D. Manning. 2015. Computational linguistics and deep learning. Computational Linguistics 41(4):701–707. find
  24. David Martens and Foster Provost. 2014. Explaining data-driven document classifications. MIS Quarterly 38(1):73–100. find
  25. Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity. In Proceedings of ACL. pages 271–278. find
  26. Paul. 2016. Interpretable machine learning: Lessons from topic modeling. In Proceedings of the CHI Workshop on Human-Centered Machine Learning. find
  27. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesnay. 2011. Scikit-learn… find
  28. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. “Why should I trust you?”: Explaining the predictions of any classifier. In Proceedings of KDD ’16. pages 1135–1144.
  29. Marko Robnik-Šikonja and Igor Kononenko. 2008. Explaining classifications for individual instances. IEEE Transactions on Knowledge and Data Engineering 20(5):589–600. find
  30. Wojciech Samek, Alexander Binder, Gregoire Montavon, Sebastian Lapuschkin, and Klaus-Robert Müller. 2017. Evaluating the visualization of what find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nguyen-2018-comparing-automatic-human-evaluation-local/