Elamur Research⌕
☰
home / papers / 2021 / Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges

Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges

Paper · 2021 · Cynthia Rudin; Chaofan Chen; Zhi Chen; Haiyang Huang; Lesia Semenova; and Chudi Zhong

Summary

This survey by Rudin and co-authors (Duke University, University of Maine) argues that high-stakes settings such as medicine and criminal justice should use models that are interpretable by design, not black boxes paired with after-the-fact explanations. The authors show how opaque models cause harm. They can conceal input errors, learn spurious shortcuts (the "Clever Hans" effect), and resist troubleshooting when the data distribution shifts. They also make fairness audits harder. Explanations added on top of such models may be misleading.

The paper defines interpretable learning as loss minimization combined with domain-specific interpretability penalties and/or hard constraints. It distinguishes fully from partially interpretable models and untangles the confused terminology around interpretability versus explainability. Its main contribution is a structured map of ten open technical challenges, each with historical background, plus a rule-of-thumb guide matching data types to model families.

The paper offers no new dataset or benchmark. Its value lies in serving as a principled starting point for statisticians and computer scientists entering the field.

Key points

  • The ten challenges are:
    • sparse logical models (decision trees, rule lists)
    • scoring systems
    • sparse, constrained GAMs
    • case-based reasoning, including matching for causal inference
    • supervised disentanglement of neural networks
    • unsupervised disentanglement of neural networks
    • dimensionality reduction for visualization
    • physics- and causally-constrained models
    • characterizing the "Rashomon set" of good models
    • interpretable reinforcement learning
  • Interpretability is formalized as an optimization problem with a loss term, a tunable penalty weight C, and optional hard constraints. Examples of such constraints include sparsity, monotonicity, and physical laws.
  • Table 1 offers rough guidance on model choice:
    • Trees and scoring systems suit fairly clean tabular data.
    • GAMs suit continuous data with low-order interactions.
    • Disentangled networks suit raw inputs such as images, text, and time series.
  • The paper cites the Almost Matching Exactly ("learning to match") framework as a route to interpretable causal inference.
  • It repeats Rudin's (2019) proposal that black boxes should not be deployed for high-stakes decisions unless no interpretable model of similar accuracy exists. It also notes that scholars dispute whether the GDPR actually grants a "Right to Explanation."

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Machine learning · 51 works

Cite

Cite as Cynthia Rudin; Chaofan Chen; Zhi Chen; Haiyang Huang; Lesia Semenova; and Chudi Zhong. 2021. Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges. arXiv:2103.11251 · Statistics Surveys, 2021.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2103.11251 · arXiv 2103.11251

@misc{semenova2021interpretable,
  title = {{Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges}},
  author = {Cynthia Rudin; Chaofan Chen; Zhi Chen; Haiyang Huang; Lesia Semenova; and Chudi Zhong},
  year = {2021},
  doi = {10.48550/arxiv.2103.11251},
  eprint = {2103.11251},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2103.11251}
}

References

336 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Agnar Aamodt and Enric Plaza. Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches. AI Communications, 7(1):39–59, 1994. find
  2. Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. In Proceedings of Neural Information Processing Systems (NeurIPS), 2018. find
  3. Tameem Adel, Zoubin Ghahramani, and Adrian Weller. Discovering interpretable representations for both deep generative and discriminative models. In Proceedings of International Conference on Machine Learning (ICML), pages 50–59, 2018. find
  4. Michael Anis Mihdi Afnan, Cynthia Rudin, Vince Conitzer, Julian Savulescu, Abhishek Mishra, Yanhe Liu, and Masoud Afnan. Ethical implementation of artificial intelligence to select embryos in In Vitro Fertilization. In Proceedings of the Fourth AAAI/ACM Conference on Artificial Intelligence, Ethics,… find
  5. Rishabh Agarwal, Nicholas Frosst, Xuezhou Zhang, Rich Caruana, and Geoffrey E Hinton. Neural additive models: Interpretable machine learning with neural nets. arXiv preprint arXiv:2004.13912, 2020. — [arXiv:2004.13912](https://arxiv.org/abs/2004.13912) find
  6. Sina Aghaei, Andres Gomez, and Phebe Vayanos. Learning optimal classification trees: Strong max-flow formulations. arXiv e-print arXiv:2002.09142, 2020. — [arXiv:2002.09142](https://arxiv.org/abs/2002.09142) find
  7. D. Agrawal, P. Bernstein, E. Bertino, S. Davidson, U. Dayal, M. Franklin, and J. Widom. Challenges and opportunities with big data: A white paper prepared for the computing community consortium committee of the computing research association. Technical report, CRA, 2012. URL http://cra.org/ccc/resou… ↗
  8. Ulrich Aı̈vodji, Julien Ferry, Sébastien Gambs, Marie-José Huguet, and Mohamed Siala. Learning fair rule lists. arXiv e-print arXiv:1909.03977, 2019. — [arXiv:1909.03977](https://arxiv.org/abs/1909.03977) find
  9. E. Amid and M. K. Warmuth. TriMAP: Large-scale Dimensionality Reduction Using Triplets. arXiv e-print arXiv:1910.00204, 2019. — [arXiv:1910.00204](https://arxiv.org/abs/1910.00204) find
  10. E. Angelino, N. Larus-Stone, D. Alabi, M. Seltzer, and C. Rudin. Learning certifiably optimal rule lists for categorical data. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2017. find
  11. Elaine Angelino, Nicholas Larus-Stone, Daniel Alabi, Margo Seltzer, and Cynthia Rudin. Certifiably optimal rule lists for categorical data. Journal of Machine Learning Research, 18:1–78, 2018. find
  12. Plamen Angelov and Eduardo Soares. Towards explainable deep neural networks (xDNN). Neural Networks, 130:185–194, 2020. find
  13. Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. Machine bias. Available from: https://www.propublica.org/article/ machine-bias-risk-assessments-in-criminal-sentencing, May 2016. ↗
  14. Elliott M Antman, Marc Cohen, Peter JLM Bernink, Carolyn H McCabe, Thomas Horacek, Gary Papuchis, Branco Mautner, Ramon Corbalan, David Radley, and Eugene Braunwald. The TIMI risk score for unstable angina/non–ST elevation MI: a method for prognostication and therapeutic decision making. Journal of… find
  15. Javier Antoràn, Umang Bhatt, Tameem Adel, Adrian Weller, and José Miguel Hernández-Lobato. Getting a clue: A method for explaining uncertainty estimates, 2020. find
  16. Cedric Archambeau, Dan Cornford, Manfred Opper, and John Shawe-Taylor. Gaussian process approximations of stochastic differential equations. In Gaussian Processes in Practice, pages 1–16. PMLR, 2007. find
  17. Maryam Ashoori and Justin D. Weisz. In AI we trust? factors that influence trustworthiness of AI-infused decision-making processes. arXiv e-print arXiv:1912.02675, 2019. — [arXiv:1912.02675](https://arxiv.org/abs/1912.02675) find
  18. Akanksha Atrey, Kaleigh Clary, and David Jensen. Exploratory not explanatory: Counterfactual analysis of saliency maps for deep reinforcement learning. In International Conference on Learning Representations (ICLR), 2020. find
  19. Marcus A. Badgeley, John R. Zech, Luke Oakden-Rayner, Benjamin S. Glicksberg, Manway Liu, William Gale, Michael V. McConnell, Bethany Percha, Thomas M. Snyder, and Joel T. Dudley. Deep learning predicts hip fracture using confounding patient and healthcare variables. npj Digital Medicine, 2(31), 201… find
  20. Josh Barnes and Piet Hut. A hierarchical O(N log N ) force-calculation algorithm. Nature, 324 (6096):446–449, 1986. find
  21. Alina Jade Barnett, Fides Regina Schwartz, Chaofan Tao, Chaofan Chen, Yinhao Ren, Joseph Y. Lo, and Cynthia Rudin. IAIA-BL: A case-based interpretable deep learning model for classification of mass lesions in digital mammography. CoRR arXiv preprint 2103.12308, abs/2103.12308, 2021. URL https://arxi… find
  22. Osbert Bastani, Yewen Pu, and Armando Solar-Lezama. Verifiable reinforcement learning via policy extraction. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pages 2499–2509, 2018. find
  23. Bruce Guenther Baumgart. Geometric modeling for computer vision. Technical report, Stanford University Department of Computer Science, 1974. find
  24. Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps and spectral techniques for embedding and clustering. In Proceedings of Neural Information Processing Systems (NeurIPS), volume 14, pages 585–591. MIT Press, 2001. find
  25. Anna C. Belkina, Christopher O. Ciccolella, Rina Anno, Richard Halpert, Josef Spidlen, and Jennifer E. Snyder-Cappione. Automated optimized parameters for t-distributed stochastic neighbor embedding improve visualization and analysis of large datasets. Nature Communications, 10(5415), 2019. find
  26. Yoshua Bengio. Learning deep architectures for AI. Now Publishers Inc, 2009. find
  27. Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8):1798– 1828, 2013.
  28. K. P. Bennett and J. A. Blue. Optimal decision trees. Technical report, R.P.I. Math Report No. 214, Rensselaer Polytechnic Institute, 1996. find
  29. James C Bezdek, Siew K Chuah, and David Leep. Generalized k-nearest neighbor rules. Fuzzy Sets and Systems, 18(3):237–256, 1986. find
  30. Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yunhan Jia, Joydeep Ghosh, Ruchir Puri, José M. F. Moura, and Peter Eckersley. Explainable machine learning in deployment. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT* ’20), page 648–65… find
  31. Adrien Bibal, Michael Lognoul, Alexandre de Streel, and Benoı̂t Frénay. Legal requirements on explainability in machine learning. Artificial Intelligence and Law, July 30, 2020. find
  32. Jacob Bien and Robert Tibshirani. Prototype Selection for Interpretable Classification. The Annals of Applied Statistics, pages 2403–2424, 2011. find
  33. Lieven Billiet, Sabine Van Huffel, and Vanya Van Belle. Interval coded scoring extensions for larger problems. In Proceedings of the IEEE Symposium on Computers and Communications, pages 198–203. IEEE, 2017. find
  34. Lieven Billiet, Sabine Van Huffel, and Vanya Van Belle. Interval Coded Scoring: A toolbox for interpretable scoring systems. PeerJ Computer Science, 4:e150, 04 2018. find
  35. Harald Binder and Gerhard Tutz. A comparison of methods for the fitting of generalized additive models. Statistics and Computing, 18(1):87–99, 2008. find
  36. Jan Niklas Böhm, Philipp Berens, and Dmitry Kobak. A Unifying Perspective on Neighbor Embeddings along the Attraction-Repulsion Spectrum. arXiv e-print arXiv:2007.08902, 2020. — [arXiv:2007.08902](https://arxiv.org/abs/2007.08902) find
  37. Roger C Bone, Robert A Balk, Frank B Cerra, R Phillip Dellinger, Alan M Fein, William A Knaus, Roland MH Schein, and William J Sibbald. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. Chest, 101(6):1644–1655, 1992. find
  38. Olivier Bousquet and André Elisseeff. Stability and generalization. Journal of Machine Learning Research, 2(Mar):499–526, 2002. find
  39. L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. Classification and Regression Trees. Wadsworth, 1984. find
  40. Leo Breiman et al. Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical Science, 16(3):199–231, 2001. find
  41. Miles Brundage, Shahar Avin, Jasmine Wang, Haydn Belfield, Gretchen Krueger, and Gillian Hadfield et al. Toward trustworthy AI development: Mechanisms for supporting verifiable claims. arXiv e-print arXiv:2004.07213, 2020. find
  42. Ernest W Burgess. Factors determining success or failure on parole. The workings of the indeterminate sentence law and the parole system in Illinois, pages 221–234, 1928. find
  43. Henrik EC Cao, Riku Sarlin, and Alexander Jung. Learning explainable decision rules via maximum satisfiability. IEEE Access, 8:218180–218185, 2020. find
  44. Dallas Card, Michael Zhang, and Noah A Smith. Deep Weighted Averaging Classifiers. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 369–378, 2019. find
  45. Emilio Carrizosa, Amaya Nogales-Gómez, and Dolores Romero Morales. Strongly agree or strongly disagree?: Rating features in support vector machines. Information Sciences, 329: 256 – 273, 2016. Special issue on Discovery Science. find
  46. Emilio Carrizosa, Cristina Molero-Rıo, and Dolores Romero Morales. Mathematical optimization in classification and regression trees. TOP, An Official Journal of the Spanish Society of Statistics and Operations Research, 29:5–33, 2021. find
  47. Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 1721–1730, 2… find
  48. David M Chan, Roshan Rao, Forrest Huang, and John F Canny. Gpu accelerated t-distributed stochastic neighbor embedding. Journal of Parallel and Distributed Computing, 131:1–13, 2019. find
  49. Pete Chapman et al. CRISP-DM 1.0 - step-by-step data mining guide. Technical report, SPSS, 2000. find
  50. Pratik Chaudhari, Anna Choromanska, Stefano Soatto, Yann LeCun, Carlo Baldassi, Christian Borgs, Jennifer Chayes, Levent Sagun, and Riccardo Zecchina. Entropy-SGD: Biasing gradient descent into wide valleys. arXiv e-print arXiv:1611.01838, 2016. — [arXiv:1611.01838](https://arxiv.org/abs/1611.01838) find
  51. Pratik Chaudhari, Anna Choromanska, Stefano Soatto, Yann LeCun, Carlo Baldassi, Christian Borgs, Jennifer Chayes, Levent Sagun, and Riccardo Zecchina. Entropy-sgd: Biasing gradient descent into wide valleys. Journal of Statistical Mechanics: Theory and Experiment, 2019(12): 124018, 2019. find
  52. Chaofan Chen and Cynthia Rudin. An optimization approach to learning falling rule lists. In Proceedings of Artificial Intelligence and Statistics (AISTATS), pages 604–612. PMLR, 2018. find
  53. Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. This Looks Like That: Deep Learning for Interpretable Image Recognition. In Proceedings of Neural Information Processing Systems (NeurIPS), pages 8930–8941, 2019. find
  54. Chaofan Chen, Kangcheng Lin, Cynthia Rudin, Yaron Shaposhnik, Sijia Wang, and Tong Wang. An interpretable model with globally consistent explanations for credit risk. Decision Support Systems, 2021a. Accepted. find
  55. Ricky T. Q. Chen, Xuechen Li, Roger B Grosse, and David K Duvenaud. Isolating sources of disentanglement in variational autoencoders. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Proceedings of Neural Information Processing Systems (NeurIPS), volume… find
  56. Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In Proceedings of Neural Information Processing Systems (NeurIPS), pages 2172–2180, 2016. find
  57. Zhi Chen, Yijie Bei, and Cynthia Rudin. Concept whitening for interpretable image recognition. Nature Machine Intelligence, 2(12):772–782, Dec 2020. find
  58. Zhi Chen, Alex Ogren, Cynthia Rudin, Cate Brinson, and Chiara Daraio. How to see hidden patterns in metamaterials with interpretable machine learning. (in progress), 2021b. find
  59. Yann Chevaleyre, Frederic Koriche, and Jean-Daniel Zucker. Rounding methods for discrete linear classification. In Proceedings of International Conference on Machine Learning (ICML), pages 651–659, 2013. find
  60. Evangelia Christodoulou, Jie Ma, Gary S. Collins, Ewout W. Steyerberg, Jan Y. Verbakel, and Ben Van Calster. A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. Journal of Clinical Epidemiology, 110:12–22, June 2019. find
  61. Peter Clark and Robin Boswell. Rule induction with CN2: Some recent improvements. In European Working Session on Learning, pages 151–163. Springer, 1991. find
  62. Peter Clark and Tim Niblett. The CN2 induction algorithm. Machine Learning, 3(4):261–283, 1989. find
  63. Andy Coenen and Adam Pearce. Understanding UMAP. https://pair-code.github.io/understanding-umap/, 2019. ↗
  64. William W Cohen. Fast effective rule induction. In Proceedings of the Twelfth International Conference on Machine Learning (ICML), pages 115–123, 1995. find
  65. Youri Coppens, Kyriakos Efthymiadis, Tom Lenaerts, Ann Nowé, Tim Miller, Rosina Weber, and Daniele Magazzeni. Distilling deep reinforcement learning policies in soft decision trees. In Proceedings of the IJCAI 2019 Workshop on Explainable Artificial Intelligence, pages 1–6, 2019. find
  66. Amanda Coston, Ashesh Rambachan, and Alexandra Chouldechova. Characterizing fairness over the set of good models under selective labels. arXiv e-print arXiv:2101.00352, 2021. — [arXiv:2101.00352](https://arxiv.org/abs/2101.00352) find
  67. T. Cover and P. Hart. Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13(1):21–27, 1967. find
  68. Mark W Craven. Extracting comprehensible models from trained neural networks. PhD thesis, University of Wisconsin-Madison Department of Computer Sciences, 1996. find
  69. Mark W. Craven and Jude W. Shavlik. Extracting tree-structured representations of trained networks. In Proceedings of the 8th International Conference on Neural Information Processing Systems (NIPS), page 24–30, 1995. find
  70. Eric Crawford and Joelle Pineau. Spatially invariant unsupervised object detection with convolutional neural networks. In Proceedings of AAAI Conference on Artificial Intelligence (AAAI), volume 33, pages 3412–3420, 2019. find
  71. Şeyda Ertekin and Cynthia Rudin. On equivalence relationships between classification and ranking algorithms. Journal of Machine Learning Research, 12:2905–2929, 2011. find
  72. Nathan Dahlin, Krishna Chaitanya Kalagarla, Nikhil Naik, Rahul Jain, and Pierluigi Nuzzo. Designing interpretable approximations to deep reinforcement learning with soft decision trees. arXiv e-print arXiv:2010.14785, 2020. — [arXiv:2010.14785](https://arxiv.org/abs/2010.14785) find
  73. Alexander D’Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D Hoffman, et al. Underspecification presents challenges for credibility in modern machine learning. arXiv e-print arXiv:2011.03395, 2020. — [arXiv:2011.03395](https://arxiv.org/abs/2011.03395) find
  74. Srijita Das, Sriraam Natarajan, Kaushik Roy, Ronald Parr, and Kristian Kersting. Fitted Q-learning for relational domains. arXiv e-print arXiv:2006.05595, 2020. — [arXiv:2006.05595](https://arxiv.org/abs/2006.05595) find
  75. Subhajit Das, Dylan Cashman, Remco Chang, and Alex Endert. BEAMES: Interactive multi-model steering, selection, and inspection for regression tasks. IEEE Computer Graphics and Applications, 39(5):20–32, 2019. find
  76. Sanjeeb Dash, Oktay Günlük, and Dennis Wei. Boolean decision rules via column generation. In Proceedings of Neural Information Processing Systems (NeurIPS), volume 31, pages 4655– 4665, 2018. find
  77. Emir Demirović, Anna Lukina, Emmanuel Hebrard, Jeffrey Chan, James Bailey, Christopher Leckie, Kotagiri Ramamohanarao, and Peter J Stuckey. MurTree: Optimal classification trees via dynamic programming and search. arXiv e-print arXiv:2007.12652, 2020. — [arXiv:2007.12652](https://arxiv.org/abs/2007.12652) find
  78. Guillaume Desjardins, Aaron Courville, and Yoshua Bengio. Disentangling factors of variation via generative entangling. arXiv e-print arXiv:1210.5474, 2012. — [arXiv:1210.5474](https://arxiv.org/abs/1210.5474) find
  79. Sunny Dhamnani, Dhruv Singal, Ritwik Sinha, Tharun Mohandoss, and Manish Dash. RAPID: Rapid and precise interpretable decision sets. In 2019 IEEE International Conference on Big Data (Big Data), pages 1292–1301. find
  80. Yashesh Dhebar, Kalyanmoy Deb, Subramanya Nageshrao, Ling Zhu, and Dimitar Filev. Interpretable-AI policies using evolutionary nonlinear decision trees for discrete action systems. arXiv e-print arXiv:2009.09521, 2020. — [arXiv:2009.09521](https://arxiv.org/abs/2009.09521) find
  81. Botty Dimanov, Umang Bhatt, Mateja Jamnik, and Adrian Weller. You shouldn’t trust me: Learning models which conceal unfairness from multiple explanation methods. In 24th European Conference on Artificial Intelligence (ECAI), 2020. find
  82. Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. Sharp minima can generalize for deep nets. arXiv e-print arXiv:1703.04933, 2017. — [arXiv:1703.04933](https://arxiv.org/abs/1703.04933) find
  83. Kien Do and Truyen Tran. Theory and evaluation metrics for learning disentangled representations. arXiv e-print arXiv:1908.09961, 2019. — [arXiv:1908.09961](https://arxiv.org/abs/1908.09961) find
  84. David Dobkin, Truxton Fulton, Dimitrios Gunopulos, Simon Kasif, and Steven Salzberg. Induction of shallow decision trees. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997. find
  85. Jiayun Dong and Cynthia Rudin. Exploring the cloud of variable importance for the set of all good models. Nature Machine Intelligence, 2(12):810–824, 2020. find
  86. David L. Donoho and Carrie Grimes. Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data. Proceedings of the National Academy of Arts and Sciences, 100(10): 5591–5596, 2003. find
  87. Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL http://archive.ics.uci.edu/ml. ↗
  88. S. A. Dudani. The Distance-Weighted k-Nearest-Neighbor Rule. IEEE Transactions on Systems, Man, and Cybernetics, SMC-6(4):325–327, 1976. find
  89. Sašo Džeroski, Luc De Raedt, and Hendrik Blockeel. Relational reinforcement learning. In International Conference on Inductive Logic Programming, pages 11–22. Springer, 1998. find
  90. Sašo Džeroski, Luc De Raedt, and Kurt Driessens. Relational reinforcement learning. Machine Learning, 43(1-2):7–52, 2001. find
  91. Cian Eastwood and Christopher KI Williams. A framework for the quantitative evaluation of disentangled representations. In Proceedings of the International Conference on Learning Representations (ICLR), 2018. find
  92. Tapio Elomaa. In defense of c4.5: Notes on learning one-level decision trees. In Proceedings of International Conference on Machine Learning (ICML), pages 62–69. Morgan Kaufmann, 1994. find
  93. A. Farhangfar, R. Greiner, and M. Zinkevich. A fast way to produce optimal fixed-depth decision trees. In International Symposium on Artificial Intelligence and Mathematics (ISAIM), 2008. find
  94. Usama Fayyad, Gregory Piatetsky-Shapiro, and Padhraic Smyth. From data mining to knowledge discovery in databases. AI Magazine, 17:37–54, 1996. find
  95. Sarah Finney, Natalia Gardiol, Leslie Pack Kaelbling, and Tim Oates. The thing that we tried didn’t work very well: Deictic representation in reinforcement learning. arXiv e-print arXiv:1301.0567, 2012. — [arXiv:1301.0567](https://arxiv.org/abs/1301.0567) find
  96. Aaron Fisher, Cynthia Rudin, and Francesca Dominici. All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously. Journal of Machine Learning Research, 20(177):1–81, 2019. find
  97. E. Fix and J.L. Hodges. Discriminatory analysis, nonparametric discrimination: Consistency properties. Technical Report 4, 1951. find
  98. Eibe Frank and Ian H Witten. Generating accurate rule sets without global optimization. In Proceedings of International Conference on Machine Learning (ICML), pages 144–151, 1998. find
  99. Alex A Freitas. Comprehensible classification models: a position paper. Explorations Newsletter, 15(1):1–10, 2014. find
  100. ACM SIGKDD Yoav Freund and Robert E Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119–139, 1997. find
  101. Jerome H Friedman. Greedy function approximation: a gradient boosting machine. Annals of Statistics, pages 1189–1232, 2001. find
  102. Jerome H Friedman. Stochastic gradient boosting. Computational Statistics & Data Analysis, 38 (4):367–378, 2002. find
  103. Jerome H Friedman and Nicholas I Fisher. Bump hunting in high-dimensional data. Statistics and Computing, 9(2):123–143, 1999. find
  104. Cong Fu, Yonghui Zhang, Deng Cai, and Xiang Ren. Atsne: Efficient and robust visualization on gpu through hierarchical optimization. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), page 176–186, New York, NY, USA, 2019. find
  105. Keinosuke Fukunaga and L Hostetler. Optimization of k nearest neighbor density estimates. IEEE Transactions on Information Theory, 19(3):320–326, 1973. find
  106. Brian F Gage, Amy D Waterman, William Shannon, Michael Boechler, Michael W Rich, and Martha J Radford. Validation of clinical classification schemes for predicting stroke: results from the national registry of atrial fibrillation. Journal of the American Medical Association (JAMA), 285(22):2864–2870… find
  107. Brian R. Gaines and Paul Compton. Induction of ripple-down rules applied to modeling large databases. Journal of Intelligent Information Systems, 5(3):211–228, 1995. find
  108. Bishwamittra Ghosh and Kuldeep S Meel. IMLI: An incremental framework for MaxSAT-based learning of interpretable classification rules. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (AIES), pages 203–210, 2019. find
  109. Siong Thye Goh and Cynthia Rudin. Box drawings for learning with imbalanced data. In Proceedings of the 20th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2014. find
  110. Oscar Gomez, Steffen Holter, Jun Yuan, and Enrico Bertini. ViCE: Visual counterfactual explanations for machine learning models. In Proceedings of the 25th International Conference on Intelligent User Interfaces (IUI’20), 2020. URL http://nyuvis-web.poly.edu/ projects/fico/intro,https://github.com/n… ↗
  111. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Proceedings of Neural Information Processing Systems (NeurIPS), volume 27, pages 2672–2680, 2014. find
  112. Alicja Gosiewska and Przemyslaw Biecek. Do not trust additive explanations. arXiv e-print arXiv:1903.11420, 2020. — [arXiv:1903.11420](https://arxiv.org/abs/1903.11420) find
  113. J Brian Gray and Guangzhe Fan. Classification tree analysis using target. Computational Statistics & Data Analysis, 52(3):1362–1372, 2008. find
  114. Charles C. Gross. Genealogy of the “Grandmother Cell”. The Neuroscientist, 8(5):512–518, 2002. find
  115. Arthur Guez, Robert D Vincent, Massimo Avoli, and Joelle Pineau. Adaptive treatment of epilepsy via batch-mode reinforcement learning. In Proceedings of AAAI Conference on Artificial Intelligence (AAAI), pages 1671–1678, 2008. find
  116. Ryuji Hamamoto, Kruthi Suvarna, Masayoshi Yamada, Kazuma Kobayashi, Norio Shinkai, Mototaka Miyake, Masamichi Takahashi, Shunichi Jinnai, Ryo Shimoyama, Akira Sakai, Ken Taka-sawa, Amina Bolatkan, Kanto Shozu, Ai Dozen, Hidenori Machino, Satoshi Takahashi, Ken Asada, Masaaki Komatsu, Jun Sese, and S… find
  117. K. Hammond. Proceedings: Second Case-Based Reasoning Workshop (DARPA). Morgan Kanf-mann Publishers, Inc., San Mateo, CA, 1989. find
  118. Asad Haris, Noah Simon, and Ali Shojaie. Generalized sparse additive models. arXiv e-print arXiv:1903.04641, 2019. — [arXiv:1903.04641](https://arxiv.org/abs/1903.04641) find
  119. Trevor J Hastie and Robert J Tibshirani. Generalized additive models, volume 43. CRC press, 1990. find
  120. Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. In Proceedings of the International Conference on Learning Representations (ICLR),… find
  121. Irina Higgins, David Amos, David Pfau, Sebastien Racaniere, Loic Matthey, Danilo Rezende, and Alexander Lerchner. Towards a definition of disentangled representations. arXiv e-print arXiv:1812.02230, 2018. — [arXiv:1812.02230](https://arxiv.org/abs/1812.02230) find
  122. Geoffrey Hinton. How to represent part-whole hierarchies in a neural network. arXiv e-print arXiv:2102.12627, 2021. — [arXiv:2102.12627](https://arxiv.org/abs/2102.12627) find
  123. Geoffrey E Hinton, Alex Krizhevsky, and Sida D Wang. Transforming auto-encoders. International Conference on Artificial Neural Networks, pages 44–51. Springer, 2011. find
  124. In Sepp Hochreiter and Jürgen Schmidhuber. Flat minima. Neural Computation, 9(1):1–42, 1997. find
  125. Robert C. Holte. Very simple classification rules perform well on most commonly used datasets. Machine Learning, 11(1):63–91, 1993. find
  126. Dat Hong, Stephen S Baek, and Tong Wang. Interpretable sequence classification via prototype trajectory. arXiv e-print arXiv:2007.01777, 2020. — [arXiv:2007.01777](https://arxiv.org/abs/2007.01777) find
  127. Hao Hu, Mohamed Siala, Emmanuel Hébrard, and Marie-José Huguet. Learning optimal decision trees with MaxSAT and its integration in Adaboost. In IJCAI-PRICAI 2020, 29th International Joint Conference on Artificial Intelligence and the 17th Pacific Rim International Conference on Artificial Intellig… find
  128. Xiyang Hu, Cynthia Rudin, and Margo Seltzer. Optimal sparse decision trees. In Proceedings of Neural Information Processing Systems (NeurIPS), 2019. find
  129. Alexey Ignatiev, Filipe Pereira, Nina Narodytska, and Joao Marques-Silva. A SAT-based approach to learn explainable decision sets. In International Joint Conference on Automated Reasoning, pages 627–645, 2018. find
  130. Mathieu Jacomy, Tommaso Venturini, Sebastien Heymann, and Mathieu Bastian. ForceAtlas2, a continuous graph layout algorithm for handy network visualization designed for the gephi software. PLoS One, 9(6):1–12, 06 2014. find
  131. Max Jaderberg, Karen Simonyan, Andrew Zisserman, and Koray Kavukcuoglu. Spatial transformer networks. arXiv e-print arXiv:1506.02025, 2015. find
  132. Mahsa Oroojeni Mohammad Javad, Stephen Olusegun Agboola, Kamal Jethwani, Abe Zeid, and Sagar Kamarthi. A reinforcement learning–based method for management of type 1 diabetes: exploratory study. JMIR diabetes, 4(3):e12905, 2019. find
  133. Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C. Lawrence Zitnick, and Ross Girshick. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
  134. Zoe Juozapaitis, Anurag Koul, Alan Fern, Martin Erwig, and Finale Doshi-Velez. Explainable reinforcement learning via reward decomposition. In IJCAI/ECAI Workshop on Explainable Artificial Intelligence, 2019. find
  135. Rishabh Kabra, Chris Burgess, Loic Matthey, Raphael Lopez Kaufman, Klaus Greff, Malcolm Reynolds, and Alexander Lerchner. Multi-object datasets. https://github.com/deepmind/multi-object-datasets/, 2019. ↗
  136. Sham M Kakade, Karthik Sridharan, and Ambuj Tewari. On the complexity of linear prediction: Risk bounds, margin bounds, and regularization. In Proceedings of Neural Information Processing Systems (NeurIPS), pages 793–800, 2009. find
  137. Harmanpreet Kaur, Harsha Nori, Samuel Jenkins, Rich Caruana, Hanna Wallach, and Jennifer Wortman Vaughan. Interpreting interpretability: Understanding data scientists’ use of interpretability tools for machine learning. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems,… find
  138. James M Keller, Michael R Gray, and James A Givens. A fuzzy k-nearest neighbor algorithm. IEEE Transactions on Systems, Man, and Cybernetics, 4:580–585, 1985. find
  139. Kristian Kersting and Kurt Driessens. Non-parametric policy gradients: A unified treatment of propositional and relational domains. In Proceedings of the 25th International Conference on Machine Learning, pages 456–463, 2008. find
  140. Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large-batch training for deep learning: Generalization gap and sharp minima. arXiv e-print arXiv:1609.04836, 2016. find
  141. Been Kim, Cynthia Rudin, and Julie A Shah. The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification. In Proceedings of Neural Information Processing Systems (NeurIPS), volume 27, pages 1952–1960, 2014. find
  142. Been Kim, Rajiv Khanna, and Oluwasanmi O Koyejo. Examples are not enough, learn to criticize! criticism for interpretability. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Proceedings of Neural Information Processing Systems (NeurIPS), volume 29, pages 2280–2288, 2016. find
  143. Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In Proceedings of International Conference on Machine Learning (ICML), pages 2668– 2677. PMLR, 2018. find
  144. Hyunjik Kim and Andriy Mnih. Disentangling by factorising. arXiv e-print arXiv:1802.05983, 2018. — [arXiv:1802.05983](https://arxiv.org/abs/1802.05983) find
  145. Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In 2nd International Conference on Learning Representations (ICLR), 2013.
  146. B Ravi Kiran, Ibrahim Sobh, Victor Talpaert, Patrick Mannion, Ahmad A Al Sallab, Senthil Yogamani, and Patrick Pérez. Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems, pages 1–18, 2021. find
  147. Nicholas Kissel and Lucas Mentch. Forward stability and model path selection. arXiv e-print arXiv:2103.03462, 2021. — [arXiv:2103.03462](https://arxiv.org/abs/2103.03462) find
  148. William A Knaus, Jack E Zimmerman, Douglas P Wagner, Elizabeth A Draper, and Diane E Lawrence. Apache-acute physiology and chronic health evaluation: a physiologically based classification system. Critical Care Medicine, 9(8):591–597, 1981. find
  149. William A Knaus, Elizabeth A Draper, Douglas P Wagner, and Jack E Zimmerman. APACHE II: a severity of disease classification system. Critical Care Medicine, 13(10):818–829, 1985. find
  150. William A Knaus, Douglas P Wagner, Elizabeth A Draper, Jack E Zimmerman, Marilyn Bergner, Paulo G Bastos, Carl A Sirio, Donald J Murphy, Ted Lotring, Anne Damiano, et al. The APACHE III prognostic system: risk prediction of hospital mortality for critically ill hospitalized adults. Chest, 100(6):161… find
  151. Dmitry Kobak and Philipp Berens. The art of using t-SNE for single-cell transcriptomics. Nature Communication, 10:5416, 2019. find
  152. Jens Kober, J Andrew Bagnell, and Jan Peters. Reinforcement learning in robotics: A survey. The International Journal of Robotics Research, 32(11):1238–1274, 2013. find
  153. Yves Kodratoff. The comprehensibility manifesto. KDD Nugget Newsletter, 94(9), 1994. find
  154. Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In Proceedings of International Conference on Machine Learning (ICML), pages 5338–5348. PMLR, 2020. find
  155. Teuvo Kohonen. Learning Vector Quantization. The Handbook of Brain Theory and Neural Networks, page 537–540, 1995. find
  156. J. L. Kolodner. Proceedings of the Case-Based Reasoning Workshop (DARPA). Kaufmmnn Publishers, Inc., San Mateo, CA, 1988. find
  157. Morgan Vladimir Koltchinskii and Dmitry Panchenko. Empirical margin distributions and bounding the generalization error of combined classifiers. The Annals of Statistics, 30(1):1–50, 2002. find
  158. Matthieu Komorowski, Leo A Celi, Omar Badawi, Anthony C Gordon, and A Aldo Faisal. The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care. Nature Medicine, 24(11):1716–1720, 2018. find
  159. Adam Kosiorek, Sara Sabour, Yee Whye Teh, and Geoffrey E Hinton. Stacked capsule autoencoders. In Proceedings of Neural Information Processing Systems (NeurIPS), pages 15512– 15522, 2019. find
  160. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6), May 2017. find
  161. Tejas D Kulkarni, Will Whitney, Pushmeet Kohli, and Joshua B Tenenbaum. Deep convolutional inverse graphics network. arXiv e-print arXiv:1503.03167, 2015. — [arXiv:1503.03167](https://arxiv.org/abs/1503.03167) find
  162. Ugur Kursuncu, Manas Gaur, and Amit Sheth. Knowledge Infused Learning (K-IL): Towards Deep Incorporation of Knowledge in Deep Learning. In Proceedings of the AAAI 2020 Spring Symposium on Combining Machine Learning and Knowledge Engineering in Practice (AAAI-MAKE 2020), volume I, 2020. find
  163. I. E. Lagaris, A. C. Likas, and D. G. Papageorgiou. Neural-network methods for boundary value problems with irregular boundaries. IEEE Transactions on Neural Networks, 11(5):1041–1049, 2000. find
  164. Isaac E Lagaris, Aristidis Likas, and Dimitrios I Fotiadis. Artificial neural networks for solving ordinary and partial differential equations. IEEE Transactions on Neural Networks, 9(5):987– 1000, 1998. find
  165. Himabindu Lakkaraju and Osbert Bastani. “How do I fool you?”: Manipulating user trust via misleading black box explanations. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (AIES), page 79–85, February 2020. find
  166. Himabindu Lakkaraju, Stephen H Bach, and Jure Leskovec. Interpretable decision sets: A joint framework for description and prediction. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 1675–1684, 2016. find
  167. Sebastian Lapuschkin, Stephan Wäldchen, Alexander Binder, Grégoire Montavon, Wojciech Samek, and Klaus-Robert Müller. Unmasking Clever Hans predictors and assessing what machines really learn. Nature Communications, 10(1096), 2019. find
  168. Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin De-tyniecki. The dangers of post-hoc interpretability: Unjustified counterfactual explanations. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 2801–2807,… find
  169. Hyafil Laurent and Ronald L Rivest. Constructing optimal binary decision trees is NP-complete. Information Processing Letters, 5(1):15–17, 1976. find
  170. Jean-Roger Le Gall, Stanley Lemeshow, and Fabienne Saulnier. A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. Journal of the American Medical Association (JAMA), 270(24):2957–2963, 1993. find
  171. Yann LeCun, Corinna Cortes, and CJ Burges. MNIST handwritten digit database. ATT Labs [Online]. Available: http://yann.lecun. com/exdb/mnist, 2, 2010. ↗
  172. Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553):436–444, 2015. find
  173. Hyuk Lee and In Seok Kang. Neural algorithm for solving differential equations. Journal of Computational Physics, 91(1):110–131, 1990. find
  174. Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention-based permutation-invariant neural networks. In Proceedings of International Conference on Machine Learning (ICML), pages 3744–3753. PMLR, 2019. find
  175. Benjamin Letham, Cynthia Rudin, Tyler H McCormick, David Madigan, et al. Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model. The Annals of Applied Statistics, 9(3):1350–1371, 2015. find
  176. Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. arXiv e-print arXiv:1712.09913, 2017. — [arXiv:1712.09913](https://arxiv.org/abs/1712.09913) find
  177. Jiwei Li, Will Monroe, Alan Ritter, Michel Galley, Jianfeng Gao, and Dan Jurafsky. Deep reinforcement learning for dialogue generation. arXiv e-print arXiv:1606.01541, 2016.
  178. Oscar Li, Hao Liu, Chaofan Chen, and Cynthia Rudin. Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions. In Proceedings of AAAI Conference on Artificial Intelligence (AAAI), 2018. find
  179. Wenmin Li, Jiawei Han, and Jian Pei. Cmar: Accurate and efficient classification based on multiple class-association rules. In Proceedings 2001 IEEE International Conference on Data Mining (ICDM), pages 369–376. find
  180. Jimmy Lin, Chudi Zhong, Diane Hu, Cynthia Rudin, and Margo Seltzer. Generalized and scalable optimal sparse decision trees. In Proceedings of International Conference on Machine Learning (ICML), pages 6150–6160. PMLR, 2020. find
  181. Yi Lin, Hao Helen Zhang, et al. Component selection and smoothing in multivariate nonparametric regression. The Annals of Statistics, 34(5):2272–2297, 2006. find
  182. George C. Linderman, Manas Rachh, Jeremy G. Hoskins, Stefan Steinerberger, and Yuval Kluger. Fast interpolation-based t-SNE for improved visualization of single-cell RNA-seq data. Nature Methods, 16:243–245, 2019. find
  183. Bing Liu, Wynne Hsu, Yiming Ma, et al. Integrating classification and association rule mining. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), volume 98, pages 80–86, 1998. find
  184. Guiliang Liu, Oliver Schulte, Wang Zhu, and Qingcan Li. Toward interpretable deep reinforcement learning with linear model u-trees. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 414–429. Springer, 2018. find
  185. Ying Liu, Brent Logan, Ning Liu, Zhiyuan Xu, Jian Tang, and Yangzhi Wang. Deep reinforcement learning for dynamic treatment regimes on medical registry data. In 2017 IEEE International Conference on Healthcare Informatics (ICHI), pages 380–385. find
  186. Samuele Lo Piano. Ethical principles in machine learning and artificial intelligence: cases from the field and possible ways forward. Humanit Soc Sci Commun, 7(9), 2020. find
  187. Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot attention. arXiv e-print arXiv:2006.15055, 2020. find
  188. Wei-Yin Loh and Yu-Shan Shih. Split selection methods for classification trees. Statistica Sinica, 7:815–840, 1997. find
  189. Max Losch, Mario Fritz, and Bernt Schiele. Interpretability beyond classification output: Semantic bottleneck networks. arXiv e-print arXiv:1907.10882, 2019. — [arXiv:1907.10882](https://arxiv.org/abs/1907.10882) find
  190. Yin Lou, Rich Caruana, and Johannes Gehrke. Intelligible models for classification and regression. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 150–158, 2012. find
  191. Yin Lou, Rich Caruana, Johannes Gehrke, and Giles Hooker. Accurate intelligible models with pairwise interactions. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 623–631, 2013. find
  192. Yin Lou, Jacob Bien, Rich Caruana, and Johannes Gehrke. Sparse partially linear additive models. Journal of Computational and Graphical Statistics, 25(4):1126–1140, 2016. find
  193. Dmitry Malioutov and Kuldeep S Meel. MLIC: A MaxSAT-based framework for learning interpretable classification rules. In International Conference on Principles and Practice of Constraint Programming, pages 312–327. Springer, 2018. find
  194. Dmitry Malioutov and Kush Varshney. Exact rule learning via boolean compressed sensing. In Proceedings of International Conference on Machine Learning (ICML), pages 765–773. PMLR, 2013. find
  195. Mario Marchand and John Shawe-Taylor. The set covering machine. Journal of Machine Learning Research, 3(Dec):723–746, 2002. find
  196. Mario Marchand and Marina Sokolova. Learning with decision lists of data-dependent features. Journal of Machine Learning Research, 6(Apr):427–451, 2005. find
  197. Charles Marx, Flavio Calmon, and Berk Ustun. Predictive multiplicity in classification. In Proceedings of International Conference on Machine Learning (ICML), pages 6765–6774. PMLR, 2020. find
  198. Loic Matthey, Irina Higgins, Demis Hassabis, and Alexander Lerchner. dsprites: Disentanglement testing sprites dataset. https://github.com/deepmind/dsprites-dataset/, 2017. ↗
  199. Michael McGough. How bad is sacramento’s air, exactly? Google results appear at odds with reality, some say. Sacramento Bee, August 7 2018. find
  200. Leland McInnes, John Healy, and James Melville. UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv e-print arXiv:1802.03426, February 2018.
  201. Manish Mehta, Rakesh Agrawal, and Jorma Rissanen. SLIQ: A fast scalable classifier for data mining. In International Conference on Extending Database Technology, pages 18–32. Springer, 1996. find
  202. Lukas Meier, Sara Van de Geer, Peter Bühlmann, et al. High-dimensional additive modeling. The Annals of Statistics, 37(6B):3779–3821, 2009. find
  203. Xuhui Meng, Zhen Li, Dongkun Zhang, and George Em Karniadakis. PPINN: Parareal physics-informed neural network for time-dependent pdes. Computer Methods in Applied Mechanics and Engineering, 370:113250, 2020. find
  204. Matt Menickelly, Oktay Günlük, Jayant Kalagnanam, and Katya Scheinberg. Optimal decision trees for categorical data via integer programming. arXiv e-print arXiv:1612.03225, January 2018. — [arXiv:1612.03225](https://arxiv.org/abs/1612.03225) find
  205. George Miller. The magical number seven, plus or minus two: Some limits on our capacity for processing information. The Psychological Review, 63:81–97, 1956. find
  206. Yao Ming, Panpan Xu, Huamin Qu, and Liu Ren. Interpretable and Steerable Sequence Learning via Prototypes. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 903–913, 2019. find
  207. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv e-print arXiv:1312.5602, 2013. find
  208. Rui P Moreno, Philipp GH Metnitz, Eduardo Almeida, Barbara Jordan, Peter Bauer, Ricardo Abizanda Campos, Gaetano Iapichino, David Edbrooke, Maurizia Capuzzo, Jean-Roger Le Gall, et al. SAPS 3 — from evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognost… find
  209. James N. Morgan and John A. Sonquist. Problems in the analysis of survey data, and a proposal. J. Amer. Statist. Assoc., 58:415–434, 1963. find
  210. Alexander Mott, Daniel Zoran, Mike Chrzanowski, Daan Wierstra, and Danilo Jimenez Rezende. Towards interpretable reinforcement learning using attention augmented agents. In Proceedings of Neural Information Processing Systems (NeurIPS), pages 12350–12359, 2019. find
  211. Nina Narodytska, Alexey Ignatiev, Filipe Pereira, and Joao Marques-Silva. Learning optimal decision trees with SAT. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI), pages 1362–1368, 2018. find
  212. Meike Nauta, Annemarie Jutte, Jesper Provoost, and Christin Seifert. This Looks Like That, Because... Explaining Prototypes for Interpretable Image Recognition. arXiv e-print arXiv:2011.02863, 2020a. — [arXiv:2011.02863](https://arxiv.org/abs/2011.02863) find
  213. Meike Nauta, Ron van Bree, and Christin Seifert. Neural prototype trees for interpretable fine-grained image recognition. arXiv e-print arXiv:2012.02046, 2020b. — [arXiv:2012.02046](https://arxiv.org/abs/2012.02046) find
  214. John Ashworth Nelder and Robert WM Wedderburn. Generalized linear models. Journal of the Royal Statistical Society: Series A (General), 135(3):370–384, 1972. find
  215. Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011. find
  216. Allen Newell, Herbert Alexander Simon, et al. Human Problem Solving. Prentice-Hall Englewood Cliffs, NJ, 1972. find
  217. S. Nijssen and E. Fromont. Optimal constraint-based decision tree induction from itemset lattices. Data Mining and Knowledge Discovery, 21(1):9–51, 2010. find
  218. Siegfried Nijssen and Elisa Fromont. Mining optimal decision trees from itemset lattices. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 530–539. ACM, 2007. find
  219. Siegfried Nijssen, Pierre Schaus, et al. Learning optimal decision trees using caching branch-and-bound search. In Proceedings of AAAI Conference on Artificial Intelligence (AAAI), 2020. find
  220. Harsha Nori, Samuel Jenkins, Paul Koch, and Rich Caruana. Interpretml: A unified framework for machine learning interpretability. arXiv e-print arXiv:1909.09223, 2019. — [arXiv:1909.09223](https://arxiv.org/abs/1909.09223) find
  221. Northpointe. Practitioner’s Guide to COMPAS Core. Technical report, Northpointe, March 2013. find
  222. Matt O’Connor. Algorithm’s ‘unexpected’ weakness raises larger concerns about AI’s potential in broader populations. Health Imaging, Artificial Intelligence section, April 05, 2021. find
  223. Guofei Pang, Lu Lu, and George Em Karniadakis. fPINNs: Fractional physics-informed neural networks. SIAM Journal on Scientific Computing, 41(4):A2603–A2626, 2019a. find
  224. Guofei Pang, Liu Yang, and George Em Karniadakis. Neural-net-induced gaussian process regression for function approximation and pde solution. Journal of Computational Physics, 384: 270–288, 2019b. find
  225. Athanassios Papagelis and Dimitrios Kalles. GA tree: genetically evolved decision trees. In Proceedings 12th IEEE Internationals Conference on Tools with Artificial Intelligence (ICTAI), pages 203–206. IEEE, 2000. find
  226. Nicolas Papernot and Patrick McDaniel. Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning. arXiv e-print arXiv:1803.04765, 2018. — [arXiv:1803.04765](https://arxiv.org/abs/1803.04765) find
  227. Sonali Parbhoo, Jasmina Bogojeska, Maurizio Zazzi, Volker Roth, and Finale Doshi-Velez. Combining kernel and model based learning for HIV therapy selection. AMIA Summits on Translational Science Proceedings, 2017:239, 2017. find
  228. Harsh Parikh, Cynthia Rudin, and Alexander Volfovsky. An application of matching after learning to stretch (MALTS) to the ACIC 2018 causal inference challenge data. Observational Studies, pages 118–130, 2019. find
  229. Karl Pearson. On lines and planes of closest fit to systems of points in space. Philosophical Magazine, 2(11):559–572, 1901. find
  230. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830… find
  231. Ashley Petersen, Daniela Witten, and Noah Simon. Fused lasso additive model. Journal of Computational and Graphical Statistics, 25(4):1005–1025, 2016. find
  232. Pavlin G. Poličar, Martin Stražar, and Blaž Zupan. openTSNE: a modular python library for t-SNE dimensionality reduction and embedding. bioRxiv, 2019. find
  233. Forough Poursabzi-Sangdeh, Daniel G. Goldstein, Jake M. Hofman, Jennifer Wortman Vaughan, and Hanna M. Wallach. Manipulating and measuring model interpretability. arXiv e-print arXiv:1802.07810, 2018. — [arXiv:1802.07810](https://arxiv.org/abs/1802.07810) find
  234. Dimitris C Psichogios and Lyle H Ungar. A hybrid neural network-first principles approach to process modeling. AIChE Journal, 38(10):1499–1511, 1992. find
  235. J. R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, 1993.
  236. Maziar Raissi. Deep hidden physics models: Deep learning of nonlinear partial differential equations. Journal of Machine Learning Research, 19(25):1–24, 2018. find
  237. Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems. arXiv e-print arXiv:1801.01236, 2018. — [arXiv:1801.01236](https://arxiv.org/abs/1801.01236) find
  238. Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019. find
  239. Maziar Raissi, Alireza Yazdani, and George Em Karniadakis. Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations. Science, 367(6481):1026–1030, 2020. find
  240. Chengping Rao, Hao Sun, and Yang Liu. Physics informed deep learning for computational elas-todynamics without labeled data. arXiv e-print arXiv:2006.08472, 2020. — [arXiv:2006.08472](https://arxiv.org/abs/2006.08472) find
  241. Pradeep Ravikumar, John Lafferty, Han Liu, and Larry Wasserman. Sparse additive models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71(5):1009–1030, 2009. find
  242. C. K. Riesbeck and R. S. Schank. Inside Case-Based Reasoning. Lawrence Erlbaum Assoc., Inc., Hillsdale, NJ, 1989. find
  243. Ronald L Rivest. Learning decision lists. Machine learning, 2(3):229–246, 1987. find
  244. Aaron M Roth, Nicholay Topin, Pooyan Jamshidi, and Manuela Veloso. Conservative Q-improvement: Reinforcement learning for an interpretable decision-tree policy. arXiv e-print arXiv:1907.01180, 2019. — [arXiv:1907.01180](https://arxiv.org/abs/1907.01180) find
  245. Sam T. Roweis and Lawrence K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323–2326, 2000. find
  246. Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1:206–215, May 2019.
  247. Cynthia Rudin and Seyda Ertekin. Learning customized and optimized lists of rules with mathematical programming. Mathematical Programming C (Computation), 10:659–702, 2018. find
  248. Cynthia Rudin and Joanna Radin. Why are we using black box models in AI when we don’t need to? A lesson from an explainable AI competition. Harvard Data Science Review, 1(2), 2019. find
  249. Cynthia Rudin and Robert E. Schapire. Margin-based ranking and an equivalence between AdaBoost and RankBoost. Journal of Machine Learning Research, 10:2193–2232, Oct 2009. find
  250. Cynthia Rudin and Yaron Shaposhnik. Globally-consistent rule-based summary-explanations for machine learning models: Application to credit-risk evaluation. May 28, 2019. Available at SSRN: https://ssrn.com/abstract=3395422, 2019. ↗
  251. Cynthia Rudin and Berk Ustun. Optimized scoring systems: Toward trust in machine learning for healthcare and criminal justice. Interfaces, 48(5):449–466, 2018. find
  252. Cynthia Rudin and Kiri L. Wagstaff. Machine learning for science and society. Machine Learning, 95(1), 2014. find
  253. Cynthia Rudin, Benjamin Letham, and David Madigan. Learning theory analysis for association rules and sequential event prediction. Journal of Machine Learning Research, 14(1):3441–3492, 2013. find
  254. Cynthia Rudin, Caroline Wang, and Beau Coker. The age of secrecy and unfairness in recidivism prediction. Harvard Data Science Review, 2(1), 2020. find
  255. Dawid Rymarczyk, Łukasz Struski, Jacek Tabor, and Bartosz Zieliński. ProtoPShare: Prototype Sharing for Interpretable Image Classification and Similarity Discovery. arXiv e-print arXiv:2011.14340, 2020. — [arXiv:2011.14340](https://arxiv.org/abs/2011.14340) find
  256. Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. Dynamic routing between capsules. In Proceedings of Neural Information Processing Systems (NeurIPS), pages 3856–3866, 2017. find
  257. Veeranjaneyulu Sadhanala, Ryan J Tibshirani, et al. Additive models with trend filtering. The Annals of Statistics, 47(6):3032–3068, 2019. find
  258. Francisco Sahli Costabal, Yibo Yang, Paris Perdikaris, Daniel E Hurtado, and Ellen Kuhl. Physics-informed neural networks for cardiac activation mapping. Frontiers in Physics, 8:42, 2020. find
  259. Ruslan Salakhutdinov and Geoff Hinton. Learning a nonlinear embedding by preserving class neighbourhood structure. In Artificial Intelligence and Statistics, pages 412–419, 2007. find
  260. Ahmad EL Sallab, Mohammed Abdou, Etienne Perot, and Senthil Yogamani. Deep reinforcement learning framework for autonomous driving. Electronic Imaging, 2017(19):70–76, 2017. find
  261. Robert E Schapire, Yoav Freund, Peter Bartlett, Wee Sun Lee, et al. Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics, 26(5):1651–1686, 1998. find
  262. Jürgen Schmidhuber. Learning factorial codes by predictability minimization. Neural computation, 4(6):863–879, 1992. find
  263. Patrick Schramowski, Wolfgang Stammer, Stefano Teso, Anna Brugger, Franziska Herbert, Xiaoting Shao, Hans-Georg Luigs, Anne-Katrin Mahlein, and Kristian Kersting. Making deep neural networks right for the right scientific reasons by interacting with their explanations. Nature Machine Intelligence, 2… find
  264. Lesia Semenova, Cynthia Rudin, and Ronald Parr. A study in rashomon curves and volumes: A new perspective on generalization and model simplicity in machine learning. arXiv e-print arXiv:1908.01755, 2019. find
  265. Sungyong Seo and Yan Liu. Differentiable Physics-informed Graph Networks. arXiv e-print arXiv:1902.02950, 2019. — [arXiv:1902.02950](https://arxiv.org/abs/1902.02950) find
  266. Tianmin Shu, Caiming Xiong, and Richard Socher. Hierarchical and interpretable skill acquisition in multi-task reinforcement learning. arXiv e-print arXiv:1712.07294, 2017. find
  267. Andrew Silva, Matthew Gombolay, Taylor Killian, Ivan Jimenez, and Sung-Hyun Son. Optimization methods for interpretable differentiable decision trees applied to reinforcement learning. In International Conference on Artificial Intelligence and Statistics, pages 1855–1865. find
  268. David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587):484–489, 2016. find
  269. AJ Six, BE Backus, and JC Kelder. Chest pain in the emergency room: value of the heart score. Netherlands Heart Journal, 16(6):191–196, 2008. find
  270. Marina Sokolova, Mario Marchand, Nathalie Japkowicz, and John S Shawe-taylor. The decision list machine. In Proceedings of Neural Information Processing Systems (NeurIPS), pages 945– 952, 2003. find
  271. Nataliya Sokolovska, Yann Chevaleyre, Karine Clément, and Jean-Daniel Zucker. The fused lasso penalty for learning interpretable medical scoring systems. In 2017 International Joint Conference on Neural Networks (IJCNN), pages 4504–4511. find
  272. Nataliya Sokolovska, Yann Chevaleyre, and Jean-Daniel Zucker. A provable algorithm for learning interpretable scoring systems. In Proceedings of Machine Learning Research Vol. 84: Artificial Intelligence and Statistics (AISTATS), pages 566–574, 2018. find
  273. David Spiegelhalter. Should we trust algorithms? Harvard Data Science Review, 2(1), 2020. find
  274. Nathan Srebro, Karthik Sridharan, and Ambuj Tewari. Smoothness, low noise and fast rates. In Proceedings of Neural Information Processing Systems (NeurIPS), pages 2199–2207, 2010. find
  275. Aaron F. Struck, Berk Ustun, Andres Rodriguez Ruiz, Jong Woo Lee, Suzette M. LaRoche, Lawrence J. Hirsch, Emily J. Gilmore, Jan Vlachy, Hiba Arif Haider, Cynthia Rudin, and M. Brandon Westover. Association of an electroencephalography-based risk score with seizure probability in hospitalized patient… find
  276. Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018. find
  277. Lei Tai, Jingwei Zhang, Ming Liu, Joschka Boedecker, and Wolfram Burgard. A survey of deep network solutions for learning control in robotics: From reinforcement to imitation. arXiv e-print arXiv:1612.07139, 2016. — [arXiv:1612.07139](https://arxiv.org/abs/1612.07139) find
  278. Jian Tang, Jingzhou Liu, Ming Zhang, and Qiaozhu Mei. Visualizing large-scale and high-dimensional data. In Proceedings of the 25th International Conference on the World Wide Web, pages 287–297. International World Wide Web Conferences Steering Committee, 2016. find
  279. Joshua B. Tenenbaum, Vin de Silva, and John C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319–2323, 2000. find
  280. Martin Than, Dylan Flaws, Sharon Sanders, Jenny Doust, Paul Glasziou, Jeffery Kline, Sally Aldous, Richard Troughton, Christopher Reid, William A Parsonage, et al. Development and validation of the emergency department assessment of chest pain score and 2h accelerated diagnostic protocol. Emergency… find
  281. The Smithsonian Institute. Mammuthus primigenius (blumbach). //3d.si.edu/object/3d/mammuthus-primigenius-blumbach: 341c96cd-f967-4540-8ed1-d3fc56d31f12, 2020. https: find
  282. Scott Thiebes, Sebastian Lins, and Ali Sunyaev. Trustworthy artificial intelligence. Electronic Markets, 2020. find
  283. Richard Tomsett, Alun Preece, Dave Braines, Federico Cerutti, Supriyo Chakraborty, Mani Srivastava, Gavin Pearson, and Lance Kaplan. Rapid trust calibration through interpretable and uncertainty-aware AI. Patterns, 1(4):100049, 2020. find
  284. Warren Torgerson. Multidimensional scaling: I theory and method. Psychometrika, 17(4):401– 419, 1952. find
  285. Loc Trinh, Michael Tsang, Sirisha Rambhatla, and Yan Liu. Interpretable and trustworthy deepfake detection via dynamic prototypes. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV, 2021), pages 1973–1983, 2021. find
  286. Berk Ustun and Cynthia Rudin. Supersparse linear integer models for optimized medical scoring systems. Machine Learning, 102(3):349–391, 2016. find
  287. Berk Ustun and Cynthia Rudin. Optimized risk scores. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 1125–1134, 2017. find
  288. Berk Ustun and Cynthia Rudin. Learning optimized risk scores. Journal of Machine Learning Research, 20(150):1–75, 2019. find
  289. Berk Ustun, Stefano Traca, and Cynthia Rudin. Supersparse linear integer models for predictive scoring systems. arXiv e-print arXiv:1306.5860, 2013. find
  290. Berk Ustun, M.B. Westover, Cynthia Rudin, and Matt T. Bianchi. Clinical prediction models for sleep apnea: The importance of medical history over symptoms. Journal of Clinical Sleep Medicine, 12(2):161–168, 2016. find
  291. Laurens van der Maaten. Accelerating t-SNE using tree-based algorithms. Journal of Machine Learning Research, 15:3221–3245, 2014. find
  292. Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, 9:2579–2605, 2008. find
  293. Koen Vanhoof and Benoı̂t Depaire. Structure of association rule classifiers: a review. In 2010 IEEE International Conference on Intelligent Systems and Knowledge Engineering (ISKE), pages 9– 12. find
  294. VN Vapnik and A Ya Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications, 16(2):264, 1971. find
  295. Abhinav Verma, Vijayaraghavan Murali, Rishabh Singh, Pushmeet Kohli, and Swarat Chaudhuri. Programmatically interpretable reinforcement learning. arXiv e-print arXiv:1804.02477, 2018. — [arXiv:1804.02477](https://arxiv.org/abs/1804.02477) find
  296. Sicco Verwer and Yingqian Zhang. Learning optimal classification trees using a binary linear program formulation. In Proceedings of AAAI Conference on Artificial Intelligence (AAAI), 2019. find
  297. Matheus Guedes Vilas Boas, Haroldo Gambini Santos, Luiz Henrique de Campos Merschmann, and Greet Vanden Berghe. Optimal decision trees for the algorithm selection problem: integer programming based approaches. International Transactions in Operational Research, 2019. find
  298. Sandra Wachter, Brent Mittelstadt, and Luciano Floridi. Why a right to explanation of automated decision-making does not exist in the general data protection regulation. International Data Privacy Law, 12 2017. find
  299. Kiri L. Wagstaff. Machine learning that matters. In Proceedings of International Conference on Machine Learning (ICML), pages 529–536, 2012. find
  300. Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The Caltech-UCSD Birds-200-2011 Dataset. Technical Report CNS-TR-2011-001, California Institute of Technology, 2011. find
  301. Fulton Wang and Cynthia Rudin. Falling rule lists. In Proceedings of Artificial Intelligence and Statistics (AISTATS), pages 1013–1022, 2015a. find
  302. Jiaxuan Wang, Jeeheh Oh, Haozhu Wang, and Jenna Wiens. Learning credible models. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2018. find
  303. Sifan Wang, Yujun Teng, and Paris Perdikaris. Understanding and mitigating gradient pathologies in physics-informed neural networks. arXiv e-print arXiv:2001.04536, 2020a. — [arXiv:2001.04536](https://arxiv.org/abs/2001.04536) find
  304. Tianyu Wang, Marco Morucci, M. Usaid Awan, Yameng Liu, Sudeepa Roy, Cynthia Rudin, and Alexander Volfovsky. Flame: A fast large-scale almost matching exactly approach to causal inference. Journal of Machine Learning Research, 22(31):1–41, 2021a. find
  305. Tong Wang. Gaining free or low-cost interpretability with interpretable partial substitute. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 6505–6514. PMLR, 09–… find
  306. Tong Wang and Cynthia Rudin. arXiv:1511.02210, 2015b. find
  307. Learning optimized Or’s of And’s. arXiv e-print Tong Wang, Jingyi Yang, Yunyi Li, and Boxiang Wang. Partially interpretable estimators (PIE): black-box-refined interpretable machine learning. CoRR arXiv 2105.02410, abs/2105.02410, 2021b. URL https://arxiv.org/abs/2105.02410. — [arXiv:2105.02410](https://arxiv.org/abs/2105.02410) ↗
  308. Yingfan Wang, Haiyang Huang, Cynthia Rudin, and Yaron Shaposhnik. Understanding how dimension reduction tools work: An empirical approach to deciphering t-SNE, UMAP, TriMap, and PaCMAP for data visualization, 2020b. find
  309. Martin Wattenberg, Fernanda Viégas, and Ian Johnson. How to use t-SNE effectively. Distill, 1 (10):e2, 2016. find
  310. Frank W Weathers, Brett T Litz, Terence M Keane, Patrick A Palmieri, Brian P Marx, and Paula P Schnurr. The PTSD checklist for DSM-5 (PCL-5) – standard [measurement instrument]. Scale available from the National Center for PTSD at www. ptsd. va. gov, 10, 2013. find
  311. Kilian Q Weinberger and Lawrence K Saul. Distance Metric Learning for Large Margin Nearest Neighbor Classification. Journal of Machine Learning Research, 10(2), 2009. find
  312. Rebecca Wexler. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times, June 13 2017. find
  313. Simon N Wood. Generalized additive models: an introduction with R. CRC press, 2017. find
  314. Chenyue Wu and Esteban G Tabak. Prototypal Analysis and Prototypal Regression. arXiv e-print arXiv:1701.08916, 2017. — [arXiv:1701.08916](https://arxiv.org/abs/1701.08916) find
  315. Feng Xie, Bibhas Chakraborty, Marcus Eng Hock Ong, Benjamin Alan Goldstein, and Nan Liu. Autoscore: A machine learning–based automatic clinical score generator and its application to mortality prediction using electronic health records. JMIR Med Inform, 8(10):e21798, October 2020. find
  316. Hongyu Yang, Cynthia Rudin, and Margo Seltzer. Scalable bayesian rule lists. In Proceedings of International Conference on Machine Learning (ICML), pages 3921–3930. PMLR, 2017. find
  317. Yibo Yang and Paris Perdikaris. Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems. Computational Mechanics, 64(2):417–434, 2019. find
  318. Yongxin Yang, Irene Garcia Morillo, and Timothy M Hospedales. Deep neural decision trees. arXiv preprint arXiv:1806.06988, 2018. — [arXiv:1806.06988](https://arxiv.org/abs/1806.06988) find
  319. Junming Yin and Yaoliang Yu. Convex-constrained sparse additive modeling and its extensions. arXiv e-print arXiv:1705.00687, 2017. — [arXiv:1705.00687](https://arxiv.org/abs/1705.00687) find
  320. Xiaoxin Yin and Jiawei Han. Cpar: Classification based on predictive association rules. In Proceedings of the 2003 SIAM International Conference on Data Mining (SDM), pages 331– 335. find
  321. Chao Yu, Jiming Liu, and Shamim Nemati. Reinforcement learning in healthcare: A survey. arXiv e-print arXiv:1908.08796, 2019. — [arXiv:1908.08796](https://arxiv.org/abs/1908.08796) find
  322. Jinqiang Yu, Alexey Ignatiev, Pierre Le Bodic, and Peter J Stuckey. Optimal decision lists using sat. arXiv e-print arXiv:2010.09919, 2020a. — [arXiv:2010.09919](https://arxiv.org/abs/2010.09919) find
  323. Jinqiang Yu, Alexey Ignatiev, Peter J Stuckey, and Pierre Le Bodic. Computing optimal decision sets with sat. In International Conference on Principles and Practice of Constraint Programming, pages 952–970. Springer, 2020b. find
  324. Tom Zahavy, Nir Ben-Zrihem, and Shie Mannor. Graying the black box: Understanding DQNs. In International Conference on Machine Learning, pages 1899–1908. find
  325. Vinicius Zambaldi, David Raposo, Adam Santoro, Victor Bapst, Yujia Li, Igor Babuschkin, Karl Tuyls, David Reichert, Timothy Lillicrap, Edward Lockhart, et al. Relational deep reinforcement learning. arXiv e-print arXiv:1806.01830, 2018. find
  326. John R. Zech et al. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. PLoS Med., 15(e1002683), 2018. find
  327. Kristine Zhang, Yuanheng Wang, Jianzhun Du, Brian Chu, Leo Anthony Celi, Ryan Kindle, and Finale Doshi-Velez. Identifying decision points for safe and interpretable reinforcement learning in hypotension treatment. arXiv e-print arXiv:2101.03309, 2021. — [arXiv:2101.03309](https://arxiv.org/abs/2101.03309) find
  328. Quanshi Zhang, Ying Nian Wu, and Song-Chun Zhu. Interpretable convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 8827–8836, 2018. find
  329. Yujia Zhang, Kuangyan Song, Yiming Sun, Sarah Tan, and Madeleine Udell. “Why should you trust my explanation?” Understanding uncertainty in LIME explanations. In Proceedings of the International Conference on Machine Learning AI for Social Good Workshop, 2019. find
  330. Hongxu Zhao, Ran Jin, Su Wu, and Jianjun Shi. PDE-constrained gaussian process model on material removal rate of wire saw slicing process. Journal of Manufacturing Science and Engineering, 133(2), 2011. find
  331. Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Object detectors emerge in deep scene CNNs. In Proceedings of the International Conference on Learning Representations (ICLR), 2014. find
  332. Bolei Zhou, David Bau, Aude Oliva, and Antonio Torralba. Interpreting deep visual representations via network dissection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018a. find
  333. Bolei Zhou, Yiyou Sun, David Bau, and Antonio Torralba. Interpretable basis decomposition for visual explanation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 119–134, 2018b. find
  334. Ding-Xuan Zhou. The covering number in learning theory. Journal of Complexity, 18(3):739–767, 2002. find
  335. Yinhao Zhu, Nicholas Zabaras, Phaedon-Stelios Koutsourelakis, and Paris Perdikaris. Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data. Journal of Computational Physics, 394:56–81, 2019. find
  336. Zhenyao Zhu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Multi-view perceptron: a deep model for learning face identity and view representations. In Proceedings of Neural Information Processing Systems (NeurIPS), pages 217–225, 2014. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rudin-2021-interpretable-machine-learning-fundamental/