Elamur Research⌕
☰
home / papers / 2021 / Explanation-Based Human Debugging of NLP Models: A Survey

Explanation-Based Human Debugging of NLP Models: A Survey

Paper · 2021 · Piyawat Lertvittayakumjorn and Francesca Toni

Summary

The survey examines explanation-based human debugging (EBHD), meaning the use of model explanations so that people can find and repair defects in trained NLP models. It defines a bug as a contamination of the training or prediction pipeline that leads to wrong outputs or misleading learned associations. Examples include spurious correlations, mislabeled data and poor out-of-distribution behaviour. The authors propose a three-stage framework: show explanations to humans, gather their feedback, and use that feedback to update the model, either once or repeatedly. They chose 15 studies through citation tracing and a systematic Semantic Scholar search. Each study is categorised by debugging context (task, model, bug source), workflow (explanation scope and method, feedback form, update type) and experimental setting. The survey also collects findings on human factors that affect feedback quality and user satisfaction. The authors conclude that the field is still young, focuses mostly on English text classification, and lacks common evaluation standards. This makes the survey a useful map for anyone designing interactive model-repair tools.

Key points

  • Literature search: keyword sets were combined into 16 queries, with the top 100 hits taken from each. Papers appearing in at least 2 result lists were kept, giving 234 for manual review and a final set of 15 core studies.
  • Coverage of the 15 studies:
    • Tasks are mainly single-input text classification, plus VQA, TQA and one NLI study (Zylberajch et al., 2021).
    • Models range from NB, SVM and LR to CNN, BERT and RoBERTa.
    • Feedback takes several forms, including labels, words, word scores, rules, attention and reasoning.
  • Illustrative case (Ribeiro et al., 2016):
    • The model was an SVM trained on 20Newsgroups.
    • Amazon Mechanical Turk workers reviewed LIME word-relevance scores and flagged artifact words.
    • The flagged words were removed and the model retrained over three rounds, generalising better after each round.
  • Main gaps:
    • Every reviewed study uses English data only.
    • Non-classification tasks are rare. NLI bugs come from syntactic heuristics, so word-level feedback may not be enough there.
  • Proposed directions:
    • A shared benchmark for fair comparison across papers.
    • Integration with tools such as the Language Interpretability Tool (LIT).
    • Evaluation with real end users rather than only simulation or crowdsourcing.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Imperial College LondonLondon · United Kingdom48 works · 12,181 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSoftware · 92 works

Cite

Cite as Piyawat Lertvittayakumjorn and Francesca Toni. 2021. Explanation-Based Human Debugging of NLP Models: A Survey. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00440

@article{lertvittayakumjorn2021explanation,
  title = {{Explanation-Based Human Debugging of NLP Models: A Survey}},
  author = {Piyawat Lertvittayakumjorn and Francesca Toni},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.1162/tacl\_a\_00440},
  url = {https://doi.org/10.1162/tacl_a_00440}
}

References

103 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Workshop on Representation Learning for NLP, pages 1–7, Berlin, Germany. Association for Computational Linguistics. https://doi.org/10.18653/v1/W16-1601 ↗
  2. Yanzhe Bekkemoen and Helge Langseth. 2021. Correcting classification: A Bayesian framework using explanation feedback to improve classification abilities. arXiv preprint arXiv: 2105.02653. — [arXiv:2105.02653](https://arxiv.org/abs/2105.02653) find
  3. Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yunhan Jia, Joydeep Ghosh, Ruchir Puri, José MF Moura, and Peter Eckersley. 2020. Explainable machine learning in deployment. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 648–657. http… — [doi:10.1145/3351095](https://doi.org/10.1145/3351095) find
  4. Gabriel Cadamuro, Ran Gilad-Bachrach, and Xiaojin Zhu. 2016. Debugging machine learning models. In ICML Workshop on Reliable Machine Learning in the Wild. find
  5. Maya Cakmak, Crystal Chao, and Andrea L. Thomaz. 2010. Designing interactions for robot active learners. IEEE Transactions on Autonomous Mental Development, 2(2):108–118. https://doi.org/10.1109/TAMD.2010 .2051030 — [doi:10.1109/tamd.2010](https://doi.org/10.1109/tamd.2010) ↗
  6. Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. E-SNLI: Natural language inference with natural language explanations. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pages 9560–9572.
  7. Lucas Carstens and Francesca Toni. 2017. Using argumentation to improve classification in natural language problems. ACM Transactions on Internet Technology (TOIT), 17(3):1–23. https://doi.org/10.1145/3017679 — [doi:10.1145/3017679](https://doi.org/10.1145/3017679) ↗
  8. Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. 2015. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day Gromit Yeuk-Yin Chan, Jun Yuan, Kyle Overton, Brian Barr, Kim Rees, Luis Gustavo Nonato, Enrico Bertini, and Claudio T. Silva. 20… — [arXiv:2007.10609](https://arxiv.org/abs/2007.10609) find
  9. Hao-Fei Cheng, Ruotong Wang, Zheng Zhang, Fiona O’Connell, Terrance Gray, F. Maxwell Harper, and Haiyi Zhu. 2019. Explaining decision-making algorithms through UI: Strategies to help non-expert stakeholders. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pages 1–12.… — [doi:10.1145/3290605](https://doi.org/10.1145/3290605) find
  10. Minseok Cho, Reinald Kim Amplayo, Seung-won Hwang, and Jonghyuck Park. 2018. Adversarial tableqa: Attention supervision for question answering on tables. In Proceedings of The 10th Asian Conference on Machine Learning, volume 95 of Proceedings of Machine Learning Research, pages 391–406. PMLR. find
  11. Minseok Cho, Gyeongbok Lee, and Seung-won Hwang. 2019. Explanatory and actionable debugging for machine learning: A tableqa demonstration. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1333–1336. find
  12. Henriette Cramer, Vanessa Evers, Satyan Ramlal, Maarten Van Someren, Lloyd Rutledge, Natalia Stash, Lora Aroyo, and Bob Wielinga. 2008. The effects of transparency on trust in and acceptance of a content-based art recommender. User Modeling and User-Adapted Interaction, 18(5):455. https://doi.org /1… — [doi:10.1007/s11257-008-9051-3](https://doi.org/10.1007/s11257-008-9051-3) ↗
  13. Marina Danilevsky, Kun Qian, Ranit Aharonov, Yannis Katsis, Ban Kawas, and Prithviraj Sen. 2020. A survey of the state of explainable AI for natural language processing. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Downloaded from http://direc…
  14. Emily Bender. 2019. The \#benderrule: On naming the languages we study and why it matters. The Gradient. readmission. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1721–1730. https://doi.org/10 .1145/2783258.2788613 find
  15. Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 447–459, Suzhou, China. Association for Computational Linguistics. find
  16. Adam Dejl, Peter He, Pranav Mangal, Hasan Mohsin, Bogdan Surdu, Eduard Voinea, Emanuele Albini, Piyawat Lertvittayakumjorn, Antonio Rago, and Francesca Toni. 2021. Argflow: A toolkit for deep argumentative explanations for neural networks. In Proceedings of the 20th International Conference on Auton… find
  17. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  18. Serge Egelman, Ed H. Chi, and Steven Dow. 2014. Crowdsourcing in HCI research. In Ways of Knowing in HCI, Springer, pages 267–289. find
  19. Rebecca Fiebrink, Dan Trueman, and Perry R. Cook. 2009. A metainstrument for interactive, on-the-fly machine learning. In Proceedings of NIME. find
  20. Nahum Gershon. 1998. Visualization of an imperfect world. IEEE Computer Graphics and Applications, 18(4):43–45. https://doi.org/10.1109/38.689662 — [doi:10.1109/38.689662](https://doi.org/10.1109/38.689662) ↗
  21. Bhavya Ghai, Q. Vera Liao, Yunfeng Zhang, Rachel Bellamy, and Klaus Mueller. 2021. find
  22. Filip Graliński, Anna Wróblewska, Tomasz Stanisławek, Kamil Grabowski, and Tomasz Górecki. 2019. GEval: Tool for debugging NLP datasets and models. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 254–262, Florence, Italy. Association… find
  23. Andrew Guillory and Jeff Bilmes. 2011. Simultaneous learning and covering with adversarial noise. In Proceedings of the 28th International Conference on International Conference on Machine Learning, ICML’11, pages 369–376, Madison, WI, USA. Omnipress. find
  24. Han Guo, Nazneen Fatema Rajani, Peter Hase, Mohit Bansal, and Caiming Xiong. 2020. Fastif: Scalable influence functions for efficient model interpretation and debugging. arXiv preprint arXiv:2012.15781. find
  25. Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018. Annotation artifacts in natural language inference data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techno…
  26. Xiaochuang Han, Byron C. Wallace, and Yulia Tsvetkov. 2020. Explaining black box predictions and unveiling data artifacts through influence functions. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5553–5563, Online. Association for Computational Li…
  27. Xing Han and Joydeep Ghosh. 2020. Model-agnostic explanations using minimal forcing subsets. arXiv preprint arXiv:2011.00639. — [arXiv:2011.00639](https://arxiv.org/abs/2011.00639) find
  28. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, Downloaded from http://direct.mit.edu/tacl/article-pdf/doi/10.1162/tacl_a_00440/1983435/tacl_a_00440.pdf by guest on 18 February 2022 — [doi:10.1162/tacl_a_00440/1983435/tacl_a_00440.pdf](https://doi.org/10.1162/tacl_a_00440/1983435/tacl_a_00440.pdf) ↗
  29. Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. 2019. Bias in bios: A case study of semantic representation bias in a high-stakes setting. In Proceedings of the Conference on Fairnes… — [doi:10.1145/3287560](https://doi.org/10.1145/3287560) find
  30. Explainable active learning (xal) toward ai explanations as interfaces for machine teachers. Proceedings of the ACM on Human-Computer Interaction, 4(CSCW3):1–28. https://doi.org/10.1145/3432934 9(8):1735–1780. https://doi.org/10 .1162/neco.1997.9.8.1735, PubMed: 9377276 — [doi:10.1145/3432934](https://doi.org/10.1145/3432934) ↗
  31. Donald Honeycutt, Mahsan Nourani, and Eric Ragan. 2020. Soliciting human-in-the-loop user feedback for interactive machine learning reduces user trust and impressions of model accuracy. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, volume 8, pages 63–72. find
  32. Maximilian Idahl, Lijun Lyu, Ujwal Gadiraju, and Avishek Anand. 2021. Towards benchmarking the utility of explanations for model debugging. arXiv preprint arXiv:2105.04505. https://doi.org/10.18653/v1/2021 .trustnlp-1.8 ↗
  33. Alon Jacovi, Ana Marasović, Tim Miller, and Yoav Goldberg. 2020. Formalizing trust in artificial intelligence: Prerequisites, causes and goals of human trust in AI. arXiv preprint arXiv:2010.07487. https://doi.org/10 .1145/3442188.3445923 — [arXiv:2010.07487](https://arxiv.org/abs/2010.07487) find
  34. Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2021–2031, Copenhagen, Denmark. Association for Computational Linguistics. https://doi.org/10.18653 /v1…
  35. Xisen Jin, Zhongyu Wei, Junyi Du, Xiangyang Xue, and Xiang Ren. 2020. Towards hierarchical importance attribution: Explaining compositional semantics for neural sequence models. In International Conference on Learning Representations. find
  36. David Johnson, Giuseppe Carenini, and Gabriel Murray. 2020. Njm-vis: Interpreting neural Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. Bag of tricks for efficient text classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Comp… find
  37. Patrick Juola. 2007. Future trends in authorship attribution. In IFIP International Conference on Digital Forensics, pages 119–132. Springer. https://doi.org/10.1007/978-0-387 -73742-3 8 — [doi:10.1007/978-0-387](https://doi.org/10.1007/978-0-387) ↗
  38. Daniel Kang, Deepti Raghavan, Peter Bailis, and Matei Zaharia. 2018. Model assertions for debugging machine learning. In NeurIPS MLSys Workshop. find
  39. Harmanpreet Kaur, Harsha Nori, Samuel Jenkins, Rich Caruana, Hanna Wallach, and Jennifer Wortman Vaughan. 2020. Interpreting interpretability: Understanding data scientists’ use of interpretability tools for machine learning. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Sy… find
  40. Rajiv Khanna, Been Kim, Joydeep Ghosh, and Sanmi Koyejo. 2019. Interpreting black box predictions using fisher kernels. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 3382–3390. PMLR. find
  41. Sung Wook Kim, Iljeok Kim, Jonghwan Lee, and Seungchul Lee. 2021. Knowledge integration into deep learning in dynamical systems: An overview and taxonomy. Journal of Mechanical Science and Technology, pages 1–12. find
  42. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Downloaded from http://direct.mit.edu/tacl/article-pdf/doi/10.1162/tacl_a_00440/1983435/tacl_a_00440.pdf by guest on 18 Februa… ↗
  43. Benjamin Hoover, Hendrik Strobelt, and Sebastian Gehrmann. 2020. exBERT: A visual analysis tool to explore learned representations in transformer models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 187–196, Online. Associat… find
  44. Pang Wei Koh and Percy Liang. 2017. Understanding black-box predictions via influence functions. In International Conference on Machine Learning, pages 1885–1894. PMLR.
  45. Sanjay Krishnan and Eugene Wu. 2017. Palm: Machine learning explanations for iterative debugging. In Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics, pages 1–6. https://doi.org/10.1145/3077257.3077271 ↗
  46. Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015. Principles of explanatory debugging to personalize interactive machine learning. In Proceedings of the 20th International Conference on Intelligent User Interfaces, pages 126–137. https://doi.org/10.1145/2678025 .2701399 — [doi:10.1145/2678025](https://doi.org/10.1145/2678025) ↗
  47. Todd Kulesza, Simone Stumpf, Margaret Burnett, Weng-Keen Wong, Yann Riche, Travis Moore, Ian Oberst, Amber Shinsel, and Kevin McIntosh. 2010. Explanatory debugging: Supporting end-user debugging of machine-learned programs. In 2010 IEEE Symposium on Visual Languages and Human-Centric Computing, page… find
  48. Todd Kulesza, Weng-Keen Wong, Simone Stumpf, Stephen Perona, Rachel White, Margaret M. Burnett, Ian Oberst, and Andrew J. Ko. 2009. Fixing the program my computer learned: Barriers for end users, challenges for the machine. In Proceedings of the 14th International Conference on Intelligent User Inte… — [doi:10.1145/1502650.1502678](https://doi.org/10.1145/1502650.1502678) find
  49. Vivian Lai and Chenhao Tan. 2019. On human predictions with explanations and predictions of machine learning models: A case study on deception detection. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 29–38. find
  50. Himabindu Lakkaraju, Julius Adebayo, and Sameer Singh. 2020. Explaining machine learning predictions: State-of-the-art, challenges, and opportunities. find
  51. NeurIPS 2020 Tutorial. find
  52. Ken Lang. 1995. Newsweeder: Learning to filter netnews. In Proceedings of the Twelfth International Conference on Machine Learning, pages 331–339. https://doi.org/10.1016/B978-1-55860-377-6.50048-7 ↗
  53. Piyawat Lertvittayakumjorn, Ivan Petej, Yang Gao, Yamuna Krishnamurthy, Anna Van Der Gaag, Robert Jago, and Kostas Stathis. 2021. Supporting complaints investigation for nursing and midwifery regulatory agencies. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguist… find
  54. Piyawat Lertvittayakumjorn, Lucia Specia, and Francesca Toni. 2020. FIND: Human-in-the-loop debugging deep text classifiers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 332–348, Online. Association for Computational Linguistics. https://do… find
  55. Piyawat Lertvittayakumjorn and Francesca Toni. 2019. Human-grounded evaluations of explanation methods for text classification. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-I… find
  56. Josua Krause, Adam Perer, and Kenney Ng. 2016. Interacting with predictions: Visual inspection of black-box machine learning models. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pages 5686–5697. https://doi.org/10 .1145/2858036.2858529 find
  57. Vivian Lai, Han Liu, and Chenhao Tan. 2020. ‘‘why is’ chicago’deceptive?’’ towards building model-driven tutorials for humans. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pages 1–13. pages 5195–5205, Hong Kong, China. Association for Computational Linguistics. ht… find
  58. Brian Y. Lim, Anind K. Dey, and Daniel Avrahami. 2009. Why and why not explanations improve the intelligibility of context-aware intelligent systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 2119–2128. find
  59. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692.
  60. Raoni Lourenço, Juliana Freire, and Dennis Shasha. 2020. Bugdoc: A system for debugging computational pipelines. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, pages 2733–2736. https://doi.org/10.1145/3318464 .3384692 — [doi:10.1145/3318464](https://doi.org/10.1145/3318464) ↗
  61. Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30:4765–4774.
  62. Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3428–3448, Florence, Italy. Association for Computational Ling…
  63. Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence, 267:1–38. https://doi.org/10.18653/v1/P19-1334 ↗
  64. Yao Ming, Panpan Xu, Huamin Qu, and Liu Ren. 2019. Interpretable and steerable sequence learning via prototypes. In Proceedings of the Brad A. Myers, David A. Weitzman, Andrew J. Ko, and Duen H. Chau. 2006. Answering why and why not questions in user interfaces. In Proceedings of the SIGCHI Conferen… find
  65. Menaka Narayanan, Emily Chen, Jeffrey He, Been Kim, Sam Gershman, and Finale Doshi-Velez. 2018. How do humans understand explanations from machine learning systems? An evaluation of the human-interpretability of explanation. arXiv preprint arXiv:1802.00682. — [arXiv:1802.00682](https://arxiv.org/abs/1802.00682) find
  66. Devi Parikh and C. Zitnick. 2011. Human-debugging of machines. NIPS WCSSWC, 2(7):3. https://doi.org/10.18653/v1/D18-1302 ↗
  67. Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. Reducing gender bias in abusive language detection. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2799–2804, Brussels, Belgium. Association for Computational Linguistics.
  68. Teodora Popordanoska, Mohit Kumar, and Stefano Teso. 2020. Machine guides, human supervises: Interactive learning with global explanations. arXiv preprint arXiv:2009.09723. — [arXiv:2009.09723](https://arxiv.org/abs/2009.09723) find
  69. Forough Poursabzi-Sangdeh, Daniel G. Goldstein, Jake M. Hofman, Jennifer Wortman Vaughan, and Hanna Wallach. 2018. Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810. — [arXiv:1802.07810](https://arxiv.org/abs/1802.07810) find
  70. Pearl Pu and Li Chen. 2006. Trust building with explanation interfaces. In Proceedings of the 11th international conference on Intelligent user interfaces, pages 93–100. find
  71. Xipeng Qiu, Tianxiang Sun, Yige Xu, Yunfan Shao, Ning Dai, and Xuanjing Huang. 2020. Pre-trained models for natural language processing: A survey. Science China Technological Sciences, pages 1–26.
  72. Downloaded from http://direct.mit.edu/tacl/article-pdf/doi/10.1162/tacl_a_00440/1983435/tacl_a_00440.pdf by guest on 18 February 2022 — [doi:10.1162/tacl_a_00440/1983435/tacl_a_00440.pdf](https://doi.org/10.1162/tacl_a_00440/1983435/tacl_a_00440.pdf) ↗
  73. Zachary C. Lipton. 2018. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue, 16(3):31–57. https://doi.org/10.1145/3236386.3241340 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 903–91… — [doi:10.1145/3236386.3241340](https://doi.org/10.1145/3236386.3241340) ↗
  74. Marco Tulio Ribeiro, Carlos Guestrin, and Sameer Singh. 2019. Are red roses red? Evaluating consistency of question-answering models. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6174–6184, Florence, Italy. Association for Computational Linguistic… find
  75. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018a. Anchors: High-precision model-agnostic explanations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32. find
  76. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018b. Semantically equivalent adversarial rules for debugging NLP models. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 856–865, Melbourne, Australia. Association for… find
  77. Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020. Beyond accuracy: Behavioral testing of NLP models with CheckList. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4902–4912, Online. Association for Computational Linguistic… find
  78. Laura Rieger, Chandan Singh, William Murdoch, and Bin Yu. 2020. Interpretations are useful: Penalizing explanations to align neural networks with prior knowledge. In International Conference on Machine Learning, pages 8116–8126. PMLR. find
  79. Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2021. A primer in bertology: What we know about how BERT works. Transactions of the Association for Computational Cynthia Rudin. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat…
  80. Eldon Schoop, Forrest Huang, and Björn Hartmann. 2020. Scram: Simple checks for realtime analysis of model training for non-expert ml programmers. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, pages 1–10. https://doi.org/10.1145/3334480 .3382879 — [doi:10.1145/3334480](https://doi.org/10.1145/3334480) ↗
  81. Patrick Schramowski, Wolfgang Stammer, Stefano Teso, Anna Brugger, Franziska Herbert, Xiaoting Shao, Hans-Georg Luigs, Anne-Katrin Mahlein, and Kristian Kersting. 2020. Making deep neural networks right for the right scientific reasons by interacting with their explanations. Nature Machine Intellige… — [doi:10.1038/s42256-020-0212-3](https://doi.org/10.1038/s42256-020-0212-3) find
  82. Daniel Selsam, Percy Liang, and David L. Dill. 2017. Developing bug-free machine learning systems with formal mathematics. In International Conference on Machine Learning, pages 3047–3056. PMLR. find
  83. Xiaoting Shao, Tjitze Rienstra, Matthias Thimm, and Kristian Kersting. 2020. Towards understanding and arguing with classifiers: Recent progress. Datenbank-Spektrum, 20(2):171–180. https://doi.org/10.1007/s13222-020 -00351-x Downloaded from http://direct.mit.edu/tacl/article-pdf/doi/10.1162/tacl_a_0… — [doi:10.1007/s13222-020](https://doi.org/10.1007/s13222-020) ↗
  84. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. ‘‘why should i trust you?’’ explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1135–1144. https://doi.org/10.1145/2939672 .2939778 ↗
  85. Linguistics, 8:842–866. https://doi.org/10.1162/tacl_a_00349 Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. 2017. Smoothgrad: Removing noise by adding noise. arXiv preprint arXiv:1706.03825. ↗
  86. Simone Stumpf, Adrian Bussone, and Dympna O’sullivan. 2016. Explanations considered harmful? User interactions with machine learning systems. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI). find
  87. Simone Stumpf, Vidya Rajaram, Lida Li, Weng-Keen Wong, Margaret Burnett, Thomas Dietterich, Erin Sullivan, and Jonathan Herlocker. 2009. Interacting meaningfully with machine learning systems: Three experiments. International Journal of Human-Computer Studies, 67(8):639–662. https://doi.org /10.1016… — [doi:10.1016/j.ijhcs.2009.03.004](https://doi.org/10.1016/j.ijhcs.2009.03.004) ↗
  88. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In International Conference on Machine Learning, pages 3319–3328. PMLR. find
  89. Ian Tenney, James Wexler, Jasmijn Bastings, Tolga Bolukbasi, Andy Coenen, Sebastian Gehrmann, Ellen Jiang, Mahima Pushkarna, Carey Radebaugh, Emily Reif, and Ann Yuan. 2020. The language interpretability tool: Extensible, interactive visualizations and analysis for NLP models. https://doi.org/10 .18… find
  90. Stefano Teso and Kristian Kersting. 2019. Explanatory interactive machine learning. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pages 239–245. https://doi.org/10.1145/3306618 .3314293 — [doi:10.1145/3306618](https://doi.org/10.1145/3306618) ↗
  91. Ehsan Toreini, Mhairi Aitken, Kovila Coopamootoo, Karen Elliott, Carlos Gonzalez Zijie J. Wang, Dongjin Choi, Shenyu Xu, and Diyi Yang. 2021. Putting humans in the natural language processing loop: A survey. arXiv preprint arXiv:2103.04044. find
  92. Thomas Wolf, Quentin Lhoest, Patrick von Platen, Yacine Jernite, Mariama Drame, Julien Plu, Julien Chaumond, Clement Delangue, Clara Ma, Abhishek Thakur, Suraj Patil, Joe Davison, Teven Le Scao, Victor Sanh, Canwen Xu, Nicolas Patry, Angie McMillan-Major, Simon Brandeis, Sylvain Gugger, François La… find
  93. Tongshuang Wu, Daniel S. Weld, and Jeffrey Heer. 2019. Local decision pitfalls in interactive machine learning: An investigation into feature selection in sentiment analysis. ACM Transactions on Computer-Human Interaction (TOCHI), 26(4):1–27. https://doi.org/10.1145/3319616 — [doi:10.1145/3319616](https://doi.org/10.1145/3319616) ↗
  94. Yao, Huihan, Chen, Ying, Ye, Qinyuan, Jin, Xisen, and Ren, Xiang. 2021. Refining Language Models with Compositional Explanations. Advances in Neural Information Processing Systems, 34. find
  95. Roozbeh Yousefzadeh and Dianne P. O’Leary. 2019. Debugging trained machine learning models using flip points. In ICLR 2019 Debugging Machine Learning Models Workshop. find
  96. Wei Emma Zhang, Quan Z. Sheng, Ahoud Alhazmi, and Chenliang Li. 2020a. Adversarial attacks on deep-learning models in natural language processing: A survey. ACM Transactions on Intelligent Systems and Technology (TIST), 11(3):1–41. https://doi.org/10.1145/3374217 ↗
  97. Downloaded from http://direct.mit.edu/tacl/article-pdf/doi/10.1162/tacl_a_00440/1983435/tacl_a_00440.pdf by guest on 18 February 2022 — [doi:10.1162/tacl_a_00440/1983435/tacl_a_00440.pdf](https://doi.org/10.1162/tacl_a_00440/1983435/tacl_a_00440.pdf) ↗
  98. Alison Smith-Renner, Ron Fan, Melissa Birchfield, Tongshuang Wu, Jordan Boyd-Graber, Daniel S. Weld, and Leah Findlater. 2020. No explainability without accountability: An empirical study of explanations and feedback in interactive ml. In Proceedings of the 2020 CHI Conference on Human Factors in Co… — [doi:10.1145/3313831.3376624](https://doi.org/10.1145/3313831.3376624) find
  99. Zelaya, and Aad van Moorsel. 2020. The relationship between trust in AI and trustworthy machine learning technologies. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 272–283. https://doi.org/10.1145/3351095 .3372834 — [doi:10.1145/3351095](https://doi.org/10.1145/3351095) ↗
  100. Yunfeng Zhang, Q. Vera Liao, and Rachel K. E. Bellamy. 2020b. Effect of confidence and explanation on accuracy and trust calibration in ai-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 295–305. https://doi.org/10.1145/3351095 .33… — [doi:10.1145/3351095](https://doi.org/10.1145/3351095) ↗
  101. Yuke Zhu, Oliver Groth, Michael Bernstein, and Li Fei-Fei. 2016. Visual7w: Grounded question answering in images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4995–5004. https://doi.org/10.1109/CVPR.2016 .540 ↗
  102. Hugo Zylberajch, Piyawat Lertvittayakumjorn, and Francesca Toni. 2021. HILDIF: Interactive debugging of NLI models using influence functions. In Proceedings of the First Workshop on Interactive Learning for Natural Language Processing, pages 1–6, Online. Association for Computational Linguistics. ht… find
  103. Downloaded from http://direct.mit.edu/tacl/article-pdf/doi/10.1162/tacl_a_00440/1983435/tacl_a_00440.pdf by guest on 18 February 2022 — [doi:10.1162/tacl_a_00440/1983435/tacl_a_00440.pdf](https://doi.org/10.1162/tacl_a_00440/1983435/tacl_a_00440.pdf) ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lertvittayakumjorn-2021-explanation-based-human-debugging-nlp/