Elamur Research⌕
☰
home / papers / 2020 / What Does My QA Model Know? Devising Controlled Probes Using Expert Knowledge

What Does My QA Model Know? Devising Controlled Probes Using Expert Knowledge

Paper · 2020 · Kyle Richardson and Ashish Sabharwal

Summary

Do multiple-choice QA (MCQA) models that score well on grade-school science benchmarks actually have the basic knowledge those benchmarks assume? Richardson and Sabharwal address this by automatically turning expert-curated resources into diagnostic question sets. From WordNet they build WordNetQA, and from public dictionaries they build DictionaryQA. Together these test definitions, hypernymy, hyponymy, synonymy and word sense disambiguation, with controlled variations such as hop distance and distractor difficulty. Two safeguards check probe quality. First, models without contextual embeddings should struggle on the probes. Second, lightly fine-tuning a strong model on probe data ("inoculation") should barely hurt its accuracy on the original task. Transformer QA models (BERT, RoBERTa) handle much of this knowledge even zero-shot and adapt quickly from small probe samples. However, they still fall short of human estimates. Their accuracy weakens as taxonomic chains get longer or distractors get harder. It also drops considerably when they must answer every question about a single concept correctly. The work offers a cheap, scalable complement to crowdsourced benchmarks for auditing what QA models really know.

Key points

  • New silver-standard probe datasets: WordNetQA (from WordNet) and DictionaryQA (from dictionaries). Data and code are at github.com/allenai/semantic_fragments.
  • In the zero-shot setting, a RoBERTa QA model trained only on science benchmarks beats a task-specific LSTM trained directly on the probes by about 26%. After inoculation with as few as 100 probe examples, it masters many probes with almost no loss on the original task.
  • The best models remain 2–10% (absolute) below conservative human performance estimates.
  • Hyponym accuracy falls by 8–15% when moving from 1-hop to 2-hop links. Cluster-level evaluation, which requires all questions about one concept to be correct, cuts WordNetQA accuracy by 14–44%.
  • BERT- and RoBERTa-based QA models perform comparably, with their scores differing by no more than 5% on almost every probe. The authors chose this domain partly because definitional and ISA knowledge appear in about 20% and 18% of sampled questions in one science benchmark.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Allen Institute for Artificial IntelligenceSeattle · United States275 works · 97,634 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Question answering · 441 works

Cite

Cite as Kyle Richardson and Ashish Sabharwal. 2020. What Does My QA Model Know? Devising Controlled Probes Using Expert Knowledge. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00331

@article{richardson2020what,
  title = {{What Does My QA Model Know? Devising Controlled Probes Using Expert Knowledge}},
  author = {Kyle Richardson and Ashish Sabharwal},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.1162/tacl\_a\_00331},
  url = {https://doi.org/10.1162/tacl_a_00331}
}

References

60 references, 30 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yonatan Belinkov and James Glass. 2019. Analysis methods in neural language processing: A survey. Transactions of the Association for Computational Linguistics, 7:49–72.
  2. Peter Clark, Oyvind Tafjord, and Kyle Richardson. 2020. Transformers as soft reasoners over language. Proceedings of IJCAI.
  3. Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, and Antoine Bordes. 2017. Supervised learning of universal sentence representations from natural language inference data. Proceedings of EMNLP.
  4. Felix Hill, Kyunghyun Cho, Anna Korhonen, and Yoshua Bengio. 2016. Learning to understand phrases by embedding the dictionary. TACL, 4:17–30.
  5. Sujay Kumar Jauhar, Peter Turney, and Eduard Hovy. 2016. Tables as semi-structured knowledge for question answering. In Proceedings of ACL. find
  6. Ernest Davis. 2016. How to write science questions that are easy for people and hard for computers. AI Magazine, 37(1):13–22. find
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep didirectional transformers for language understanding. In Proceedings of NAACL. find
  8. Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. In Proceedings of EMNLP.
  9. Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. 2019. How can we know what language models know? arXiv preprint arXiv: 1911.12543.
  10. Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson Liu, Matthew Peters, Michael Schmitz, and Luke Zettlemoyer. 2018. AllenNLP: A deep semantic natural language processing platform. arXiv preprint arXiv:1803.07640.
  11. Dongyeop Kang, Tushar Khot, Ashish Sabharwal, and Eduard H. Hovy. 2018. AdvEntuRe: Adversarial training for textual entailment with knowledge-guided examples. In Proceedings of ACL. find
  12. Atticus Geiger, Ignacio Cases, Lauri Karttunen, and Chris Potts. 2019. Posing fair generalization tasks for natural language inference. In Proceedings of EMNLP.
  13. Aniruddha Kembhavi, Minjoon Seo, Dustin Schwenk, Jonghyun Choi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Are you smarter than a sixth grader? Textbook question answering for multimodal machine comprehension. In Proceedings of CVPR. find
  14. Atticus Geiger, Kyle Richardson, and Christopher Potts. 2020. Modular representation underlies systematic generalization in neural natural language inference models. arXiv preprint arXiv: 2004.14623. — [arXiv:2004.14623](https://arxiv.org/abs/2004.14623) find
  15. Tushar Khot, Peter Clark, Michal Guerquin, Peter Jansen, and Ashish Sabharwal. 2020. QASC: A dataset for question answering via sentence composition. In Proceedings of AAAI. find
  16. Max Glockner, Vered Shwartz, and Yoav Goldberg. 2018. Breaking NLI systems with sentences that require simple lexical inferences. In Proceedings of ACL.
  17. Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. RACE: Large-scale reading comprehension dataset from examinations. In Proceedings of EMNLP.
  18. Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018. Annotation artifacts in natural language inference data. In Proceedings of NAACL.
  19. Chen Liang, Xiao Yang, Neisarg Dave, Drew Wham, Bart Pursel, and C. Lee Giles. 2018. Distractor generation for multiple choice questions using learning to rank. In Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications. find
  20. Catherine Havasi, Robert Speer, and Jason Alonso. 2007. ConceptNet 3: A flexible, multilingual semantic network for common sense knowledge. In Proceedings of Recent Advances in NLP. find
  21. Nelson F. Liu, Roy Schwartz, and Noah A. Smith. 2019a. Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasets. Proceedings of NAACL.
  22. John Hewitt and Percy Liang. 2019. Designing and interpreting probes with control tasks. In Proceedings of EMNLP. find
  23. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. RoBERTa: A robustly optimized bert retraining approach. arXiv preprint arXiv:1907.11692.
  24. Miller, and Sebastian Riedel. 2019. Language models as knowledge bases? In Proceedings of EMNLP.
  25. Mohammad Taher Pilehvar and Jose Camacho-Collados. 2019. WiC: The word-in-context dataset for evaluating context-sensitive meaning representations. In Proceedings of NAACL.
  26. R. Thomas McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. Proceedings of ACL.
  27. Adam Poliak, Aparajita Haldar, Rachel Rudinger, J. Edward Hu, Ellie Pavlick, Aaron Steven White, and Benjamin Van Durme. 2018a. Collecting diverse natural language inference problems for sentence representation evaluation. In Proceedings of EMNLP.
  28. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? A new dataset for open book question answering. In Proceedings of EMNLP. find
  29. George A. Miller. 1995. Wordnet: a Lexical Database for English. Communications of the ACM, 38(11):39–41.
  30. Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, and Benjamin Van Durme. 2018b. Hypothesis only baselines in natural language inference. In Proceedings of *SEM.
  31. Nikita Nangia and Samuel R. Bowman. 2019. Human vs. muppet: A conservative estimate of human performance on the glue benchmark. arXiv preprint arXiv:1905.10425. find
  32. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. find
  33. Roberto Navigli and Simone Paolo Ponzetto. 2010. Babelnet: Building a very large multilingual semantic network. In Proceedings of ACL.
  34. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683.
  35. Ian Niles and Adam Pease. 2001. Towards a standard upper ontology. In Proceedings of the International Conference on Formal Ontology in Information Systems-Volume. find
  36. Aarne Ranta. 2017. Explainable machine translation with interlingual trees as certificates. In Proceedings of the Conference on Logic and Machine Learning in Natural Language. find
  37. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of EMNLP.
  38. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of NAACL.
  39. Matthew E. Peters, Mark Neumann, I. V. Logan, L. Robert, Roy Schwartz, Vidur Joshi, Sameer Singh, and Noah A. Smith. 2019. Knowledge enhanced contextual word representations. In Proceedings of EMNLP. find
  40. Sathish Reddy, Dinesh Raghu, Mitesh M. Khapra, and Sachindra Joshi. 2017. Generating natural language question-answer pairs from a knowledge graph using a rnn based question generation model. In Proceedings of EACL. find
  41. Kyle Richardson, Hai Hu, Lawrence S. Moss, and Ashish Sabharwal. 2020. Probing natural language inference models through semantic fragments. In Proceedings of AAAI.
  42. Ohad Rozen, Vered Shwartz, Roee Aharoni, and Ido Dagan. 2019. Diversify your datasets: Analyzing generalization via controlled variance in adversarial datasets. In Proceedings of CoNLL. find
  43. Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H. find
  44. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020. WINO-GRANDE: An adversarial winograd schema challenge at scale. Proceedings of AAAI. find
  45. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of NeurIPS, pages 5998–6008.
  46. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019. SocialIQA: Commonsense reasoning about social interactions. In Proceedings of EMNLP.
  47. Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020. Causal mediation analysis for interpreting neural NLP: The case of gender bias. arXiv preprint arXiv: 2004.12265. — [arXiv:2004.12265](https://arxiv.org/abs/2004.12265) find
  48. Timo Schick and Hinrich Schütze. 2020. Rare words: A major problem for contextualized embeddings and how to fix it by attentive mimicking. Proceedings of AAAI. find
  49. Alex Warstadt, Yu Cao, Ioana Grosu, Wei Peng, Hagen Blix, Yining Nie, Anna Alsop, Shikha Bordia, Haokun Liu, Alicia Parrish, Sheng-Fu Wang, Jason Phang, Anhad Mohananey, Phu Mon Htut, Paloma Jeretič, and Samuel R. Bowman. 2019. Investigating BERT’s knowledge of language: Five analysis methods with…
  50. Dominic Seyler, Mohamed Yahya, and Klaus Berberich. 2017. Knowledge questions from knowledge graphs. In Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval. find
  51. Meet Shah, Xinlei Chen, Marcus Rohrbach, and Devi Parikh. 2019. Cycle-consistency for robust visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. find
  52. Chenglei Si, Shuohang Wang, Min-Yen Kan, and Jing Jiang. 2019. What does BERT learn from multiple-choice reading comprehension datasets? arXiv preprint arXiv:1910.12391. — [arXiv:1910.12391](https://arxiv.org/abs/1910.12391) find
  53. Noah Webster. 1913. Webster’s revised unabridged dictionary of the english language, G. & C. Merriam Company. find
  54. Johannes Welbl, Nelson F Liu, and Matt Gardner. 2017. Crowdsourcing multiple choice science questions. In Proceedings of the 3rd Workshop on Noisy User-generated Text.
  55. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771.
  56. Alon Talmor, Yanai Elazar, Yoav Goldberg, and Jonathan Berant. 2019a. oLMpics – On what language model pre-training captures. ArXiv:, arXiv preprint arXiv:1912.13283.
  57. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019b. Commonsense QA: A question answering challenge targeting commonsense knowledge. In Proceedings of NAACL. find
  58. Hitomi Yanaka, Koji Mineshima, Daisuke Bekki, and Kentaro Inui. 2020. Do neural models learn systematicity of monotonicity inference in natural language? In Proceedings of ACL. find
  59. Niket Tandon, Gerard De Melo, and Gerhard Weikum. 2017. Webchild 2.0: Fine-grained commonsense knowledge distillation. In Proceedings of ACL. find
  60. Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. 2018. SWAG: A large-scale adversarial dataset for grounded commonsense inference. In Proceedings of EMNLP. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/richardson-2020-what-does-my-qa-model/