Elamur Research
☰
home / papers / 2019 / Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches

Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1904.01172

Summary

This survey maps the fast-growing area of natural language inference (NLI), understood broadly as a machine's ability to draw conclusions that a text implies but never states, using knowledge of the world and reasoning. The scope goes beyond the classic textual entailment setup. The authors organize the field into three strands:

  • Benchmarks: reference resolution, textual entailment, question answering and plausible inference tasks, plus guidance on task formulation, avoiding data bias, collection methods and difficulty.
  • Knowledge resources: linguistic, common and commonsense knowledge, drawn from annotated corpora, ontologies and knowledge graphs.
  • Approaches: symbolic, statistical and neural.

An appendix compares how dozens of datasets were built, from RTE-1 through WinoGrande and AlphaNLI. The authors conclude that strong scores, including those of pre-trained models like BERT, do not prove real understanding. Systems often exploit superficial artifacts and depend on large training sets. They therefore call for evaluation of explanations and generalization, possibly with almost no training data. The paper serves as a practical entry point for newcomers choosing datasets and resources.

Key points

  • Knowledge is split into three types: linguistic, common (explicit facts, sometimes domain-specific) and commonsense (unstated, roughly universal). Intuitive physics and intuitive psychology are highlighted as core commonsense domains.
  • The appendix codes each dataset's creation steps (context, answer, alternatives, annotation, validation) as expert-made (M), automatically generated (A), text-mined (T) or crowdsourced (C).
  • Dataset sizes vary enormously. RTE-1 has 1.37K items and COPA 1.00K, while SWAG has 114K, SNLI 570K and GLUE 1.44M.
  • Earlier resources such as MUC, ACE and early QA tracks are excluded because they mainly needed linguistic context rather than outside knowledge.
  • Proposed future directions include explanation-based evaluation, modeled on the VCR benchmark, and metrics for domain adaptation and generalization. The authors also suggest test setups that supply only a few examples instead of a training set.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · AI & machine learning

Cited by

78 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 73 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1904.01172 · arXiv 1904.01172

@misc{storks2019recent,
  title = {{Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches}},
  author = {Storks, Shane and Gao, Qiaozi and Chai, Joyce Y.},
  year = {2019},
  doi = {10.48550/arxiv.1904.01172},
  eprint = {1904.01172},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1904.01172}
}

References

312 references, 57 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Agrawal, A., Lu, J., Antol, S., Mitchell, M., Zitnick, C. L., Batra, D., & Parikh, D. (2015). VQA: Visual Question Answering. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV 2015), Santiago, Chile. IEEE. find
  2. Agrawal, A., Lu, J., Antol, S., Mitchell, M., Zitnick, C. L., Parikh, D., & Batra, D. (2017). VQA: Visual Question Answering. Int. J. Comput. Vision, 123(1), 4–31. find
  3. Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sünderhauf, N., Reid, I., Gould, S., & van den Hengel, A. (2018). Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018), Salt Lake City, UT, USA. IEEE. find our card
  4. Andrade, D., Bai, B., Rajendran, R., & Watanabe, Y. (2018). Leveraging knowledge bases for future prediction with memory comparison networks. AI Communications. find
  5. Angeli, G., & Manning, C. D. (2014). NaturalLI: Natural Logic Inference for Common Sense Reasoning. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), pp. 534–545, Doha, Qatar. Association for Computational Linguistics. find
  6. Aristotle (1989). Prior Analytics. Hackett Publishing. find
  7. Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., & Ives, Z. (2007). DBpedia: A Nucleus for a Web of Open Data. In Aberer, K., Choi, K.-S., Noy, N., Allemang, D., Lee, K.-I., Nixon, L., Golbeck, J., Mika, P., Maynard, D., Mizoguchi, R., Schreiber, G., & Cudré-Mauroux, P. (Eds.), Proceedings of the Semantic Web Challenge 2007 Co-Located with ISWC 2007 + ASWC 2007, Lecture Notes in Computer Science, pp. 722–735, Busan, Korea. Springer Berlin Heidelberg. find
  8. Bahdanau, D., Cho, K., & Bengio, Y. (2015). Neural Machine Translation by Jointly Learning to Align and Translate. In Proceedings of the 3rd International Conference on Learning Representations (ICLR 2015). find our card
  9. Bahl, L. R., Jelinek, F., & Mercer, R. L. (1983). A maximum likelihood approach to continuous speech recognition. IEEE Transactions on Pattern Analysis & Machine Intelligence, 5(2), 179–190. find
  10. Baillargeon, R. (2004). Infants’ physical world. Current Directions in Psychological Science, 13(3), 89–94. find
  11. Baker, C. F., Fillmore, C. J., & Lowe, J. B. (1998). The Berkeley FrameNet Project. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics (ACL ’98/COLING ’98), COLING ’98, pp. 86–90, Montreal, QC, Canada. Association for Computational Linguistics. find our card
  12. Banarescu, L., Bonial, C., Cai, S., Georgescu, M., Griffitt, K., Hermjakob, U., Knight, K., Koehn, P., Palmer, M., & Schneider, N. (2013). Abstract Meaning Representation for Sembanking. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, pp. 178–186, Sofia, Bulgaria. Association for Computational Linguistics. find our card
  13. Bar-Haim, R., Dagan, I., Dolan, B., Ferro, L., Giampiccolo, D., Magnini, B., & Szpektor, I. (2006). The Second PASCAL Recognising Textual Entailment Challenge. In Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment, Venice, Italy. find
  14. Bauer, L., Wang, Y., & Bansal, M. (2018). Commonsense for Generative Multi-Hop Question Answering Tasks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 4220–4230, Brussels, Belgium. Association for Computational Linguistics. find
  15. Bejan, C. A., & Harabagiu, S. M. (2008). A Linguistic Resource for Discovering Event Structures and Resolving Event Coreference. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC-2008), Marrakech, Morocco. European Language Resources Association (ELRA). find
  16. Belinkov, Y., Poliak, A., Shieber, S. M., Van Durme, B., & Rush, A. M. (2019). Don’t Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), Florence, Italy. Association for Computational Linguistics. find our card
  17. Belz, A., & Reiter, E. (2006). Comparing Automatic and Human Evaluation of NLG Systems. In 11th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics. find
  18. Bentivogli, L., Clark, P., Dagan, I., & Giampiccolo, D. (2010). The Sixth PASCAL Recognizing Textual Entailment Challenge. In Proceedings of the Third Text Analysis Conference (TAC 2010), Gaithersburg, MD, USA. National Institute of Standards and Technology. find
  19. Bentivogli, L., Clark, P., Dagan, I., & Giampiccolo, D. (2011). The Seventh PASCAL Recognizing Textual Entailment Challenge. In Proceedings of the Fourth Text Analysis Conference (TAC 2011), Gaithersburg, MD, USA. National Institute of Standards and Technology. find
  20. Bentivogli, L., Dagan, I., Dang, H. T., Giampiccolo, D., & Magnini, B. (2009). The Fifth PAS-CAL Recognizing Textual Entailment Challenge. In Proceedings of the Second Text Analysis Conference (TAC 2009), p. 15, Gaithersburg, MD, USA. National Institute of Standards and Technology. find
  21. Berant, J., Dagan, I., & Goldberger, J. (2011). Global Learning of Typed Entailment Rules. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (HLT ’11), HLT ’11, pp. 610–619, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  22. Bhagavatula, C., Bras, R. L., Malaviya, C., Sakaguchi, K., Holtzman, A., Rashkin, H., Downey, D., Yih, S. W.-t., & Choi, Y. (2019). Abductive commonsense reasoning. arXiv: 1908.05739. ↗ our card
  23. Birnbaum, L. (1991). Rigor mortis: A response to Nilsson’s “Logic and artificial intelligence”. Artificial Intelligence, 47(1-3), 57–77. find
  24. Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017). Enriching Word Vectors with Subword Information. Transactions of the Association for Computational Linguistics, 5, 135–146. find
  25. Bollacker, K., Evans, C., Paritosh, P., Sturge, T., & Taylor, J. (2008). Freebase: A Collaboratively Created Graph Database for Structuring Human Knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, SIGMOD ’08, pp. 1247–1250, New York, NY, USA. ACM. find
  26. Boole, G. (1854). An Investigation of the Laws of Thought: On Which Are Founded the Mathematical Theories of Logic and Probabilities. Dover Publications. find
  27. Bos, J., Basile, V., Evang, K., Venhuizen, N. J., & Bjerva, J. (2017). The Groningen Meaning Bank. In Handbook of Linguistic Annotation, pp. 463–496. Springer. find
  28. Bosselut, A., Levy, O., Holtzman, A., Ennis, C., Fox, D., & Choi, Y. (2018). Simulating Action Dynamics with Neural Process Networks. In Proceedings of the 6th International Conference on Learning Representations (ICLR 2018). find
  29. Bosselut, A., Rashkin, H., Sap, M., Malaviya, C., Celikyilmaz, A., & Choi, Y. (2019). COMET: Commonsense Transformers for Automatic Knowledge Graph Construction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), Florence, Italy. Association for Computational Linguistics. find our card
  30. Botschen, T., Sorokin, D., & Gurevych, I. (2018). Frame- and Entity-Based Knowledge for Common-Sense Argumentative Reasoning. In Proceedings of the 5th Workshop on Argument Mining, pp. 90–96, Brussels, Belgium. Association for Computational Linguistics. find
  31. Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015). A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP 2015), pp. 632–642, Lisbon, Portugal. Association for Computational Linguistics. find
  32. Brendel, W., & Bethge, M. (2019). Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet. In Proceedings of the 7th International Conference on Learning Representations (ICLR 2019), New Orleans, LA, USA. find
  33. Brown, P. F., Cocke, J., Della Pietra, S. A., Della Pietra, V. J., Jelinek, F., Lafferty, J. D., Mercer, R. L., & Roossin, P. S. (1990). A statistical approach to machine translation. Computational linguistics, 16(2), 79–85. find
  34. Brown, P. F., Della Pietra, V. J., deSouza, P. V., Lai, J. C., & Mercer, R. L. (1992). Class-Based n-gram Models of Natural Language. Computational Linguistics, 18(4), 467–480. find
  35. Camacho-Collados, J., & Pilehvar, M. T. (2018). From Word to Sense Embeddings: A Survey on Vector Representations of Meaning. J. Artif. Int. Res., 63(1), 743–788. find
  36. Cambria, E., Olsher, D., & Rajagopal, D. (2014a). SenticNet 3: A Common and Common-Sense Knowledge Base for Cognition-Driven Sentiment Analysis. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI-14), Québec City, QC, Canada. AAAI Press. find
  37. Cambria, E., Song, Y., Wang, H., & Howard, N. (2014b). Semantic Multidimensional Scaling for Open- Domain Sentiment Analysis. IEEE Intelligent Systems, 29(2), 44–51. find
  38. Cambria, E., Song, Y., Wang, H., & Hussain, A. (2011). Isanette: A Common and Common Sense Knowledge Base for Opinion Mining. In 2011 IEEE 11th International Conference on Data Mining Workshops, pp. 315–322, Vancouver, BC, Canada. IEEE. find
  39. Cambria, E., Speer, R., Havasi, C., & Hussain, A. (2010). SenticNet: A Publicly Available Semantic Resource for Opinion Mining. In AAAI Fall Symposium on Commonsense Knowledge, Menlo Park, CA, USA. AAAI Press. find
  40. Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Jr, E. R. H., & Mitchell, T. M. (2010). Toward an Architecture for Never-Ending Language Learning. In Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI-10), Atlanta, GA, USA. AAAI Press. find
  41. Cer, D., Diab, M., Agirre, E., Lopez-Gazpio, I., & Specia, L. (2017). SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp. 1–14, Vancouver, BC, Canada. Association for Computational Linguistics. find
  42. Chaganty, A., & Liang, P. (2016). How Much is 131 Million Dollars? Putting Numbers in Perspective with Compositional Descriptions. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL 2016), pp. 578–587, Berlin, Germany. Association for Computational Linguistics. find
  43. Chai, J. Y., Gao, Q., She, L., Yang, S., Saba-Sadiya, S., & Xu, G. (2018). Language to Action: Towards Interactive Task Learning with Physical Agents. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI 2018), pp. 2–9, Stockholm, Sweden. International Joint Conferences on Artificial Intelligence Organization. find
  44. Chambers, N., & Jurafsky, D. (2008). Unsupervised Learning of Narrative Event Chains. In Proceedings of ACL-08: HLT, Columbus, OH, USA. Association for Computational Linguistics. find
  45. Charniak, E., Altun, Y., Braz, R. d. S., Garrett, B., Kosmala, M., Moscovich, T., Pang, L., Pyo, C., Sun, Y., Wy, W., Yang, Z., Zeller, S., & Zorn, L. (2000). Reading Comprehension Programs in a Statistical-language-processing Class. In Proceedings of the 2000 ANLP/NAACL Workshop on Reading Comprehension Tests As Evaluation for Computer-Based Language Understanding Sytems, ANLP/NAACL-ReadingComp ’00, pp. 1–5, Seattle, WA, USA. Association for Computational Linguistics. find
  46. Chelba, C., Mikolov, T., Schuster, M., Ge, Q., Brants, T., Koehn, P., & Robinson, T. (2014). One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling. In 15th Annual Conference of the International Speech Communication Association (INTERSPEECH 2014), Singapore, Singapore. ISCA Archive. find our card
  47. Chen, Q., Zhu, X., Ling, Z.-H., Inkpen, D., & Wei, S. (2018). Neural Natural Language Inference Models Enhanced with External Knowledge. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2406–2417, Melbourne, Australia. Association for Computational Linguistics. find
  48. Chen, Q., Zhu, X., Ling, Z., Wei, S., Jiang, H., & Inkpen, D. (2017). Enhanced LSTM for Natural Language Inference. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL 2017), Vancouver, BC, Canada. Association for Computational Linguistics. find
  49. Chen, Z., & Ji, H. (2009). Graph-based Event Coreference Resolution. In Proceedings of the 2009 Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-4), pp. 54–57, Suntec, Singapore. Association for Computational Linguistics. find
  50. Chen, Z., Cui, Y., Ma, W., Wang, S., Liu, T., & Hu, G. (2018). HFL-RC System at SemEval-2018 Task 11: Hybrid Multi-Aspects Model for Commonsense Reading Comprehension. arXiv: 1803.05655. ↗
  51. Chklovski, T. (2003). Learner: A System for Acquiring Commonsense Knowledge by Analogy. In Proceedings of the 2nd International Conference on Knowledge Capture (K-CAP ’03), K-CAP ’03, pp. 4–12, New York, NY, USA. ACM. find
  52. Chklovski, T., & Pantel, P. (2004). VerbOcean: Mining the Web for Fine-Grained Semantic Verb Relations. In Lin, D., & Wu, D. (Eds.), Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP 2004), pp. 33–40, Barcelona, Spain. Association for Computational Linguistics. find
  53. Cho, K., van Merrienboer, B., Bahdanau, D., & Bengio, Y. (2014). On the Properties of Neural Machine Translation: Encoder-Decoder Approaches. In Proceedings of the Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation (SSST-8), Doha, Qatar. Association for Computational Linguistics. find our card
  54. Choi, E., He, H., Iyyer, M., Yatskar, M., Yih, W.-t., Choi, Y., Liang, P., & Zettlemoyer, L. (2018). QuAC : Question Answering in Context. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium. Association for Computational Linguistics. find our card
  55. Chu, Z., Wang, H., Gimpel, K., & McAllester, D. (2017). Broad Context Language Modeling as Reading Comprehension. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2017), pp. 52–57, Valencia, Spain. Association for Computational Linguistics. find
  56. Clark, C., & Gardner, M. (2018). Simple and Effective Multi-Paragraph Reading Comprehension. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018), pp. 845–855, Melbourne, Australia. Association for Computational Linguistics. find our card
  57. Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., & Tafjord, O. (2018). Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge. arXiv: 1803.05457. ↗
  58. Collell, G., Gool, L. V., & Moens, M.-F. (2018). Acquiring Common Sense Spatial Knowledge through Implicit Spatial Templates. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18), New Orleans, LA, USA. AAAI Press. find
  59. Dagan, I., Glickman, O., & Magnini, B. (2005). The PASCAL Recognising Textual Entailment Challenge. In Quiñonero-Candela, J., Dagan, I., Magnini, B., & d’Alché-Buc, F. (Eds.), Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment, Vol. 3944, pp. 177–190. Springer Berlin Heidelberg, Berlin, Heidelberg. find
  60. Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019). Transformer-XL: Attentive Language Models beyond a Fixed-Length Context. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), pp. 2978– 2988, Florence, Italy. Association for Computational Linguistics. find
  61. DARPA (2018). Machine Common Sense. Broad Agency Announcement HR001119S0005, Defense Advanced Research Projects Agency, Arlington, VA, USA. find
  62. Das, A., Datta, S., Gkioxari, G., Lee, S., Parikh, D., & Batra, D. (2018). Embodied Question Answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2018), Salt Lake City, UT, USA. IEEE. find
  63. Das, R., Munkhdalai, T., Yuan, X., Trischler, A., & McCallum, A. (2019). Building Dynamic Knowledge Graphs from Text using Machine Reading Comprehension. In Proceedings of the 7th International Conference on Learning Representations (ICLR 2019), New Orleans, LA, USA. find
  64. Davidson, H. A. (1992). Alfarabi, Avicenna, and Averroes on Intellect: Their Cosmologies, Theories of the Active Intellect, and Theories of Human Intellect. Oxford University Press on Demand. find
  65. Davis, E. (2017). Logical Formalizations of Commonsense Reasoning: A Survey. Journal of Artificial Intelligence Research, 59, 651–723. find
  66. Davis, E., & Marcus, G. (2015). Commonsense reasoning and commonsense knowledge in artificial intelligence. Commun. ACM, 58(9), 92–103. find
  67. Davis, E., Morgenstern, L., & Ortiz, C. (2018). The Winograd Schema Challenge. https://cs.nyu.edu/faculty/davise/papers/WinogradSchemas/WS.html. ↗
  68. Davis, E., Morgenstern, L., & Ortiz, C. L. (2017). The First Winograd Schema Challenge at IJCAI-16. AI Magazine; La Canada, 38(3), 97–98. find
  69. de Marneffe, M.-C., Simons, M., & Tonhauser, J. (2019). The CommitmentBank: Investigating projection in naturally occurring discourse. In Proceedings of Sinn Und Bedeutung 23, Barcelona, Spain. find our card
  70. Dechter, R. (2013). Reasoning with Probabilistic and Deterministic Graphical Models: Exact Algorithms. Synthesis Lectures on Artificial Intelligence and Machine Learning, 7(3), 1–191. find
  71. Denkowski, M., & Lavie, A. (2011). Meteor 1.3: Automatic Metric for Reliable Optimization and Evaluation of Machine Translation Systems. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pp. 85–91, Edinburgh, Scotland. Association for Computational Linguistics. find
  72. Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv: 1810.04805. ↗
  73. Doddington, G., Mitchell, A., Przybocki, M., Ramshaw, L., Strassel, S., & Weischedel, R. (2004). The automatic content extraction (ACE) program – tasks, data, and evaluation. In Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC ’04), Lisbon, Portugal. European Language Resources Association (ELRA). find
  74. Dolan, W. B., & Brockett, C. (2005). Automatically Constructing a Corpus of Sentential Paraphrases. In Proceedings of the Third International Workshop on Paraphrasing (IWP2005), Jeju Island, Korea. find our card
  75. Dua, D., Wang, Y., Dasigi, P., Stanovsky, G., Singh, S., & Gardner, M. (2019). DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs. In Proceedings of the 17th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019), Minneapolis, MN, USA. Association for Computational Linguistics. find our card
  76. Dzikovska, M. O., Nielsen, R. D., Brew, C., Leacock, C., Giampiccolo, D., Bentivogli, L., Clark, P., Dagan, I., & Dang, H. T. (2013). SemEval-2013 Task 7: The Joint Student Response Analysis and 8th Recognizing Textual Entailment Challenge. In Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval-2013), p. 13, Atlanta, GA, USA. Association for Computational Linguistics. find
  77. Elliott, D., & Keller, F. (2014). Comparing Automatic Evaluation Measures for Image Description. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 452–457. Association for Computational Linguistics. find
  78. Emami, A., De La Cruz, N., Trischler, A., Suleman, K., & Cheung, J. C. K. (2018). A Knowledge Hunting Framework for Common Sense Reasoning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), pp. 1949–1958, Brussels, Belgium. Association for Computational Linguistics. find
  79. Etzioni, O., Banko, M., Soderland, S., & Weld, D. S. (2008). Open information extraction from the web. Communications of the ACM, 51(12), 68. find our card
  80. Etzioni, O., Cafarella, M., Downey, D., Popescu, A.-M., Shaked, T., Soderland, S., Weld, D. S., & Yates, A. (2005). Unsupervised named-entity extraction from the Web: An experimental study. Artificial Intelligence, 165(1), 91–134. find
  81. Fellbaum, C. (1999). WordNet : An Electronic Lexical Database, Vol. 2nd printing of Language, Speech, and Communication. A Bradford Book, Cambridge, Mass. find
  82. Fillmore, C. J. (1976). The Need for a Frame Semantics within Linguistics. Statistical Methods in Linguistics, 5–29. find
  83. Fillmore, C. J. (1977). Scenes-and-Frames Semantics. Fundamental Studies in Computer Science, 59, 55–82. find
  84. Fillmore, C. J., Baker, C. F., & Sato, H. (2002). The FrameNet Database and Software Tools.. In Proceedings of the Third International Conference on Language Resources and Evaluation (LREC’02), Las Palmas, Canary Islands - Spain. European Language Resources Association (ELRA). find
  85. Forbes, M., & Choi, Y. (2017). Verb Physics: Relative Physical Knowledge of Actions and Objects. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL 2017), pp. 266–276, Vancouver, Canada. Association for Computational Linguistics. find
  86. Fouhey, D. F., Kuo, W., Efros, A. A., & Malik, J. (2018). From Lifestyle VLOGs to Everyday Interactions. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018), Salt Lake City, UT, USA. IEEE. find
  87. Franke, J., Niehues, J., & Waibel, A. (2018). Robust and Scalable Differentiable Neural Computer for Question Answering. In Proceedings of the Workshop on Machine Reading for Question Answering (MRQA), pp. 47–59, Melbourne, Australia. Association for Computational Linguistics. find
  88. Gabrilovich, E., Ringgaard, M., & Subramanya, A. (2013). FACC1: Freebase annotation of ClueWeb corpora, Version 1 (Release date 2013-06-26, Format version 1, Correction level 0). Note: http://lemurproject.org/clueweb09/FACC1/Cited by, 5. ↗
  89. Gao, Q., Doering, M., Yang, S., & Chai, J. (2016). Physical Causality of Action Verbs in Grounded Language Understanding. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL 2016), pp. 1814–1824, Berlin, Germany. Association for Computational Linguistics. find
  90. Gao, Q., Yang, S., Chai, J., & Vanderwende, L. (2018). What Action Causes This? Towards Naive Physical Action-Effect Prediction. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 934–945, Melbourne, Australia. Association for Computational Linguistics. find
  91. Giampiccolo, D., Dang, H. T., Magnini, B., Dagan, I., & Dolan, B. (2008). The Fourth PASCAL Recognizing Textual Entailment Challenge. In Proceedings of the First Text Analysis Conference (TAC 2008), p. 9, Gaithersburg, MD, USA. National Institute of Standards and Technology. find
  92. Giampiccolo, D., Magnini, B., Dagan, I., & Dolan, B. (2007). The Third PASCAL Recognizing Textual Entailment Challenge. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, RTE ’07, pp. 1–9, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  93. Gildea, D., & Jurafsky, D. (2000). Automatic Labeling of Semantic Roles. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics (ACL 2000), pp. 512–520, Hong Kong. Association for Computational Linguistics. find our card
  94. Glickman, O. (2006). Applied Textual Entailment. Ph.D. Thesis, Bar Ilan University. find
  95. Glockner, M., Shwartz, V., & Goldberg, Y. (2018). Breaking NLI systems with sentences that require simple lexical inferences. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018), pp. 650–655, Melbourne, Australia. Association for Computational Linguistics. find our card
  96. Gordon, A. S. (2016). Commonsense Interpretation of Triangle Behavior. In Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA. AAAI Press. find
  97. Gordon, D. B., Kembhavi, A., Rastegari, M., Redmon, J., Fox, D., & Farhadi, A. (2018). IQA: Visual Question Answering in Interactive Environments. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4089–4098. find
  98. Gordon, J., & Schubert, L. (2011). Discovering Commonsense Entailment Rules Implicit in Sentences. In Proceedings of the TextInfer 2011 Workshop on Textual Entailment, pp. 59–63, Edinburgh, Scottland, UK. Association for Computational Linguistics. find
  99. Gordon, J., & Schubert, L. K. (2012). Using Textual Patterns to Learn Expected Event Frequencies. In Proceedings of the Joint Workshop on Automatic Knowledge Base Construction and Web-Scale Knowledge Extraction, AKBC-WEKEX ’12, pp. 122–127, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  100. Gordon, J., Van Durme, B., & Schubert, L. (2010). Evaluation of Commonsense Knowledge with Mechanical Turk. In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk, pp. 159–162, Los Angeles. Association for Computational Linguistics. find
  101. Goyal, R., Kahou, S. E., Michalski, V., Materzyńska, J., Westphal, S., Kim, H., Haenel, V., Fruend, I., Yianilos, P., Mueller-Freitag, M., Hoppe, F., Thurau, C., Bax, I., & Memisevic, R. (2017). The "something something" video database for learning and evaluating visual common sense. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV 2017). find
  102. Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S. G., Grefenstette, E., Ramalho, T., Agapiou, J., Badia, A. P., Hermann, K. M., Zwols, Y., Ostrovski, G., Cain, A., King, H., Summerfield, C., Blunsom, P., Kavukcuoglu, K., & Hassabis, D. (2016). Hybrid computing using a neural network with dynamic external memory. Nature, 538(7626), 471–476. find
  103. Grishman, R., & Sundheim, B. (1996). Message Understanding Conference-6: A Brief History. In Proceedings of the 16th Conference on Computational Linguistics - Volume 1, COLING ’96, pp. 466–471, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  104. Gupta, A., & Durrett, G. (2019). Tracking Discrete and Continuous Entity State for Process Understanding. In Proceedings of the Third Workshop on Structured Prediction for NLP, pp. 7–12, Minneapolis, MN, USA. Association for Computational Linguistics. find
  105. Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S., & Smith, N. A. (2018). Annotation Artifacts in Natural Language Inference Data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2018), pp. 107–112, New Orleans, LA, USA. Association for Computational Linguistics. find
  106. Habernal, I., Wachsmuth, H., Gurevych, I., & Stein, B. (2018). The Argument Reasoning Comprehension Task: Identification and Reconstruction of Implicit Warrants. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2018), pp. 1930–1940, New Orleans, Louisiana. Association for Computational Linguistics. find
  107. Harnad, S. (2000). Minds, Machines and Turing. Journal of Logic, Language and Information, 9(4), 425–445. find
  108. Hartshorne, J. K., Bonial, C., & Palmer, M. (2013). The VerbCorner project: Toward an empirically-based semantic decomposition of verbs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP 2013), pp. 1438–1442, Seattle, WA, USA. Association for Computational Linguistics. find
  109. Havasi, C., Speer, R., & Alonso, J. B. (2007). ConceptNet 3: A flexible, multilingual semantic network for common sense knowledge. In Recent Advances in Natural Language Processing (RANLP-07), Borovets, Bulgaria. Association for Computational Linguistics. find
  110. Heider, F., & Simmel, M. (1944). An experimental study of apparent behavior. The American Journal of Psychology, 57, 243–259. find
  111. Heilbron, F. C., Escorcia, V., Ghanem, B., & Niebles, J. C. (2015). ActivityNet: A large-scale video benchmark for human activity understanding. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015), pp. 961–970, Boston, MA, USA. IEEE. find
  112. Henaff, M., Weston, J., Szlam, A., Bordes, A., & LeCun, Y. (2017). Tracking the World State with Recurrent Entity Networks. In Proceedings of the 5th International Conference on Learning Representations (ICLR 2017), p. 14, Palais des Congrès Neptune, Toulon, France. find
  113. Hickl, A., Bensley, J., Williams, J., Roberts, K., Rink, B., & Shi, Y. (2006). Recognizing textual entailment with LCC’s GROUNDHOG system. In Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment, Venice, Italy. find
  114. Hilbert, D., & Ackermann, W. (1999). Principles of Mathematical Logic. American Mathematical Soc. find
  115. Hill, F., Bordes, A., Chopra, S., & Weston, J. (2015). The Goldilocks Principle: Reading Children’s Books with Explicit Memory Representations. arXiv: 1511.02301. ↗ our card
  116. Hirschman, L., Light, M., Breck, E., & Burger, J. D. (1999). Deep Read: A Reading Comprehension System. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics (ACL 1999), pp. 325–332, College Park, Maryland, USA. Association for Computational Linguistics. find
  117. Hobbs, J. R., Stickel, M. E., Appelt, D. E., & Martin, P. (1993). Interpretation as abduction. Artificial Intelligence, 63(1), 69–142. find
  118. Hochreiter, S., & Schmidhuber, J. (1997). Long Short-Term Memory. Neural Comput., 9(8), 1735– 1780. find
  119. Hoffart, J., Suchanek, F. M., Berberich, K., & Weikum, G. (2012). YAGO2: A spatially and temporally enhanced knowledge base from Wikipedia. Artificial Intelligence, 194, 28–61. find
  120. Huang, G., Liu, Z., van der Maaten, L., & Weinberger, K. Q. (2016). Densely Connected Convolutional Networks. In Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016), Las Vegas, NV, USA. IEEE. find
  121. Huang, L., Bras, R. L., Bhagavatula, C., & Choi, Y. (2019). Cosmos QA: Machine reading comprehension with contextual commonsense reasoning. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP 2019), Hong Kong, China. Association for Computational Linguistics. find our card
  122. Hudson, D. A., & Manning, C. D. (2018). Compositional Attention Networks for Machine Reasoning. In Proceedings of the 6th International Conference on Learning Representations (ICLR 2018), Vancouver, BC, Canada. find
  123. Hudson, D. A., & Manning, C. D. (2019). GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering. In 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2019), Long Beach, CA, USA. IEEE. find our card
  124. Iftene, A. (2008). UAIC Participation at RTE4. In Proceedings of the First Text Analysis Conference (TAC 2008), Gaithersburg, MD, USA. National Institute of Standards and Technology. find
  125. Iyer, S., Dandekar, N., & Csernai, K. (2017). First Quora Dataset Release: Question Pairs. https://data.quora.com/First-Quora-Dataset-Release-Question-Pairs. ↗
  126. Jastrz˛ebski, S., Bahdanau, D., Hosseini, S., Noukhovitch, M., Bengio, Y., & Cheung, J. (2018). Commonsense mining as knowledge base completion? A study on the impact of novelty. In Proceedings of the Workshop on Generalization in the Age of Deep Learning, pp. 8–16. Association for Computational Linguistics. find
  127. Jia, R., & Liang, P. (2017). Adversarial Examples for Evaluating Reading Comprehension Systems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP 2017), pp. 2021–2031. Association for Computational Linguistics. find our card
  128. Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Zitnick, C. L., & Girshick, R. (2017). CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017). IEEE. find our card
  129. Kafle, K., & Kanan, C. (2017). Visual Question Answering: Datasets, Algorithms, and Future Challenges. Computer Vision and Image Understanding, 163, 3–20. find
  130. Kamath, A., & Das, R. (2018). A Survey on Semantic Parsing. In Proceedings of the 1st Conference on Automated Knowledge Base Construction (AKBC 2019), Amherst, MA, USA. Association for Computational Linguistics. find
  131. Khashabi, D., Chaturvedi, S., Roth, M., Upadhyay, S., & Roth, D. (2018). Looking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2018), New Orleans, LA, USA. Association for Computational Linguistics. find our card
  132. Khot, T., Sabharwal, A., & Clark, P. (2018). SciTail: A Textual Entailment Dataset from Science Question Answering. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18), p. 9, New Orleans, LA, USA. AAAI Press. find
  133. Kiddon, C., Zettlemoyer, L., & Choi, Y. (2016). Globally Coherent Text Generation with Neural Checklist Models. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP 2016), pp. 329–339, Austin, TX, USA. Association for Computational Linguistics. find
  134. Kim, S., Hong, J.-H., Kang, I., & Kwak, N. (2019). Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information. In Proceedings of the Thirty-Third AAAI Conference on Artificial Inteligence (AAAI-19), Honolulu, HI, USA. AAAI Press. find
  135. Kingsbury, P., Palmer, M., & Marcus, M. (2002). Adding Semantic Annotation to the Penn Tree-Bank. In Proceedings of the Second International Conference on Human Language Technology Research (HLT ’02), p. 5, San Diego, CA, USA. Morgan Kaufmann Publishers Inc. find
  136. Kočiský, T., Schwarz, J., Blunsom, P., Dyer, C., Hermann, K. M., Melis, G., & Grefenstette, E. (2018). The NarrativeQA Reading Comprehension Challenge. Transactions of the Association for Computational Linguistics, 6, 317–328. find our card
  137. Kotzias, D., Denil, M., De Freitas, N., & Smyth, P. (2015). From group to individual labels using deep features. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 597–606, Sydney, Australia. ACM, ACM. find
  138. Kozareva, Z., & Hovy, E. (2011). Learning Temporal Information for States and Events. In Proceedings of the 2011 IEEE Fifth International Conference on Semantic Computing (ICSC ’11), ICSC ’11, pp. 424–429, Washington, DC, USA. IEEE Computer Society. find
  139. Kuo, Y.-L., & Hsu, J. Y.-J. (2010). Bridging Common Sense Knowledge Bases with Analogy by Graph Similarity. In Proceedings of the 2nd AAAI Conference on Collaboratively-Built Knowledge Sources and Artificial Intelligence, AAAIWS’10-02, pp. 22–27, Atlanta, GA, USA. AAAI Press. find
  140. Lai, A., & Hockenmaier, J. (2014). Illinois-LH: A Denotational and Distributional Approach to Semantics. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval-2014), pp. 329–334, Dublin, Ireland. Association for Computational Linguistics and Dublin City University. find
  141. Lai, G., Xie, Q., Liu, H., Yang, Y., & Hovy, E. (2017). RACE: Large-scale ReAding Comprehension Dataset From Examinations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP 2017), pp. 785–794, Copenhagen, Denmark. Association for Computational Linguistics. find our card
  142. Lakoff, G. (1970). Linguistics and natural logic. Synthese, 22(1), 151–271. find
  143. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019). ALBERT: A lite BERT for self-supervised learning of language representations. arXiv: 1909.11942. ↗
  144. Lee, K., Artzi, Y., Choi, Y., & Zettlemoyer, L. (2015). Event detection and factuality assessment with non-expert supervision. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP 2015), pp. 1643–1648, Lisbon, Portugal. Association for Computational Linguistics. find
  145. Lehnert, W. (1977). Human and Computational Question Answering*. Cognitive Science, 1(1), 47–73. find
  146. Lenat, D. B., & Guha, R. V. (1989). Building Large Knowledge-Based Systems; Representation and Inference in the Cyc Project (1st edition). Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA. find
  147. Levesque, H. J. (2011). The Winograd Schema Challenge. In AAAI Spring Symposium on Logical Formalizations of Commonsense Reasoning, Stanford, CA, USA. AAAI Press. find
  148. Levesque, H. J., Davis, E., & Morgenstern, L. (2012). The Winograd schema challenge.. In Proceedings of the Thirteenth International Conference on the Principles of Knowledge Representation and Reasoning (KR2012), Rome, Italy. AAAI Press. find
  149. Levin, B. (1993). English Verb Classes and Alternations: A Preliminary Investigation. University of Chicago Press, Chicago. find
  150. Lewis, C. I., Langford, C. H., & Lamprecht, P. (1959). Symbolic Logic. Dover publications New York. find
  151. Li, B., Lee-Urban, S., Johnston, G., & Riedl, M. O. (2013). Story Generation with Crowdsourced Plot Graphs. In Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence (AAAI-13), AAAI’13, pp. 598–604, Bellevue, Washington. AAAI Press. find
  152. Li, X., Taheri, A., Tu, L., & Gimpel, K. (2016). Commonsense Knowledge Base Completion. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pp. 1445–1455. Association for Computational Linguistics. find
  153. Lin, C.-Y. (2004). ROUGE: A Package for Automatic Evaluation of Summaries. In Marie-Francine Moens, S. S. (Ed.), Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, pp. 74–81, Barcelona, Spain. Association for Computational Linguistics. find
  154. Liu, H., & Singh, P. (2004). ConceptNet — A Practical Commonsense Reasoning Tool-Kit. BT Technology Journal, 22(4), 211–226. find
  155. Liu, N. F., Gardner, M., Belinkov, Y., Peters, M. E., & Smith, N. A. (2019). Linguistic Knowledge and Transferability of Contextual Representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2019), pp. 1073–1094. Association for Computational Linguistics. find our card
  156. Liu, Q., Jiang, H., Ling, Z.-H., Zhu, X., Wei, S., & Hu, Y. (2017). Combing Context and Commonsense Knowledge Through Neural Networks for Solving Winograd Schema Problems. In 2017 AAAI Spring Symposium Series. find
  157. Liu, X., He, P., Chen, W., & Gao, J. (2019a). Multi-Task Deep Neural Networks for Natural Language Understanding. arXiv: 1901.11504. ↗ our card
  158. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019b). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv: 1907.11692. ↗
  159. Lund, K., & Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instruments, & Computers, 28(2), 203–208. find
  160. MacCartney, B., & Manning, C. D. (2007). Natural Logic for Textual Inference. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pp. 193–200, Prague, Czech Republic. Association for Computational Linguistics. find
  161. Mahdisoltani, F., Biega, J., & Suchanek, F. M. (2013). YAGO3: A Knowledge Base from Multilingual Wikipedias. In Proceedings of the 6th Biennial Conference on Innovative Data Systems Research (CIDR 2013), Asilomar, CA, USA. find our card
  162. Mak, A. (2019). When Is Technology Too Dangerous to Release to the Public?.. find
  163. Manjunatha, V., Saini, N., & Davis, L. S. (2018). Explicit Bias Discovery in Visual Question Answering Models. arXiv: 1811.07789. ↗
  164. Manning, C., & Hudson, D. (2018). Towards real-world visual reasoning.. find
  165. Marasović, A. (2018). NLP’s generalization problem, and how researchers are tackling it.. find
  166. Marcus, M. P., Santorini, B., & Marcinkiewicz, M. A. (1993). Building a Large Annotated Corpus of English: The Penn Treebank. Computational Linguistics, 1(2). find our card
  167. Marelli, M., Menini, S., Baroni, M., Bentivogli, L., Bernardi, R., & Zamparelli, R. (2014a). A SICK cure for the evaluation of compositional distributional semantic models. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014), Reykjavik, Iceland. European Language Resources Association (ELRA). find
  168. Marelli, M., Bentivogli, L., Baroni, M., Bernardi, R., Menini, S., & Zamparelli, R. (2014b). SemEval-2014 Task 1: Evaluation of Compositional Distributional Semantic Models on Full Sentences through Semantic Relatedness and Textual Entailment. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval-2014), Dublin, Ireland. Association for Computational Linguistics. find our card
  169. Maslow, A. H. (1943). A theory of human motivation. Psychological Review, 50(4), 370–396. find
  170. McCarthy, J. (1968). Programs with Common Sense. In Semantic Information Processing, pp. 403–418. MIT Press. find
  171. McCarthy, J. (1976). An example for natural language understanding and the AI problems it raises. Formalizing Common Sense: Papers by John McCarthy, 355. find
  172. Medelyan, O., & Legg, C. (2008). Integrating Cyc and Wikipedia: Folksonomy meets rigorously defined common-sense. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (AAAI-08), Chicago, IL, USA. AAAI Press. find
  173. Mihaylov, T., Clark, P., Khot, T., & Sabharwal, A. (2018). Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), pp. 2381–2391, Brussels, Belgium. Association for Computational Linguistics. find
  174. Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space. In Proceedings of the 1st International Conference on Learning Representations (ICLR 2013), Scottsdale, AZ, USA. find
  175. Miller, G. A. (1995). WordNet: A Lexical Database for English. Commun. ACM, 38(11), 39–41. find our card
  176. Miller, G. A., Galanter, E., & Pribram, K. H. (1960). Plans and the Structure of Behavior. Plans and the Structure of Behavior. Henry Holt and Co, New York, NY, US. find
  177. Miller, T., Hempelmann, C., & Gurevych, I. (2017). SemEval-2017 Task 7: Detection and Interpretation of English Puns. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp. 58–68, Vancouver, Canada. Association for Computational Linguistics. find
  178. Miltsakaki, E., Prasad, R., Joshi, A., & Webber, B. (2004). The Penn Discourse Treebank. In Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC-2004), p. 4, Lisbon, Portugal. Evaluation and Language Resources Distribution Agency. find our card
  179. Minard, A.-L., Speranza, M., Urizar, R., Altuna, B., van Erp, M., Schoen, A., & van Son, C. (2016). MEANTIME, the NewsReader Multilingual Event and Time Corpus. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC-2016), Portorož, Slovenia. European Language Resources Association (ELRA). find
  180. Minsky, M. (1974). A Framework for Representing Knowledge. AI Memos. find
  181. Minsky, M. (2000). Commonsense-based interfaces. Communications of the ACM, 43(8), 66–73. find
  182. Mishra, B. D., Huang, L., Tandon, N., Yih, W.-t., & Clark, P. (2018). Tracking State Changes in Procedural Text: A Challenge Dataset and Models for Process Paragraph Comprehension. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2018), New Orleans, LA, USA. Association for Computational Linguistics. find
  183. Moore, R. C. (1982). The Role of Logic in Knowledge Representation and Commonsense Reasoning. In Proceedings of the Second AAAI Conference on Artificial Intelligence (AAAI ’82), AAAI’82, pp. 428–433, Pittsburgh, PA, USA. AAAI Press. find
  184. Morgan, A. D. (1847). Formal Logic: Or, The Calculus of Inference, Necessary and Probable. Taylor and Walton. find
  185. Morgenstern, L. (2016). Pronoun http://commonsensereasoning.org/disambiguation.html. ↗
  186. Morgenstern, L., Davis, E., & Ortiz, C. L. (2016). Planning, Executing, and Evaluating the Winograd Schema Challenge. AI Magazine, 37(1), 50–54. find
  187. Morgenstern, L., & Ortiz, C. (2015). The Winograd Schema Challenge. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI-15), Austin, TX, USA. AAAI Press. Disambiguation Problems. find
  188. Mostafazadeh, N., Chambers, N., He, X., Parikh, D., Batra, D., Vanderwende, L., Kohli, P., & Allen, J. (2016). A Corpus and Cloze Evaluation Framework for Deeper Understanding of Commonsense Stories. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2016), San Diego, CA, USA. Association for Computational Linguistics. find
  189. Mostafazadeh, N., Roth, M., Louis, A., Chambers, N., & Allen, J. (2017). LSDSem 2017 Shared Task: The Story Cloze Test. In Proceedings of the 2nd Workshop on Linking Models of Lexical, Sentential and Discourse-Level Semantics, pp. 46–51, Valencia, Spain. Association for Computational Linguistics. find
  190. Negri, M., Marchetti, A., Mehdad, Y., Bentivogli, L., & Giampiccolo, D. (2012). Semeval-2012 Task 8: Cross-lingual Textual Entailment for Content Synchronization. In Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval-2012), pp. 399–407, Montréal, Canada. Association for Computational Linguistics. find
  191. Negri, M., Marchetti, A., Mehdad, Y., Bentivogli, L., & Giampiccolo, D. (2013). Semeval-2013 Task 8: Cross-lingual Textual Entailment for Content Synchronization. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval-2013), pp. 25–33, Atlanta, Georgia, USA. Association for Computational Linguistics. find
  192. Nematzadeh, A., Burns, K., Grant, E., Gopnik, A., & Griffiths, T. (2018). Evaluating Theory of Mind in Question Answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), pp. 2392–2400, Brussels, Belgium. Association for Computational Linguistics. find
  193. Ng, H. T., Teo, L. H., & Kwan, J. L. P. (2000). A Machine Learning Approach to Answering Questions for Reading Comprehension Tests. In 2000 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, pp. 124–132, Hong Kong, China. Association for Computational Linguistics. find
  194. Niven, T., & Kao, H.-Y. (2019). Probing Neural Network Comprehension of Natural Language Arguments. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), Florence, Italy. Association for Computational Linguistics. find
  195. Nunberg, G. (1987). Position Paper on Common-sense and Formal Semantics. In Proceedings of the 1987 Workshop on Theoretical Issues in Natural Language Processing (TINLAP ’87), TINLAP ’87, pp. 129–133, Las Cruces, NM, USA. Association for Computational Linguistics. find
  196. Olsher, D. (2014). Semantically-based priors and nuanced knowledge core for Big Data, Social AI, and language understanding. Neural Networks, 58, 131–147. find
  197. Ortiz, C. (2016). Why We Need a Physically Embodied Turing Test and What It Might Look Like. AI Magazine, 37(1), 55–62. find
  198. Ostermann, S., Modi, A., Roth, M., Thater, S., & Pinkal, M. (2018). MCScript: A Novel Dataset for Assessing Machine Comprehension Using Script Knowledge. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  199. Paperno, D., Kruszewski, G., Lazaridou, A., Pham, N. Q., Bernardi, R., Pezzelle, S., Baroni, M., Boleda, G., & Fernandez, R. (2016). The LAMBADA dataset: Word prediction requiring a broad discourse context. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL 2016), pp. 1525–1534, Berlin, Germany. Association for Computational Linguistics. find our card
  200. Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2002). Bleu: A Method for Automatic Evaluation of Machine Translation. In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics (ACL 2002), pp. 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. find our card
  201. Paulheim, H. (2018). How much is a Triple? Estimating the Cost of Knowledge Graph Creation. In Proceedings of the 17th International Semantic Web Conference (ISWC 2018), Monterey, CA, USA. Springer. find
  202. Peirce, C. S. (1883). A Theory of Probable Inference. In Peirce, C. S. (Ed.), Studies in Logic by Members of the Johns Hopkins University, pp. 126–181. Little, Brown, and Company, Boston, MA. From the Commens Bibliography | http://www.commens.org/bibliography/collection_article/peirce-charles-s-1883-theory-probable-inference-studies-logic. ↗
  203. Pennington, J., Socher, R., & Manning, C. (2014). Glove: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), pp. 1532–1543, Doha, Qatar. Association for Computational Linguistics. find
  204. Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep Contextualized Word Representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2018), pp. 2227–2237, New Orleans, Louisiana. Association for Computational Linguistics. find
  205. Pilehvar, M. T., & Camacho-Collados, J. (2019). WiC: The Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2019), pp. 1267–1273, Minneapolis, MN, USA. Association for Computational Linguistics. find our card
  206. Plutchik, R. (1980). A general psychoevolutionary theory of emotion. In Theories of Emotion, pp. 3–31. Academic Press. find
  207. Pohl, A. (2012). Classifying the Wikipedia Articles into the OpenCyc Taxonomy. In Proceedings of the Web of Linked Entities Workshop in Conjunction with the 11th International Semantic Web Conference (ISWC 2012), Boston, MA, USA. find
  208. Poliak, A., Haldar, A., Rudinger, R., Hu, J. E., Pavlick, E., White, A. S., & Van Durme, B. (2018a). Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), Brussels, Belgium. Association for Computational Linguistics. find our card
  209. Poliak, A., Naradowsky, J., Haldar, A., Rudinger, R., & Van Durme, B. (2018b). Hypothesis Only Baselines in Natural Language Inference. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pp. 180–191, New Orleans, LA, USA. Association for Computational Linguistics. find
  210. Ponzetto, S. P., & Strube, M. (2007). Deriving a Large Scale Taxonomy from Wikipedia. In Proceedings of the 22nd National Conference on Artificial Intelligence, AAAI’07, pp. 1440– 1445, Vancouver, BC, Canada. AAAI Press. find
  211. Poole, D. (1990). A methodology for using a default and abductive reasoning system. International Journal of Intelligent Systems, 5(5), 521–548. find
  212. Pradhan, S. S., Hovy, E., Marcus, M., Palmer, M., Ramshaw, L., & Weischedel, R. (2007). OntoNotes: A Unified Relational Semantic Representation. In International Conference on Semantic Computing (ICSC 2007), pp. 517–526. find
  213. Punyakanok, V., Roth, D., Yih, W.-t., & Zimak, D. (2004). Semantic Role Labeling via Integer Linear Programming Inference. In Proceedings of the 20th International Conference on Computational Linguistics (COLING ’04), COLING ’04, Geneva, Switzerland. Association for Computational Linguistics. find
  214. Quillian, M. R. (1966). Semantic Memory. Tech. rep. 2, Bolt Beranek and Newman Inc., Cambridge, MA, USA. find
  215. Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving Language Understanding with Unsupervised Learning.. find
  216. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language Models are Unsupervised Multitask Learners. https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf. ↗
  217. Rahman, A., & Ng, V. (2012). Resolving Complex Cases of Definite Pronouns: The Winograd Schema Challenge. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL 2012), pp. 777–789, Jeju Island, Korea. Association for Computational Linguistics. find our card
  218. Raina, R., Ng, A. Y., & Manning, C. D. (2005). Robust textual inference via learning and abductive reasoning. In Proceedings of the Twentieth National Conference on Artificial Intelligence (AAAI-05), pp. 1099–1105, Pittsburgh, PA, USA. AAAI Press. find
  219. Rajpurkar, P., Jia, R., & Liang, P. (2018). Know What You Don’t Know: Unanswerable Questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018), Melbourne, Australia. Association for Computational Linguistics. find our card
  220. Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016). SQuAD: 100,000+ Questions for Machine Comprehension of Text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP 2016), Austin, TX, USA. Association for Computational Linguistics. find our card
  221. Rashkin, H., Bosselut, A., Sap, M., Knight, K., & Choi, Y. (2018a). Modeling Naive Psychology of Characters in Simple Commonsense Stories. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018), Melbourne, Australia. Association for Computational Linguistics. find
  222. Rashkin, H., Sap, M., Allaway, E., Smith, N. A., & Choi, Y. (2018b). Event2Mind: Commonsense Inference on Events, Intents, and Reactions. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018), Melbourne, Australia. Association for Computational Linguistics. find
  223. Reddy, S., Chen, D., & Manning, C. D. (2018). CoQA: A Conversational Question Answering Challenge. arXiv: 1808.07042. ↗ our card
  224. Reiss, S. (2004). Multifaceted Nature of Intrinsic Motivation: The Theory of 16 Basic Desires. Review of General Psychology, 8(3), 179–193. find
  225. Richardson, M., Burges, C. J. C., & Renshaw, E. (2013). MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP 2013), p. 11, Seattle, WA, USA. Association for Computational Linguistics. find our card
  226. Rodosthenous, C., & Michael, L. (2016). A Hybrid Approach to Commonsense Knowledge Acquisition. In Proceedings of the 8th European Starting AI Researcher Symposium, The Hague, the Netherlands. find
  227. Roemmele, M., Bejan, C. A., & Gordon, A. S. (2011). Choice of Plausible Alternatives: An Evaluation of Commonsense Causal Reasoning. In AAAI Spring Symposium on Logical Formalizations of Commonsense Reasoning, p. 6, Stanford, CA, USA. find
  228. Roemmele, M., & Gordon, A. (2018). An Encoder-decoder Approach to Predicting Causal Relations in Stories. In Proceedings of the First Workshop on Storytelling, pp. 50–59, New Orleans, LA, USA. Association for Computational Linguistics. find
  229. Rogers, A. (2019). How the Transformers broke NLP leaderboards.. find
  230. Rohrbach, A., Torabi, A., Rohrbach, M., Tandon, N., Pal, C., Larochelle, H., Courville, A., & Schiele, B. (2017). Movie Description. Int. J. Comput. Vision, 123(1), 94–120. find
  231. Rudinger, R., Naradowsky, J., Leonard, B., & Van Durme, B. (2018a). Gender Bias in Coreference Resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2018), pp. 8–14, New Orleans, LA, USA. Association for Computational Linguistics. find our card
  232. Rudinger, R., White, A. S., & Van Durme, B. (2018b). Neural Models of Factuality. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2018), pp. 731–744, New Orleans, LA, USA. Association for Computational Linguistics. find our card
  233. Saito, I., Nishida, K., Asano, H., & Tomita, J. (2018). Commonsense Knowledge Base Completion and Generation. In Proceedings of the 22nd Conference on Computational Natural Language Learning, pp. 141–150. Association for Computational Linguistics. find
  234. Sakaguchi, K., Bras, R. L., Bhagavatula, C., & Choi, Y. (2019). WINOGRANDE: An adversarial winograd schema challenge at scale. arXiv: 1907.10641. ↗
  235. Samardžić, T., & Merlo, P. (2016). Aspect-based Learning of Event Duration Using Parallel Corpora.. find
  236. Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019). DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. In Proceedings of the 5th Workshop on Energy Efficient Machine Learning and Cognitive Computing at NeurIPS 2019, Vancouver, BC, Canada. find
  237. Sap, M., LeBras, R., Allaway, E., Bhagavatula, C., Lourie, N., Rashkin, H., Roof, B., Smith, N. A., & Choi, Y. (2019a). ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning. In Proceedings of the Thirty-Third AAAI Conference on Artificial Inteligence (AAAI-19), Honolulu, HI, USA. AAAI Press. find
  238. Sap, M., Rashkin, H., Chen, D., LeBras, R., & Choi, Y. (2019b). SocialIQA: Commonsense Reasoning about Social Interactions. arXiv: 1904.09728. ↗ our card
  239. Schank, R. C., & Abelson, R. P. (1988). Scripts, Plans, Goals, and Understanding. Readings in Cognitive Science: A Perspective from Psychology and Artificial Intelligence. Morgan Kaufmann, San Mateo, CA, US. find
  240. Schmitt, M., & Schütze, H. (2019). SherLIiC: A typed event-focused lexical inference benchmark for evaluating natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), pp. 902–914, Florence, Italy. Association for Computational Linguistics. find our card
  241. Schoenmackers, S., Etzioni, O., Weld, D. S., & Davis, J. (2010). Learning First-order Horn Clauses from Web Text. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing (EMNLP 2010), EMNLP ’10, pp. 1088–1098, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  242. Schuler, K. K. (2005). VerbNet: A Broad-Coverage, Comprehensive Verb Lexicon. Dissertation, University of Pennsylvania. find
  243. Schwartz, R., Dodge, J., Smith, N. A., & Etzioni, O. (2019). Green AI. arXiv: 1907.10597. ↗ our card
  244. Schwartz, R., Sap, M., Konstas, I., Zilles, L., Choi, Y., & Smith, N. A. (2017). The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task. In Proceedings of the 21st Conference on Computational Natural Language (CoNLL 2017), Vancouver, BC, Canada. Association for Computational Linguistics. find
  245. Selman, B., & Levesque, H. J. (1990). Abductive and Default Reasoning: A Computational Core. In Proceedings of the Eighth National Conference on Artificial Intelligence (AAAI ’90), AAAI’90, pp. 343–348, Boston, MA, USA. AAAI Press. find
  246. Seo, M., Min, S., Farhadi, A., & Hajishirzi, H. (2017). Query-Reduction Networks for Question Answering. In Proceedings of the 5th International Conference on Learning Representations (ICLR 2017), Toulon, France. find
  247. Sharma, R., Allen, J., Bakhshandeh, O., & Mostafazadeh, N. (2018). Tackling the Story Ending Biases in The Story Cloze Test. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018), pp. 752–757, Melbourne, Australia. Association for Computational Linguistics. find
  248. Singh, P. (2002). The Public Acquisition of Commonsense Knowledge. In AAAI Spring Symposium Series, Palo Alto, CA, USA. find
  249. Socher, R., Perelygin, A., Wu, J. Y., Chuang, J., Manning, C. D., Ng, A. Y., & Potts, C. (2013). Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP 2013), p. 12, Seattle, WA, USA. Association for Computational Linguistics. find
  250. Song, H.-j., Baillargeon, R., & Fisher, C. (2005). Can infants attribute to an agent a disposition to perform a particular action?. Cognition, 98(2), B45–B55. find
  251. Speer, R., Chin, J., & Havasi, C. (2017). ConceptNet 5.5: An Open Multilingual Graph of General Knowledge.. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17) and the Twenty-Ninth Innovative Applications of Artificial Intelligence Conference (IAAI-17), San Francisco, CA, USA. AAAI Press. find
  252. Speer, R., Havasi, C., & Lieberman, H. (2008). AnalogySpace: Reducing the Dimensionality of Common Sense Knowledge. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (AAAI-08), Chicago, IL, USA. find
  253. Spithourakis, G., Augenstein, I., & Riedel, S. (2016). Numerically Grounded Language Models for Semantic Error Correction. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP 2016), pp. 987–992, Austin, Texas. Association for Computational Linguistics. find
  254. Spithourakis, G., & Riedel, S. (2018). Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numbers. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018), pp. 2104–2115, Melbourne, Australia. Association for Computational Linguistics. find
  255. Strubell, E., Ganesh, A., & McCallum, A. (2019). Energy and Policy Considerations for Deep Learning in NLP. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), pp. 3645–3650, Florence, Italy. Association for Computational Linguistics. find our card
  256. Suchanek, F. M., Kasneci, G., & Weikum, G. (2007). YAGO: A Core of Semantic Knowledge Unifying WordNet and Wikipedia. In Proceedings of the 16th International Conference on World Wide Web, p. 10, Amherst, MA, USA. ACM. find
  257. Sugawara, S., Yokono, H., & Aizawa, A. (2017). Prerequisite Skills for Reading Comprehension: Multi-Perspective Analysis of MCTest Datasets and Systems. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17), San Francisco, CA, USA. AAAI Press. find
  258. Suhr, A., Lewis, M., Yeh, J., & Artzi, Y. (2017). A Corpus of Natural Language for Visual Reasoning. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL 2017), pp. 217–223. Association for Computational Linguistics. find
  259. Sukhbaatar, S., Szlam, A., Weston, J., & Fergus, R. (2015). End-To-End Memory Networks. In Cortes, C., Lawrence, N. D., Lee, D. D., Sugiyama, M., & Garnett, R. (Eds.), Advances in Neural Information Processing Systems 28 (NIPS 2015), pp. 2440–2448. Curran Associates, Inc. find
  260. Sun, K., Yu, D., Chen, J., Yu, D., Choi, Y., & Cardie, C. (2019). DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehension. Transactions of the Association for Computational Linguistics, 7(0), 217–231. find
  261. Sundheim, B. M. (1993). Tipster/MUC-5: Information extraction system evaluation. In Proceedings of the 5th Conference on Message Understanding, MUC5 ’93, pp. 27–44, Baltimore, MD, USA. Association for Computational Linguistics. find
  262. Talmor, A., Herzig, J., Lourie, N., & Berant, J. (2019). CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge. In Proceedings of the 17th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019), Minneapolis, MN, USA. Association for Computational Linguistics. find
  263. Tandon, N., de Melo, G., Suchanek, F., & Weikum, G. (2014a). WebChild: Harvesting and organizing commonsense knowledge from the web. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining (WSDM ’14), pp. 523–532, New York, NY, USA. ACM. find
  264. Tandon, N., De Melo, G., & Weikum, G. (2014b). Acquiring Comparative Commonsense Knowledge from the Web. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI-14), AAAI’14, pp. 166–172, Québec City, QC, Canada. AAAI Press. find
  265. Tandon, N., de Melo, G., & Weikum, G. (2017). WebChild 2.0 : Fine-Grained Commonsense Knowledge Distillation. In Proceedings of ACL 2017, System Demonstrations, pp. 115–120, Vancouver, BC, Canada. Association for Computational Linguistics. find
  266. Tandon, N., Mishra, B. D., Grus, J., Yih, W.-t., Bosselut, A., & Clark, P. (2018). Reasoning about Actions and State Changes by Injecting Commonsense Knowledge. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), Brussels, Belgium. Association for Computational Linguistics. find
  267. Taylor, A., Marcus, M., & Santorini, B. (2003). The Penn Treebank: An Overview. In Treebanks, Vol. 20 of Text, Speech and Language Technology. Springer, Dordrecht. find
  268. Taylor, W. L. (1953). “Cloze Procedure”: A New Tool for Measuring Readability. Journalism Bulletin, 30(4), 415–433. find
  269. Tjong Kim Sang, E. F., & De Meulder, F. (2003). Introduction to the CoNLL-2003 Shared Task: Language-independent Named Entity Recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, CONLL ’03, pp. 142–147, Edmonton, AB, Canada. Association for Computational Linguistics. find
  270. Trichelair, P., Emami, A., Trischler, A., Suleman, K., & Cheung, J. C. K. (2018). How reasonable are common-sense reasoning tasks: A case-study on the winograd schema challenge and SWAG. arXiv: 1811.01778. ↗
  271. Tsuchida, M., & Ishikawa, K. (2011). IKOMA at TAC2011: A Method for Recognizing Textual Entailment using Lexical-level and Sentence Structure-level features. In Proceedings of the Fourth Text Analysis Conference (TAC 2011), Gaithersburg, MD, USA. National Institute of Standards and Technology. find
  272. Turing, A. M. (1950). Computing machinery and intelligence. Mind, 49, 433–460. find
  273. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is All you Need. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., & Garnett, R. (Eds.), Advances in Neural Information Processing Systems 30, pp. 5998–6008. Curran Associates, Inc. find
  274. Voorhees, E. M. (2000). The TREC-8 Question Answering Track. Natural Language Engineering, 7, 361–378. find
  275. Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2019). SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems. arXiv: 1905.00537. ↗ our card
  276. Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2018). GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Brussels, Belgium. Association for Computational Linguistics. find our card
  277. Wang, Z., Zhang, Y., & Chang, C.-Y. (2017). Integrating Order Information and Event Relation for Script Event Prediction. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP 2017), pp. 57–67, Copenhagen, Denmark. Association for Computational Linguistics. find
  278. Warstadt, A., Singh, A., & Bowman, S. R. (2018). Neural Network Acceptability Judgments. arXiv: 1805.12471. ↗
  279. Weston, J., Bordes, A., Chopra, S., Rush, A. M., van Merriënboer, B., Joulin, A., & Mikolov, T. (2016). Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks. In Proceedings of the 4th International Conference on Learning Representations (ICLR 2016), San Juan, Puerto Rico. find our card
  280. Weston, J., Chopra, S., & Bordes, A. (2015). Memory Networks. In Proceedings of the 3rd International Conference on Learning Representations (ICLR 2015), San Diego, CA, USA. find
  281. White, A. S., Rastogi, P., Duh, K., & Van Durme, B. (2017). Inference is Everything: Recasting Semantic Resources into a Unified Evaluation Framework. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (IJCNLP 2017), pp. 996– 1005, Taipei, Taiwan. Asian Federation of Natural Language Processing. find our card
  282. White, A. S., & Rawlins, K. (2018). The role of veridicality and factivity in clause selection. In Proceedings of the 48th Annual Meeting of the North East Linguistic Society, Amherst, MA, USA. GLSA Publications. find
  283. Williams, A., Nangia, N., & Bowman, S. R. (2017). A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2018), New Orleans, LA, USA. Association for Computational Linguistics. find
  284. Winn, O., & Muresan, S. (2018). ‘Lighter’ Can Still Be Dark: Modeling Comparative Color Descriptions. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018), pp. 790–795, Melbourne, Australia. Association for Computational Linguistics. find
  285. Winograd, T. (1972). Understanding Natural Language. Academic Press, New York. find
  286. Woods, W. A. (1975). What’s in a Link: Foundations for Semantic Networks. Tech. rep. BBN-3072, Bolt Beranek and Newman Inc., Cambridge, MA, USA. find
  287. Wu, W., Li, H., Wang, H., & Zhu, K. Q. (2011). Towards a Probabilistic Taxonomy of Many Concepts. http://www.microsoft.com/en-us/research/wp-content/uploads/2011/03/paper-2.pdf. ↗
  288. Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., & others (2016). Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. arXiv: 1609.08144. ↗
  289. Xie, Q., Lai, G., Dai, Z., & Hovy, E. (2017). Large-scale Cloze Test Dataset Created by Teachers. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP 2017), Copenhagen, Denmark. Association for Computational Linguistics. find
  290. Xu, F. F., Lin, B. Y., & Zhu, K. (2018a). Automatic Extraction of Commonsense LocatedNear Knowledge. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018), pp. 96–101, Melbourne, Australia. Association for Computational Linguistics. find
  291. Xu, J., Zhou, H., Young, T., Zhao, H., Huang, M., & Zhu, X. (2018b). Commonsense Knowledge Aware Conversation Generation with Graph Attention. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI 2018), pp. 4623–4629, Stockholm, Sweden. IJCAI. find
  292. Yang, D., Lavie, A., Dyer, C., & Hovy, E. (2015). Humor Recognition and Humor Anchor Extraction. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 2367–2376, Lisbon, Portugal. Association for Computational Linguistics. find
  293. Yang, S., Gao, Q., Sadiya, S., & Chai, J. (2018a). Commonsense Justification for Action Explanation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), pp. 2627–2637, Brussels, Belgium. Association for Computational Linguistics. find
  294. Yang, Y., Birnbaum, L., Wang, J.-P., & Downey, D. (2018b). Extracting Commonsense Properties from Embeddings with Limited Human Guidance. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018), pp. 644–649, Melbourne, Australia. Association for Computational Linguistics. find
  295. Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019). XLNet: Generalized Autoregressive Pretraining for Language Understanding. arXiv: 1906.08237. ↗ our card
  296. Yatskar, M. (2018). A Qualitative Comparison of CoQA, SQuAD 2.0 and QuAC. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2019), Minneapolis, MN, USA. Association for Computational Linguistics. find
  297. Zadeh, L. A. (1975). The concept of a linguistic variable and its application to approximate reasoning—I. Information Sciences, 8(3), 199–249. find
  298. Zang, L.-J., Cao, C., Cao, Y.-N., Wu, Y.-M., & CAO, C.-G. (2013). A Survey of Commonsense Knowledge Acquisition. Journal of Computer Science and Technology, 28(4), 689–719. find
  299. Zellers, R. (2019). Why We Released Grover.. find
  300. Zellers, R., Bisk, Y., Farhadi, A., & Choi, Y. (2019). From Recognition to Cognition: Visual Commonsense Reasoning. In Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019), Long Beach, CA, USA. IEEE. find
  301. Zellers, R., Bisk, Y., Schwartz, R., & Choi, Y. (2018). SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), Brussels, Belgium. Association for Computational Linguistics. find
  302. Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., & Choi, Y. (2019a). HellaSwag: Can a Machine Really Finish Your Sentence?. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. find
  303. Zellers, R., Holtzman, A., Rashkin, H., Bisk, Y., Farhadi, A., Roesner, F., & Choi, Y. (2019b). Defending Against Neural Fake News. arXiv: 1905.12616. ↗ our card
  304. Zhang, C., & Chai, J. (2009). What do We Know about Conversation Participants: Experiments on Conversation Entailment. In Proceedings of the SIGDIAL 2009 Conference, pp. 206–215. Association for Computational Linguistics. find
  305. Zhang, C., & Chai, J. (2010). Towards Conversation Entailment: An Empirical Investigation. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pp. 756–766. Association for Computational Linguistics. find
  306. Zhang, H., Liu, X., Pan, H., Song, Y., Wing-Ki, C., & Leung (2019). ASER: A Large-scale Eventuality Knowledge Graph. arXiv: 1905.00270. ↗
  307. Zhang, S., Liu, X., Liu, J., Gao, J., Duh, K., & Van Durme, B. (2018). ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension. arXiv: 1810.12885. ↗ our card
  308. Zhang, S., Rudinger, R., Duh, K., & Van Durme, B. (2017). Ordinal Common-sense Inference. Transactions of the Association for Computational Linguistics, 5, 379–395. find our card
  309. Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., & Liu, Q. (2019). ERNIE: Enhanced Language Representation with Informative Entities. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), pp. 1441–1451, Florence, Italy. Association for Computational Linguistics. find
  310. Zhou, B., Khashabi, D., Ning, Q., & Roth, D. (2019). "Going on a vacation" takes longer than "Going for a walk": A Study of Temporal Commonsense Understanding. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP 2019), Hong Kong, China. Association for Computational Linguistics. find
  311. Zhu, Y., Groth, O., Bernstein, M., & Fei-Fei, L. (2016). Visual7W: Grounded Question Answering in Images. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016), Las Vegas, NV, USA. IEEE. find our card
  312. Zhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., & Fidler, S. (2015). Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books. In IEEE International Conference on Compter Vision (ICCV 2015), pp. 19–27, Santiago, Chile. IEEE. find our card