Elamur Research
☰
home / papers / 2020 / Experience Grounds Language

Experience Grounds Language

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2020.emnlp-main.703.pdf

Summary

This position paper argues that progress in language understanding is limited because models learn almost entirely from text, cut off from the physical world and the social interaction that give utterances meaning. The authors draw on NLP, cognitive science, linguistics, and developmental ideas to propose "World Scope" (WS), a five-level framework for auditing how much context a learner can access: Corpus, Internet, Perception, Embodiment, and Social. They trace the field's history from curated corpora such as the Penn Treebank to web-scale pretraining. Scaling text-only models has produced strong benchmark results but with shrinking gains, and these models still struggle with inferences that depend on everyday experience. The authors conclude that most current NLP sits at WS2 and is therefore pursuing a goal other than genuine language learning. They urge the community to move deliberately toward grounding, embodiment, and social agency. The framework matters because it offers a roadmap and a vocabulary for judging whether new tasks and models actually broaden the context available to language learners.

Key points

  • The five World Scopes are WS1 Corpus, WS2 Internet, WS3 Perception (multimodal), WS4 Embodiment, and WS5 Social. Most present-day NLP operates at WS2.
  • The levels have operational boundaries. A learner reaches WS4 only if its possible actions and their consequences cannot be enumerated. It reaches WS5 only if achieving its goals depends on cooperating with a human in the loop.
  • The paper illustrates diminishing returns on LAMBADA:
    • GPT2 (1.5B), Megatron-LM (8.3B), and TuringNLG (17B) score within a few points of one another, all below 68%.
    • The 175B model of Brown et al. (2020) gains roughly 8 points but still leaves about 25% unsolved.
  • Data and model scale have grown dramatically. Training data rose from 1.6 billion tokens (Mikolov et al., 2013) to 840 billion Common Crawl tokens (Pennington et al., 2014). Model size grew from about 10^8 parameters (ELMo) to about 10^11 (Brown et al., 2020).
  • The authors argue that mature computer vision and speech recognition, commodity robotics, and simulators or videogames now make research at WS3, WS4, and WS5 feasible.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

155 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 148 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.emnlp-main.703

@inproceedings{bisk2020experience,
  title = {{Experience Grounds Language}},
  author = {Bisk, Yonatan and Holtzman, Ari and Thomason, Jesse and Andreas, Jacob and Bengio, Yoshua and Chai, Joyce and Lapata, Mirella and Lazaridou, Angeliki and May, Jonathan and Nisnevich, Aleksandr and Pinto, Nicolas and Turian, Joseph},
  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.703},
  url = {https://doi.org/10.18653/v1/2020.emnlp-main.703}
}

References

216 references, 34 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, et al. 2020. Towards a human-like open-domain chatbot. arXiv preprint arXiv:2001.09977. find our card
  2. Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. 2017. Bottom-up and top-down attention for image captioning and visual question answering. Visual Question Answering Challenge at CVPR 2017. find
  3. Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton van den Hengel. 2018. Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of the IEEE Conference on Computer Visi… find our card
  4. Jacob Andreas and Dan Klein. 2016. Reasoning about pragmatics with neural listeners and speakers. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1173–1182, Austin, Texas. find our card
  5. Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015. Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision, pages 2425–2433. find
  6. John Langshaw Austin. 1975. How to do things with words. Oxford university press. find
  7. Philip Bachman, R Devon Hjelm, and William Buchwalter. 2019. Learning representations by maximizing mutual information across views. In Advances in Neural Information Processing Systems 32. find
  8. Anton Bakhtin, Laurens van der Maaten, Justin Johnson, Laura Gustafson, and Ross Girshick. 2019. Phyre: A new benchmark for physical reasoning. In Advances in Neural Information Processing Systems 32 (NIPS 2019). find
  9. Dare A. Baldwin, Ellen M. Markman, Brigitte Bill, Renee N. Desjardins, Jane M. Irwin, and Glynnis Tid-ball. 1996. Infants’ reliance on a social criterion for establishing word-object relations. Child Development, 67(6):3135–3153. find
  10. H.B. Barlow. 1989. Unsupervised learning. Neural Computation, 1(3):295–311. find
  11. Marco Baroni, Silvia Bernardini, Adriano Ferraresi, and Eros Zanchetta. 2009. The wacky wide web: a collection of very large linguistically processed web-crawled corpora. Language resources and evaluation, 43(3):209–226. find
  12. Rachel Barr. 2013. Memory constraints on infant learning from picture books, television, and touchscreens. Child Development Perspectives, 7(4):205–210. find
  13. Lawrence W Barsalou. 2008. Grounded cognition. Annu. Rev. Psychol., 59:617–645. find
  14. Emily M Bender and Alexander Koller. 2020. Climbing towards nlu: On meaning, form, and understanding in the age of data. In Association for Computational Linguistics (ACL). find our card
  15. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137–1155. find
  16. Leon Bergen, Roger Levy, and Noah Goodman. 2016. Pragmatic reasoning through semantic inference. Semantics and Pragmatics, 9. find
  17. Yonatan Bisk, Kevin Shih, Yejin Choi, and Daniel Marcu. 2018. Learning Interpretable Spatial Operations in a Rich 3D Blocks World . In Proceedings of the Thirty-Second Conference on Artificial Intelligence (AAAI-18). find
  18. Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. 2020. PIQA: Reasoning about physical commonsense in natural language. In Thirty-Fourth AAAI Conference on Artificial Intelligence. find our card
  19. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent dirichlet allocation. Journal of Machine Learning Research, 3:993–1022. find our card
  20. Paul Bloom. 2002. How children learn the meanings of words. MIT press. find
  21. Valts Blukis, Yannick Terme, Eyvind Niklasson, Ross A. Knepper, and Yoav Artzi. 2019. Learning to map natural language instructions to physical quadcopter control using simulated flight. In 3rd Conference on Robot Learning (CoRL). find
  22. Peter F Brown, Peter V deSouza, Robert L Mercer, Vincent J Della Pietra, and Jenifer C Lai. 1992. Class-based n-gram models of natural language. Computational Linguistics, 18. find
  23. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… find
  24. Elia Bruni, Gemma Boleda, Marco Baroni, and Nam-Khanh Tran. 2012. Distributional semantics in technicolor. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 136–145, Jeju Island, Korea. find
  25. Elia Bruni, Nam Khanh Tran, and Marco Baroni. 2014. Multimodal distributional semantics. Journal of Artificial Intelligence Research, 49:1–47. find
  26. Alexandre Campeau-Lecours, Hugo Lamontagne, Simon Latour, Philippe Fauteux, Véronique Maheu, François Boucher, Charles Deguire, and Louis-Joseph Caron L’Ecuyer. 2019. Kinova modular robot arms for service robotics applications. In Rapid Automation: Concepts, Methodologies, Tools, and Applications, p… find
  27. Rahma Chaabouni, Eugene Kharitonov, Diane Bouchacourt, Emmanuel Dupoux, and Marco Baroni. 2020. Compositionality and generalization in emergent languages. In Association for Computational Linguistics (ACL). find
  28. Joyce Y. Chai, Rui Fang, Changsong Liu, and Lanbo She. 2017. Collaborative language grounding toward situated human-robot dialogue. AI Magazine, 37(4):32–45. find
  29. Joyce Y. Chai, Qiaozi Gao, Lanbo She, Shaohua Yang, Sari Saba-Sadiya, and Guangyue Xu. 2018. Language to action: Towards interactive task learning with physical agents. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18). find
  30. Soravit Changpinyo, Wei-Lun Chao, and Fei Sha. 2017. Predicting visual exemplars of unseen classes for zero-shot learning. In ICCV. find
  31. Wei-Lun Chao, Soravit Changpinyo, Boqing Gong, and Fei Sha. 2016. An empirical study and analysis of generalized zero-shot learning for object recognition in the wild. In ECCV, pages 52–68, Cham. Springer International Publishing. find
  32. Eugene Charniak. 1977. A framed painting: The representation of a common sense knowledge fragment. Cognitive Science, 1(4):355–394. find
  33. Chun-Yen Chen, Dian Yu, Weiming Wen, Yi Mang Yang, Jiaping Zhang, Mingyang Zhou, Kevin Jesse, Austin Chau, Antara Bhowmick, Shreenath Iyer, et al. 2018. Gunrock: Building a human-like social bot by leveraging large scale real user data. Alexa Prize Proceedings. find
  34. Noam Chomsky. 1965. Aspects of the Theory of Syntax. MIT Press. find
  35. Noam Chomsky. 1980. Language and learning: the debate between Jean Piaget and Noam Chomsky. Harvard University Press. find
  36. Noam Chomsky. 1981. Lectures on Government and Binding. Mouton de Gruyter. find
  37. Kenneth Church. 2007. A pendulum swung too far. Linguistic Issues in Language Technology – LiLT, 2. find
  38. L. Coleman and P. Kay. 1981. The english word “lie". Linguistics, 57. find
  39. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: deep neural networks with multitask learning. In ICML. find
  40. Marc-Alexandre Côté, Ákos Kádár, Xingdi Yuan, Ben Kybartas, Tavian Barnes, Emery Fine, James Moore, Ruo Yu Tao, Matthew Hausknecht, Layla El Asri, Mahmoud Adada, Wendy Tay, and Adam Trischler. 2018. Textworld: A learning environment for text-based games. ArXiv, abs/1806.11532. find our card
  41. Erwin Coumans and Yunfei Bai. 2016–2019. Pybullet, a python module for physics simulation for games, robotics and machine learning. http://pybullet. org . find
  42. Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra. 2018. Embodied question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 2054–2063. find
  43. Scott Deerwester, Susan T. Dumais, George W. Furnas, Thomas K. Landauer, and Richard Harshman. 1988. Improving information retrieval with latent semantic indexing. In Proceedings of the 51st Annual Meeting of the American Society for Information Science 25, pages 36 – 40. find
  44. Scott Deerwester, Susan T. Dumais, George W. Furnas, Thomas K. Landauer, and Richard Harshman. 1990. Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6):391–407. find
  45. David L. Chen and Raymond J. Mooney. 2008. Learning to sportscast: A test of grounded language acquisition. In Proceedings of the 25th International Conference on Machine Learning (ICML), Helsinki, Finland. find
  46. Gerald Dejong. 1981. Generalizations based on explanations. In Proceedings of the 7th international joint conference on Artificial intelligence (IJCAI). find
  47. SF Chen and Joshua Goodman. 1996. An empirical study of smoothing techniques for language modeling. In Association for Computational Linguistics, pages 310–318. find
  48. David DeVault, Iris Oved, and Matthew Stone. 2006. Societal grounding is essential to meaningful language use. In Proceedings of the National Conference on Artificial Intelligence, volume 21, page 747. find
  49. Maxime Chevalier-Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen, and Yoshua Bengio. 2019. Babyai: First steps towards grounded language learning with a human in the loop. In ICLR’2019. find
  50. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In North American Chapter of the Association for Computational Linguistics (NAACL). find
  51. Emily Dinan, Samuel Humeau, Bharath Chintagunta, and Jason Weston. 2019. Build it break it fix it for dialogue safety: Robustness from adversarial human attack. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Nat… find our card
  52. Susan T. Dumais. 2004. Latent semantic analysis. Annual Review of Information Science and Technology, 38(1):188–230. find
  53. Robin IM Dunbar. 1993. Coevolution of neocortical size, group size and language in humans. Behavioral and brain sciences, 16(4):681–694. find
  54. Yanai Elazar, Abhijit Mahabal, Deepak Ramachandran, Tania Bedrax-Weiss, and Dan Roth. 2019. How large are lions? inducing distributions over quantitative attributes. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3973–3983. find
  55. Desmond Elliott, Stella Frank, Khalil Sima’an, and Lucia Specia. 2016. Multi30k: Multilingual english-german image descriptions. In Workshop on Vision and Langauge at ACL ’16. find our card
  56. J Elman. 1990. Finding structure in time. Cognitive Science, 14(2):179–211. find
  57. Katrin Erk and Sebastian Padó. 2008. A structured vector space model for word meaning in context. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 897–906, Honolulu, Hawaii. find
  58. Susan Ervin-Tripp. 1973. Some strategies for the first two years. In Timothy E. Moore, editor, Cognitive Development and Acquisition of Language, pages 261 – 286. Academic Press, San Diego. find
  59. Ali Farhadi, M Hejrati, M Sadeghi, Peter Young, Cyrus Rashtchian, Julia Hockenmaier, and David Forsyth. 2010. Every picture tells a story: Generating sentences from images. In European Conference on Computer Vision. Springer. find
  60. Michael C Frank and Noah D Goodman. 2012. Predicting pragmatic reasoning in language games. Science, 336(6084):998–998. find our card
  61. Ruiji Fu, Jiang Guo, Bing Qin, Wanxiang Che, Haifeng Wang, and Ting Liu. 2014. Learning semantic hierarchies via word embeddings. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1199–1209. find
  62. Matt Gardner, Yoav Artzi, Victoria Basmova, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hanna Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noa… find our card
  63. Max Glockner, Vered Shwartz, and Yoav Goldberg. 2018. Breaking nli systems with sentences that require simple lexical inferences. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 650–655. find our card
  64. I J Good. 1953. The population frequencies of species and the estimation of population parameters. Biometrika, 40:237–264. find
  65. Herbert P Grice. 1975. Logic and conversation. In Speech acts, pages 41–58. Brill. find
  66. Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A Smith. 2018. Annotation artifacts in natural language inference data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technol… find
  67. Stevan Harnad. 1990. The symbol grounding problem. Physica D, 42:335–346. find
  68. Yansong Feng and Mirella Lapata. 2010. Topic models for image annotation and text illustration. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 831–839, Los Angeles, California. find
  69. Zellig S Harris. 1954. Distributional structure. Word, 10:146–162. find
  70. J. R. Firth. 1957. A synopsis of linguistic theory, 1930-1955. Studies in Linguistic Analysis. find
  71. David Harwath, Adrià Recasens, Dídac Surís, Galen Chuang, Antonio Torralba, and James Glass. 2019. Jointly discovering visual objects and spoken words from raw sensory input. International Journal of Computer Vision. find
  72. Cliff Fitzgerald. 2013. Developing baxter. In 2013 IEEE Conference on Technologies for Practical Robot Applications (TePRA). find
  73. W. Nelson Francis. 1964. A standard sample of present-day english for use with digital computers. Report to the U.S Office of Education on Cooperative Research Project No. E-007. find
  74. David Harwath, Wei-Ning Hsu, and James Glass. 2020. Learning hierarchical discrete linguistic units from visually-grounded speech. In ICLR 2020. find
  75. He He, Derek Chen, Anusha Balakrishnan, and Percy Liang. 2018. Decoupling strategy and generation in negotiation dialogues. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2333–2343. find
  76. Susan J. Hespos and Elizabeth S. Spelke. 2004. Conceptual precursors to language. Nature, 430. find
  77. G. E. Hinton, J. L. McClelland, and D. E. Rumelhart. 1986. Distributed representations. Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 1: Foundations. find
  78. Geoffrey E. Hinton. 1990. Preface to the special issue on connectionist symbol processing. Artificial Intelligence, 46(1):1 – 4. find
  79. Julia Hirschberg and Christopher D Manning. 2015. Advances in natural language processing. Science, 349(6245):261–266. find
  80. Douglas Hofstadter and Emmanuel Sander. 2013. Surfaces and essences: Analogy as the fuel and fire of thinking. Basic Books. find
  81. Ari Holtzman, Jan Buys, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. In ICLR 2020. find
  82. Po-Yao Huang, Junjie Hu, Xiaojun Chang, and Alexander Hauptmann. 2020. Unsupervised multimodal neural machine translation with pseudo visual pivoting. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8226–8237, Online. find
  83. Gabriel Ilharco, Yuan Zhang, and Jason Baldridge. 2019. Large-scale representation learning from visually grounded untranscribed speech. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 55–65, Hong Kong, China. find
  84. Daniel L. James and Risto Miikkulainen. 1995. Sard-net: A self-organizing feature map for sequences. In Advances in Neural Information Processing Systems 7 (NIPS’94), pages 577–584, Denver, CO. Cambridge, MA: MIT Press. find
  85. Michael I Jordan. 2019. Artificial intelligence – the revolution hasn’t happened yet. Harvard Data Science Review. find
  86. Divyansh Kaushik, Eduard Hovy, and Zachary Lipton. 2020. Learning the difference that makes a difference with counterfactually-augmented data. In International Conference on Learning Representations. find
  87. Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019. CTRL: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858. find our card
  88. Reinhard Kneser and Hermann Ney. 1995. Improved backing-off for m-gram language modeling. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. find
  89. Teuvo Kohonen. 1984. Self-Organization and Associative Memory. Springer. find
  90. Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, Michael S. Bernstein, and Fei-Fei Li. 2017. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International Journa… find our card
  91. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, pages 1097–1105. Curran Associates, Inc. find
  92. George Lakoff. 1973. Hedges: A study in meaning criteria and the logic of fuzzy concepts. Journal of Philosophical Logic, 2:458–508. find our card
  93. George Lakoff. 1980. Metaphors We Live By. University of Chicago Press. find
  94. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. Albert: A lite bert for self-supervised learning of language representations. In International Conference on Learning Representations. find
  95. Ronald W Langacker. 1987. Foundations of cognitive grammar: Theoretical prerequisites, volume 1. Stanford university press. find
  96. Ronald W Langacker. 1991. Foundations of Cognitive Grammar: descriptive application., volume 2. Stanford university press. find
  97. Angeliki Lazaridou, Karl Moritz Hermann, Karl Tuyls, and Stephen Clark. 2018. Emergence of linguistic communication from referential games with symbolic and pixel input. In Internationl Conference on Learning Representations. find
  98. Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni. 2017. Multi-agent cooperation and the emergence of (natural) language. In ICLR 2017. find
  99. Angeliki Lazaridou, Nghia The Pham, and Marco Baroni. 2016. The red one!: On learning to refer to things based on discriminative properties. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 213–218, Berlin, Germany. find
  100. Matthew Le, Y-Lan Boureau, and Maximilian Nickel. 2019. Revisiting the evaluation of theory of mind through question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-I… find
  101. Mike Lewis, Denis Yarats, Yann Dauphin, Devi Parikh, and Dhruv Batra. 2017. Deal or no deal? end-to-end learning of negotiation dialogues. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2443–2453. find
  102. Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. 2019a. VisualBERT: A Simple and Performant Baseline for Vision and Language. In Work in Progress. find our card
  103. Yong-Lu Li, Liang Xu, Xinpeng Liu, Xijie Huang, Yue Xu, Mingyang Chen, Ze Ma, Shiyi Wang, Hao-Shu Fang, and Cewu Lu. 2019b. HAKE: Human Activity Knowledge Engine. arXiv:1904.06539. — [arXiv:1904.06539](https://arxiv.org/abs/1904.06539) find
  104. Ling-Yi Lin, Rong-Ju Cherng, and Yung-Jung Chen. 2017. Effect of touch screen tablet use on fine motor development of young children. Physical & Occupational Therapy In Pediatrics, 37(5):457–467. PMID: 28071977. find
  105. Tal Linzen. 2020. How can we accelerate progress towards human-like linguistic generalization? In Association for Computational Linguistics (ACL). find our card
  106. Changsong Liu and Joyce Yue Chai. 2015. Learning to mediate perceptual differences in situated human-robot dialogue. In Proceedings of the 29th AAAI Conference on Artificial Intelligence, pages 2288– 2294. find
  107. Chia-Wei Liu, Ryan Lowe, Iulian Serban, Mike Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural L… find
  108. Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV). find
  109. Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. In Advances in Neural Information Processing Systems, pages 13–23. find our card
  110. Li Lucy and Jon Gauthier. 2017. Are distributional representations ready for the real world? evaluating word vectors for grounded perceptual meaning. In Proceedings of the First Workshop on Language Grounding for Robotics, pages 76–85, Vancouver, Canada. Association for Computational Linguistics. find
  111. Matt MacMahon, Brian Stankiewicz, and Benjamin Kuipers. 2006. Walk the talk: Connecting language, knowledge, and action in route instructions. In Proceedings of the 21st National Conference on Artificial Intelligence (AAAI-2006), Boston, MA, USA. find
  112. Gary Marcus and Ernest Davis. 2019. Rebooting AI: Building Artificial Intelligence We Can Trust. Pantheon. find
  113. Mitchell P Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of english: The penn treebank. Computational Linguistics, 19:313–330. find our card
  114. Lara J Martin, Prithviraj Ammanabrolu, Xinyu Wang, William Hancock, Shruti Singh, Brent Harrison, and Mark O Riedl. 2018. Event representations for automated story generation with deep neural nets. In Thirty-Second AAAI Conference on Artificial Intelligence. find
  115. Cynthia Matuszek. 2018. Grounded language learning: Where robotics and nlp meet (early career spotlight). In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI), Stockholm, Sweden. find
  116. Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017. Learned in translation: Contextualized word vectors. In Advances in Neural Information Processing Systems, pages 6297–6308. find our card
  117. James L. McClelland, Felix Hill, Maja Rudolph, Jason Baldridge, and Hinrich Schütze. 2019. Extending Machine Language Models toward Human-Level Language Understanding. arXiv:1912.05877. — [arXiv:1912.05877](https://arxiv.org/abs/1912.05877) find
  118. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems, 26. find
  119. Margaret Mitchell, Jesse Dodge, Amit Goyal, Kota Yamaguchi, Karl Stratos, Xufeng Han, Alyssa Mensch, Alexander C. Berg, Tamara L. Berg, and Hal Daumé III. 2012. Midge: Generating image descriptions from computer vision detections. In European Chapter of the Association for Computational Linguistics… find
  120. Tom M Mitchell. 1980. The need for biases in learning generalizations. Department of Computer Science, Laboratory for Computer Science Research. find
  121. Raymond J. Mooney. 2008. Learning to connect language and perception. In Proceedings of the 23rd AAAI Conference on Artificial Intelligence (AAAI), pages 1598–1601, Chicago, IL. Senior Member Paper. find
  122. Raymond J Mooney and Gerald Dejong. 1985. Learning schemata for natural language processing. In Proceedings of the Ninth International Joint Conference on Artificial Intelligence (IJCAI-85). find
  123. Daniele Moro and Casey Kennington. 2018. Multimodal visual and simulated muscle activations for grounded semantics of hand-related descriptions. In Workshop on the Semantics and Pragmatics of Dialogue. SEMDIAL. find
  124. Roozbeh Mottaghi, Mohammad Rastegari, Abhinav Gupta, and Ali Farhadi. 2016. “what happens if...” learning to predict the effect of forces in images. In Computer Vision – ECCV 2016, pages 269–285, Cham. Springer International Publishing. find
  125. Adithyavairavan Murali, Tao Chen, Kalyan Vasudev Alwala, Dhiraj Gandhi, Lerrel Pinto, Saurabh Gupta, and Abhinav Gupta. 2019. Pyrobot: An open-source robotics framework for research and benchmarking. arXiv preprint arXiv:1906.08236. — [arXiv:1906.08236](https://arxiv.org/abs/1906.08236) find
  126. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In North American Chapter of the Association for Computational Linguistics (NAACL). find
  127. Aida Nematzadeh, Kaylee Burns, Erin Grant, Alison Gopnik, and Tom Griffiths. 2018. Evaluating theory of mind in question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2392–2400. find
  128. John R Pierce. 1969. Whither speech recognition? The journal of the acoustical society of america, 46(4B):1049–1051. find
  129. NVIDIA. 2019. NVIDIA Isaac software development kit. https://developer.nvidia.com/ isaac-sdk . Accessed 2019-12-09. ↗
  130. Nicolas Pinto, David Doukhan, James J DiCarlo, and David D Cox. 2009. A high-throughput screening approach to discovering good forms of biologically inspired visual representation. PLoS computational biology, 5(11):e1000579. find
  131. Elinor Ochs. 1993. Constructing social identity: A language socialization perspective. Research on language and social interaction, 26(3):287–306. find
  132. Jordan B. Pollack. 1987. On Connectionist Models of Natural Language Processing. Ph.D. thesis, University of Illinois. find
  133. William O’Grady. 2005. How Children Learn Language. Cambridge University Press. find
  134. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. find
  135. Veronica Ornaghi, Jens Brockmeier, and Ilaria Grazzani Gavazzi. 2011. The role of language games in children’s understanding of mental states: A training study. Journal of cognition and development, 12(2):239–259. find
  136. Alexander Ororbia, Ankur Mali, Matthew Kelly, and David Reitter. 2019. Like a baby: Visually situated neural language acquisition. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5127–5136, Florence, Italy. find
  137. Denis Paperno, Germán Kruszewski, Angeliki Lazaridou, Ngoc-Quan Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernández. 2016. The LAMBADA dataset: Word prediction requiring a broad discourse context. In Proceedings of the 54th Annual Meeting of the Association fo… find our card
  138. Rohan Paul, Jacob Arkin, Derya Aksaray, Nicholas Roy, and Thomas M Howard. 2018. Efficient grounding of abstract spatial concepts for natural language interaction with robot platforms. The International Journal of Robotics Research, 37(10):1269–1299. find
  139. Haoruo Peng, Daniel Khashabi, and Dan Roth. 2015. Solving hard coreference problems. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 809–819. find
  140. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar. find
  141. Don Perlis. 2016. Five dimensions of reasoning in the wild. In Association for the Advancement of Artificial Intelligence (AAAI). find
  142. Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don’t know: Unanswerable questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 784–789. find our card
  143. Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards empathetic open-domain conversation models: A new benchmark and dataset. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5370–5381, Florence, Italy. find
  144. Michaela Regneri, Marcus Rohrbach, Dominikus Wetzel, Stefan Thater, Bernt Schiele, and Manfred Pinkal. 2013. Grounding action descriptions in videos. Transactions of the Association for Computational Linguistics (TACL), 1:25–36. find
  145. Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M. Smith, Y-Lan Boureau, and Jason Weston. 2020. Recipes for building an open-domain chatbot. In arXiv. find our card
  146. Stephanie Rosenthal, Joydeep Biswas, and Manuela Veloso. 2010. An effective personal mobile robot agent through symbiotic human-robot interaction. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1-Volume 1, pages 915–922. International Foundatio… find
  147. Corby Rosset. 2020. Turing-NLG: A 17-billion-parameter language model by Microsoft. find
  148. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. 2015. ImageNet Large Scale Visual Recognition Challenge. find our card
  149. International Journal of Computer Vision (IJCV), 115(3):211–252. human-robot dialogue. In Proceedings of 15th SIG-DIAL Meeting on Discourse and Dialogue. find
  150. Jacqueline Sachs, Barbara Bard, and Marie L Johnson. 1981. Language learning with restricted input: Case studies of two hearing children of deaf parents. Applied Psycholinguistics, 2(1):33–54. find
  151. Haoyue Shi, Jiayuan Mao, Kevin Gimpel, and Karen Livescu. 2019. Visually grounded neural syntax acquisition. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1842–1861, Florence, Italy. find
  152. Ian Sample and Alex Hern. 2014. Scientists dispute whether computer ‘eugene goostman‘ passed turing test. The Guardian, 9. find
  153. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019. Social IQa: Commonsense reasoning about social interactions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processi… find
  154. Rosario Scalise, Jesse Thomason, Yonatan Bisk, and Siddhartha Srinivasa. 2019. Improving robot success detection using static object data. In Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems. find
  155. Roger C. Schank and Robert P. Abelson. 1977. Scripts, Plans, Goals and Understanding: an Inquiry into Human Knowledge Structures. L. Erlbaum, Hillsdale, NJ. find
  156. Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. 2019. Megatron-lm: Training multi-billion parameter language models using gpu model parallelism. arXiv preprint arXiv:1909.08053. find
  157. Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. 2020. ALFRED: A benchmark for interpreting grounded instructions for everyday tasks. Computer Vision and Pattern Recognition (CVPR). find our card
  158. Carina Silberer and Mirella Lapata. 2014. Learning grounded meaning representations with autoencoders. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 721–732, Baltimore, Maryland. find
  159. Marten van Schijndel, Aaron Mueller, and Tal Linzen. 2019. Quantity doesn’t buy quality syntax with neural language models. arXiv preprint arXiv:1909.00111. find
  160. Jivko Sinapov, Connor Schenck, and Alexander Stoytchev. 2014. Learning relational object categories using behavioral exploration and multimodal perception. In IEEE International Conference on Robotics and Automation. find
  161. David Schlangen. 2019a. Grounded agreement games: Emphasizing conversational grounding in visual dialogue settings. arXiv. find
  162. Linda Smith and Michael Gasser. 2005. The development of embodied cognition: Six lessons from babies. Artificial life, 11(1-2):13–29. find
  163. David Schlangen. 2019b. Language tasks and language games: On methodology in current natural language processing research. arXiv. find
  164. Paul Smolensky. 1990. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46:159–216. find
  165. Iulian V. Serban, Chinnadhurai Sankar, Mathieu Germain, Saizheng Zhang, Zhouhan Lin, Sandeep Subramanian, Taesup Kim, Michael Pieper, Sarath Chandar, Nan Rosemary Ke, Sai Rajeshwar, Alexandre de Brebisson, Jose M. R. Sotelo, Dendi Suhubdy, Vincent Michalski, Alexandre Nguyen, Joelle Pineau, and Yosh… — [arXiv:1709.02349](https://arxiv.org/abs/1709.02349) find
  166. Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. 2018. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2556–2565,… find our card
  167. Lanbo She, Shaohua Yang, Yu Cheng, Yunyi Jia, Joyce Y. Chai, and Ning Xi. 2014. Back to the blocks world: Learning new actions through situated Richard Socher, Brody Huval, Christopher Manning, and Andrew Ng. 2012. Semantic compositionality through recursive matrix-vector spaces. In Empirical Method… find
  168. T. Srinivasan, R. Sanabria, and F. Metze. 2020. Looking enhances listening: Recovering missing speech using images. In ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6304–6308. find
  169. Mark Steedman. 2008. Last words: On becoming a discipline. Computational Linguistics, 34(1):137–144. find
  170. Greg J Stephens, Lauren J Silbert, and Uri Hasson. 2010. Speaker–listener neural coupling underlies successful communication. Proceedings of the National Academy of Sciences, 107(32):14425–14430. find
  171. Alane Suhr, Claudia Yan, Jack Schluger, Stanley Yu, Hadi Khader, Marwa Mouallem, Iris Zhang, and Yoav Artzi. 2019a. Executing instructions in situated collaborative interactions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint… find
  172. Alane Suhr, Stephanie Zhou, Ally Zhang, Iris Zhang, Huajun Bai, and Yoav Artzi. 2019b. A corpus for reasoning about natural language grounded in photographs. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6418–6428, Florence, Italy. find our card
  173. Chen Sun, Fabien Baradel, Kevin Murphy, and Cordelia Schmid. 2019a. Contrastive bidirectional transformer for temporal representation learning. arxiv:1906.05743. — [arXiv:1906.05743](https://arxiv.org/abs/1906.05743) find
  174. Chen Sun, Austin Myers, Carl Vondrick, Kevin Murphy, and Cordelia Schmid. 2019b. VideoBERT: A Joint Model for Video and Language Representation Learning. In International Conference on Computer vision. find our card
  175. Yee-Whye Teh. 2006. A hierarchical bayesian language model based on pitman-yor processes. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 985–992, Sydney, Australia. find
  176. Stefanie Tellex, Nakul Gopalan, Hadas Kress-Gazit, and Cynthia Matuszek. 2020. Robots that use language. The Annual Review of Control, Robotics, and Autonomous Systems, 15. find
  177. Stefanie Tellex, Ross Knepper, Adrian Li, Daniela Rus, and Nicholas Roy. 2014. Asking for help using inverse semantics. In Proceedings of Robotics: Science and Systems (RSS), Berkeley, California. find
  178. Stefanie Tellex, Thomas Kollar, Steven Dickerson, Matthew R Walter, Ashis Gopal Banerjee, Seth Teller, and Nicholas Roy. 2011. Understanding natural language commands for robotic navigation and mobile manipulation. In Proceedings of the National Conference on Artificial Intelligence. find
  179. Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. BERT rediscovers the classical NLP pipeline. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4593– 4601, Florence, Italy. find our card
  180. Esther Thelen and Linda B. Smith. 1996. A Dynamic Systems Approach to the Development of Cognition and Action. MIT Press. find
  181. Jesse Thomason, Daniel Gordon, and Yonatan Bisk. 2019a. Shifting the baseline: Single modality performance on visual navigation & QA. In North American Chapter of the Association for Computational Linguistics (NAACL). find
  182. Jesse Thomason, Michael Murray, Maya Cakmak, and Luke Zettlemoyer. 2019b. Vision-and-dialog navigation. In Conference on Robot Learning (CoRL). find
  183. Jesse Thomason, Aishwarya Padmakumar, Jivko Sinapov, Justin Hart, Peter Stone, and Raymond J. Mooney. 2017. Opportunistic active learning for grounding natural language descriptions. In Proceedings of the 1st Annual Conference on Robot Learning (CoRL). find
  184. Jesse Thomason, Aishwarya Padmakumar, Jivko Sinapov, Nick Walker, Yuqian Jiang, Harel Yedidsion, Justin Hart, Peter Stone, and Raymond J. Mooney. 2020. Jointly improving parsing and perception for natural language commands through human-robot dialog. The Journal of Artificial Intelligence Research (… find
  185. Jesse Thomason, Jivko Sinapov, Maxwell Svetlik, Peter Stone, and Raymond J. Mooney. 2016. Learning multi-modal grounded linguistic semantics by playing “I spy”. In International Joint Conference on Artificial Intelligence (IJCAI). find
  186. Emanuel Todorov, Tom Erez, and Yuval Tassa. 2012. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5026–5033. IEEE. find
  187. Michael Tomasello. 2009. Constructing a language. Harvard university press. find
  188. Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, J. Zico Kolter, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2019. Multimodal transformer for unaligned multimodal language sequences. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6558– 6569,… find our card
  189. Joseph Turian, Lev-Arie Ratinov, and Yoshua Bengio. 2010. Word representations: A simple and general method for semi-supervised learning. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 384–394. find our card
  190. Alan M Turing. 1950. Computing machinery and intelligence. Mind. find
  191. Peter D Turney and Patrick Pantel. 2010. From frequency to meaning: Vector space models of semantics. Journal of artificial intelligence research, 37:141–188. find
  192. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008. find
  193. Gabriella Vigliocco, Pamela Perniss, and David Vinson. 2014. Language as a multimodal phenomenon: implications for language learning, processing and evolution. find
  194. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). find
  195. Harm de Vries, Dzmitry Bahdanau, and Christopher Manning. 2020. Towards ecologically valid research on language user interfaces. In arXiv. find
  196. Matthew Walter, Sachithra Hemachandra, Bianca Homberg, Stefanie Tellex, and Seth Teller. 2013. Learning semantic maps from natural language descriptions. In Proceedings of Robotics: Science and Systems (RSS), Berlin, Germany. find
  197. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V Le. 2019b. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in Neural Information Processing Systems 32 (NIPS 2019). find our card
  198. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2019a. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In International Conference on Learning Representations. find our card
  199. Mark Yatskar, Luke Zettlemoyer, and Ali Farhadi. 2016. Situation recognition: Visual semantic role labeling for image understanding. In Conference on Computer Vision and Pattern Recognition. find
  200. Xin Wang, Jiawei Wu, Junkun Chen, Lei Li, Yuan-Fang Wang, and William Yang Wang. 2019b. Vatex: A large-scale, high-quality multilingual dataset for video-and-language research. In The IEEE International Conference on Computer Vision (ICCV). find
  201. Amir Zadeh, Michael Chan, Paul Pu Liang, Edmund Tong, and Louis-Philippe Morency. 2019. Social-iq: A question answering benchmark for artificial social intelligence. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). find
  202. Ronald Wardhaugh. 2011. An introduction to sociolinguistics, volume 28. John Wiley & Sons. find
  203. Rowan Zellers, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019a. From recognition to cognition: Visual commonsense reasoning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). find
  204. Joseph Weizenbaum. 1966. Eliza – a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1):36–45. find
  205. Lloyd R Welch. 2003. Hidden markov models and the baum-welch algorithm. IEEE Information Theory Society Newsletter, 53(4):1–24. find
  206. Gregory M Werner and Michael G Dyer. 1991. Evolution of communication in artificial organisms. ALife. find
  207. Terry Winograd. 1971. Procedures as a representation for data in a computer program for understanding natural language. Technical report, Massachusetts Institute of Technology, Project MAC. find
  208. Ludwig Wittgenstein. 1953. Philosophical Investigations. Macmillan. find
  209. Ludwig Wittgenstein. 1958. The blue and brown books. Basil Blackwell. find
  210. Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, Li Yi, Angel X. Chang, Leonidas J. Guibas, and Hao Su. 2020. SAPIEN: A simulated part-based interactive environment. In Computer Vision and Pattern Recognition (CVPR). find
  211. Rowan Zellers, Ari Holtzman, Elizabeth Clark, Lianhui Qin, Ali Farhadi, and Yejin Choi. 2020. Evaluating machines by their real-world language use. arXiv preprint arXiv:2004.03607. — [arXiv:2004.03607](https://arxiv.org/abs/2004.03607) find
  212. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019b. Defending against neural fake news. In Thirty-third Conference on Neural Information Processing Systems. find our card
  213. Li Zhou, Jianfeng Gao, Di Li, and Heung-Yeung Shum. 2020. The design and implementation of xiaoice, an empathetic social chatbot. Computational Linguistics, 46(1):53–93. find
  214. Luowei Zhou, Hamid Palangi, Lei Zhang, Houdong Hu, Jason J. Corso, and Jianfeng Gao. 2019. Unified vision-language pre-training for image captioning and vqa. In Thirty-Fourth AAAI Conference on Artificial Intelligence. find
  215. Semih Yagcioglu, Aykut Erdem, Erkut Erdem, and Nazli Ikizler-Cinbis. 2018. RecipeQA: A challenge dataset for multimodal comprehension of cooking recipes. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1358–1368, Brussels, Belgium. find
  216. Diyi Yang, Jiaao Chen, Zichao Yang, Dan Jurafsky, and Eduard Hovy. 2019a. Let’s make your request more persuasive: Modeling persuasive strategies via semi-supervised neural nets on crowdfunding platforms. find