Elamur Research⌕
☰
home / papers / 2017 / Grounded Language Learning in a Simulated 3D World

Grounded Language Learning in a Simulated 3D World

Paper · 2017 · Karl Moritz Hermann, Felix Hill, Simon Green et al.

Summary

Teaching machines to connect words with what they see and do remains a hard problem. Earlier systems such as SHRDLU depended on hand-written rules, and later work mostly used simplified, highly structured settings. This DeepMind paper adds a text-instruction channel to an extended version of DeepMind Lab, a 3D simulator. In this environment, an agent sees only raw pixels and receives reward when it picks up the object that a written command describes. The agent is a single neural network made of four parts: a convolutional vision module, an LSTM language encoder, a mixing module, and a two-layer LSTM action module. It is trained with an actor-critic method together with unsupervised auxiliary objectives, starting with almost no built-in knowledge. Language learning only succeeds when reward-based and unsupervised signals are combined. The trained agent understands new word combinations without extra training, and it picks up new words faster as its vocabulary grows. Curricula make harder relational and multi-task instructions learnable. The study suggests that grounded, embodied language can emerge end-to-end rather than being hand-coded.

Key points

  • Environment: DeepMind Lab is extended with a language channel. Layouts, objects, colours, patterns and positions are randomly sampled from billions of possible configurations.
  • Agent and training: The vision module (CNN), language module (LSTM), mixing module and action module (2-layer LSTM) are optimised jointly. Training uses actor-critic RL (Mnih et al., 2016) plus unsupervised auxiliary losses.
  • Generalisation: The agent shows zero-shot comprehension of unseen instructions and applies familiar modifiers and relations to unfamiliar objects. Its rate of word acquisition speeds up over training.
  • Curriculum: Retrieving an object in a specified room required a four-lesson curriculum. Reported results average the best five of 16 random hyperparameter settings.
  • Multi-task learning: One agent learned three tasks (Selection, Next to, In room) using a two-lesson curriculum. The tasks used six colour terms and 40 object shapes, so the agent had to be sensitive to word order.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢Google DeepMindLondon · United Kingdom72 works · 84,326 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

Cite

Cite as Karl Moritz Hermann, Felix Hill, Simon Green, Fumin Wang, Ryan Faulkner, Hubert Soyer, David Szepesvari, Wojciech Marian Czarnecki, Max Jaderberg, Denis Teplyashin, Marcus Wainwright, Chris Apps, Demis Hassabis and Phil Blunsom. 2017. Grounded Language Learning in a Simulated 3D World. arXiv:1706.06551.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1706.06551 · arXiv 1706.06551

@misc{hermann2017grounded,
  title = {{Grounded Language Learning in a Simulated 3D World}},
  author = {Karl Moritz Hermann, Felix Hill, Simon Green, Fumin Wang, Ryan Faulkner, Hubert Soyer, David Szepesvari, Wojciech Marian Czarnecki, Max Jaderberg, Denis Teplyashin, Marcus Wainwright, Chris Apps, Demis Hassabis and Phil Blunsom},
  year = {2017},
  doi = {10.48550/arxiv.1706.06551},
  eprint = {1706.06551},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1706.06551}
}

References

40 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dilip Arumugam, Siddharth Karamcheti, Nakul Gopalan, Lawson L. S. Wong, and Stefanie Tellex. Accurately and efficiently interpreting human-robot instructions of varying granularities. CoRR, abs/1704.06616, 2017. URL http://arxiv.org/abs/1704.06616. — [arXiv:1704.06616](https://arxiv.org/abs/1704.06616) ↗
  2. Charles Beattie, Joel Z. Leibo, Denis Teplyashin, Tom Ward, Marcus Wainwright, Heinrich Küttler, Andrew Lefrancq, Simon Green, Vı́ctor Valdés, Amir Sadik, Julian Schrittwieser, Keith Anderson, Sarah York, Max Cant, Adam Cain, Adrian Bolton, Stephen Gaffney, Helen King, Demis Hassabis, Shane Legg,… — DeepMind Lab
  3. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. Semantic parsing on freebase from question-answer pairs. In EMNLP, pages 1533–1544. ACL, 2013. ISBN 978-1-937284-97-8. URL http://dblp.uni-trier.de/db/conf/emnlp/emnlp2013.html\#BerantCFL13. ↗
  4. David L. Chen and Raymond J. Mooney. Learning to sportscast: A test of grounded language acquisition. In Proceedings of the 25th International Conference on Machine Learning (ICML), Helsinki, Finland, July 2008. URL http://www.cs.utexas.edu/users/ai-lab/?chen:icml08. ↗
  5. Noam Chomsky. A review of BF Skinner’s Verbal Behavior. Language, 35(1):26–58, 1959. find
  6. Leonidas AA Doumas, John E Hummel, and Catherine M Sandhofer. A theory of the discovery and predication of relational concepts. Psychological review, 115(1):1, 2008. find
  7. Sachithra Hemachandra, Matthew R. Walter, Stefanie Tellex, and Seth Teller. Learning spatially-semantic representations from natural language descriptions and scene classifications. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, May 2014. find
  8. Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997. find
  9. Max Jaderberg, Volodymyr Mnih, Wojciech Marian Czarnecki, Tom Schaul, Joel Z Leibo, David Silver, and Koray Kavukcuoglu. Reinforcement learning with unsupervised auxiliary tasks. In International Conference on Learning Representations, 2016.
  10. Samantha Krening, Brent Harrison, Karen M Feigh, Charles Isbell, Mark Riedl, and Andrea Thomaz. Learning from explanations using sentiment and advice in RL. In 2016 IEEE Transactions on Cognitive and Developmental Systems. find
  11. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012.
  12. Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4):541–551, 1989. find
  13. Tomas Mikolov, Armand Joulin, and Marco Baroni. A roadmap towards machine intelligence. arXiv preprint arXiv:1511.08130, 2015. find
  14. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529–533, 2015. find
  15. Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy P Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In International Conference on Machine Learning, 2016. find
  16. Karthik Narasimhan, Tejas Kulkarni, and Regina Barzilay. Language understanding for text-based games using deep reinforcement learning. Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2015.
  17. Junhyuk Oh, Xiaoxiao Guo, Honglak Lee, Richard L Lewis, and Satinder Singh. Action-conditional video prediction using deep networks in Atari games. In Advances in Neural Information Processing Systems 28, 2015.
  18. Steven Pinker. The bootstrapping problem in language acquisition. Mechanisms of language acquisition, pages 399–441, 1987. find
  19. Steven Pinker. Language learnability and language development, volume 7. Harvard University Press, 2009. find
  20. W. V. O. Quine. Word & Object. MIT Press, 1960. find
  21. Paul C Quinn, Peter D Eimas, and Stacey L Rosenkrantz. Evidence for representations of perceptually similar natural categories by 3-month-old and 4-month-old infants. Perception, 22(4):463–475, 1993. find
  22. Timothy T Rogers and James L McClelland. Semantic cognition: A parallel distributed processing approach. MIT press, 2004. find
  23. Deb K Roy and Alex P Pentland. Learning words from sights and sounds: A computational model. Cognitive science, 26(1):113–146, 2002. find
  24. Carina Silberer and Mirella Lapata. Grounded models of semantic representation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2012. find
  25. Jeffrey Mark Siskind. Grounding Language in Perception, pages 207–227. Springer Netherlands, Dordrecht, 1995. ISBN 978-94-011-0273-5. doi: 10.1007/978-94-011-0273-5 12. URL http://dx.doi.org/10.1007/978-94-011-0273-5_12. — [doi:10.1007/978-94-011-0273-5](https://doi.org/10.1007/978-94-011-0273-5) ↗
  26. Jeffrey Mark Siskind. Grounding the lexical semantics of verbs in visual perception using force dynamics and event logic. J. Artif. Intell. Res. (JAIR), 15:31–90, 2001. doi: 10.1613/jair.790. URL https://doi.org/10.1613/jair.790. — [doi:10.1613/jair.790](https://doi.org/10.1613/jair.790) ↗
  27. Linda B Smith, Susan S Jones, and Barbara Landau. Naming in young children: A dumb attentional mechanism? Cognition, 60(2):143–171, 1996. find
  28. Luc Steels. The symbol grounding problem has been solved. so what’s next. Symbols and embodiment: Debates on meaning and cognition, pages 223–244, 2008. find
  29. Jesse Thomason, Shiqi Zhang, Raymond Mooney, and Peter Stone. Learning to interpret natural language commands through human-robot dialog. In Proceedings of the 24th International Conference on Artificial Intelligence, IJCAI’15, pages 1923–1929. find
  30. AAAI Press, 2015. ISBN 978-1-57735-738-4. URL http://dl.acm.org/citation.cfm?id=2832415.2832516. ↗
  31. Tijmen Tieleman and Geoffrey Hinton. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning, 4(2), 2012. find
  32. Ivan Vendrov, Ryan Kiros, Sanja Fidler, and Raquel Urtasun. Order-embeddings of images and language. CoRR, abs/1511.06361, 2015. URL http://arxiv.org/abs/1511.06361. ↗
  33. Stella Vosniadou and William F Brewer. Mental models of the earth: A study of conceptual change in childhood. Cognitive psychology, 24(4):535–585, 1992. find
  34. Matthew R. Walter, Sachithra Hemachandra, Bianca Homberg, Stefanie Tellex, and Seth Teller. A framework for learning semantic maps from grounded natural language descriptions. The International Journal of Robotics Research, 33(9):1167–1190, 2014. doi: 10.1177/0278364914537359. URL http://dx.doi.org/… — [doi:10.1177/0278364914537359](https://doi.org/10.1177/0278364914537359) find
  35. S. I. Wang, P. Liang, and C. Manning. Learning language games through interaction. In Association for Computational Linguistics (ACL), 2016. find
  36. Terry Winograd. Understanding natural language. Cognitive psychology, 3(1):1–191, 1972. find
  37. Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard S Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual attention. International Conference of Machine Learning, 2(3):5, 2015.
  38. Haonan Yu and Jeffrey Mark Siskind. Grounded language learning from video described with sentences. In ACL, pages 53–63. The Association for Computer Linguistics, 2013. ISBN 978-1-937284-50-3. find
  39. Haonan Yu, Haichao Zhang, and Wei Xu. A deep compositional framework for human-like language acquisition in virtual environment. CoRR, abs/1703.09831, 2017. URL https://arxiv.org/abs/1703.09831. — [arXiv:1703.09831](https://arxiv.org/abs/1703.09831) ↗
  40. Luke S. Zettlemoyer and Michael Collins. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence, UAI’05, pages 658– 666, Arlington, Virginia, United States, 2…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hermann-2017-grounded-language-learning-simulated-3d/