Elamur Research
☰
home / papers / 2020 / ALFWorld: Aligning Text and Embodied Environments for Interactive Learning

ALFWorld: Aligning Text and Embodied Environments for Interactive Learning

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/2010.03768

Summary

Embodied agents that follow household instructions usually learn from fixed sets of expert demonstrations in visually complex simulators. That makes high-level planning hard to learn and hard to generalize. The authors present ALFWorld, a framework that pairs every ALFRED scene in the THOR simulator with an equivalent interactive text game built on TextWorld. Both versions share one latent world state described in PDDL. Agents can therefore practice tasks with abstract text commands and later carry them out with low-level physical actions. On top of this framework they build BUTLER, a modular agent with three parts. BUTLER::Brain is a text policy trained with imitation learning. BUTLER::Vision uses a Mask R-CNN detector to turn camera frames into textual observations. BUTLER::Body is a controller that converts each text step into embodied primitives. Pre-training in text transfers zero-shot to unseen embodied tasks. Compared with training directly in the visual world, it is about 7× faster and performs better. The work supports the idea that language-level priors help agents cope with unfamiliar physical environments.

Key points

  • ALFWorld includes the six ALFRED task categories. These are moving one object to a target spot, inspecting an object under a lamp, and cleaning, heating or cooling an object before putting it somewhere. The sixth is relocating a pair of objects. It has 3,553 training tasks, 140 seen and 134 unseen evaluation tasks. The scenes span 120 rooms: 30 each of kitchens, bedrooms, bathrooms and living rooms.
  • Agents receive only goal descriptions. ALFRED's step-by-step human instructions are left out, and tasks involving slicing or portable containers are excluded.
  • Text games are generated from PDDL scene descriptions with a context-sensitive grammar. They use high-level commands such as goto, take, put, open, heat, cool and clean, and a single command can stand for many low-level actions.
  • BUTLER::Brain generates commands token by token. Variants that chose from engine-provided candidate commands overfit and achieved no embodied transfer.
  • Agents pre-trained in text generalize to unseen embodied settings better than agents trained on demonstration corpora or from scratch in the visual environment.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

233 papers of our library cite it (of the 140673 whose bibliographies we read); 18 of them have a page here, 215 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2010.03768 · arXiv 2010.03768

@misc{shridhar2020alfworld,
  title = {{ALFWorld: Aligning Text and Embodied Environments for Interactive Learning}},
  author = {Shridhar, Mohit and Yuan, Xingdi and Côté, Marc-Alexandre and Bisk, Yonatan and Trischler, Adam and Hausknecht, Matthew},
  year = {2020},
  doi = {10.48550/arxiv.2010.03768},
  eprint = {2010.03768},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2010.03768}
}

References

58 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Adhikari, A., Yuan, X., Côté, M.-A., Zelinka, M., Rondeau, M.-A., Laroche, R., Poupart, P., Tang, J., Trischler, A., and Hamilton, W. L. (2020). Learning dynamic belief graphs to generalize on text-based games. In Neural Information Processing Systems (NeurIPS). find
  2. Ammanabrolu, P. and Hausknecht, M. (2020). Graph constrained reinforcement learning for natural language action spaces. In International Conference on Learning Representations. find
  3. Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sünderhauf, N., Reid, I., Gould, S., and van den Hengel, A. (2018a). Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of the IEEE Conference on Computer Vision and Patt… find our card
  4. Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sünderhauf, N., Reid, I., Gould, S., and van den Hengel, A. (2018b). Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of the IEEE Conference on Computer Vision and Patt… find our card
  5. Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., and Parikh, D. (2015). VQA: Visual Question Answering. In International Conference on Computer Vision (ICCV). find
  6. Asai, M. and Fukunaga, A. (2017). Classical planning in deep latent space: Bridging the subsymbolic-symbolic boundary. arXiv preprint arXiv:1705.00154. — [arXiv:1705.00154](https://arxiv.org/abs/1705.00154) find
  7. Ba, L. J., Kiros, J. R., and Hinton, G. E. (2016). Layer normalization. CoRR, abs/1607.06450. find our card
  8. Banks, B., Wingfield, C., and Connell, L. (2020). Linguistic distributional knowledge and sensorimotor grounding both contribute to semantic category production. find
  9. Bisk, Y., Holtzman, A., Thomason, J., Andreas, J., Bengio, Y., Chai, J., Lapata, M., Lazaridou, A., May, J., Nisnevich, A., Pinto, N., and Turian, J. (2020). Experience Grounds Language. In Empirical Methods in Natural Language Processing. find our card
  10. Chevalier-Boisvert, M., Bahdanau, D., Lahlou, S., Willems, L., Saharia, C., Nguyen, T. H., and Bengio, Y. (2019). BabyAI: First steps towards grounded language learning with a human in the loop. In International Conference on Learning Representations. find
  11. Cho, K., van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)… find
  12. Côté, M.-A., Kádár, A., Yuan, X., Kybartas, B., Barnes, T., Fine, E., Moore, J., Tao, R. Y., Hausknecht, M., Asri, L. E., Adada, M., Tay, W., and Trischler, A. (2018). Textworld: A learning environment for text-based games. CoRR, abs/1806.11532. find our card
  13. Das, A., Datta, S., Gkioxari, G., Lee, S., Parikh, D., and Batra, D. (2018). Embodied Question Answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). find
  14. Dove, G. (2014). Thinking in words: language as an embodied medium of thought. Topics in cognitive science, 6(3):371–389. find
  15. Gehrmann, S., Deng, Y., and Rush, A. (2018). Bottom-up abstractive summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. find our card
  16. Gordon, D., Kembhavi, A., Rastegari, M., Redmon, J., Fox, D., and Farhadi, A. (2018). Iqa: Visual question answering in interactive environments. In Computer Vision and Pattern Recognition (CVPR), 2018 IEEE Conference on. find
  17. Gulcehre, C., Ahn, S., Nallapati, R., Zhou, B., and Bengio, Y. (2016). Pointing the unknown words. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). find
  18. Ha, D. and Schmidhuber, J. (2018). Recurrent world models facilitate policy evolution. In Advances in Neural Information Processing Systems 31. find
  19. Hausknecht, M. and Stone, P. (2015). Deep recurrent q-learning for partially observable mdps. arXiv preprint arXiv:1507.06527. — [arXiv:1507.06527](https://arxiv.org/abs/1507.06527) find
  20. Hausknecht, M. J., Ammanabrolu, P., Côté, M.-A., and Yuan, X. (2020). Interactive fiction games: A colossal adventure. In AAAI. find
  21. He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017). Mask r-cnn. In Proceedings of the IEEE international conference on computer vision. find
  22. He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. find
  23. Helmert, M. (2006). The Fast Downward planning system. Journal of Artificial Intelligence Research. find
  24. Hu, H., Yarats, D., Gong, Q., Tian, Y., and Lewis, M. (2019). Hierarchical decision making by generating and following natural language instructions. In Advances in Neural Information Processing Systems. find
  25. Hu, R., Rohrbach, M., Andreas, J., Darrell, T., and Saenko, K. (2017). Modeling relationships in referential expressions with compositional modular networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. find
  26. Jiang, Y., Gu, S. S., Murphy, K. P., and Finn, C. (2019). Language as an abstraction for hierarchical deep reinforcement learning. In Advances in Neural Information Processing Systems. find
  27. Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Zitnick, C. L., and Girshick, R. (2017). Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In CVPR. find our card
  28. Johnson, J., Karpathy, A., and Fei-Fei, L. (2016). Densecap: Fully convolutional localization networks for dense captioning. In Proceedings of the IEEE conference on computer vision and pattern recognition. find
  29. Kaelbling, L. P. and Lozano-Pérez, T. (2011). Hierarchical task and motion planning in the now. In 2011 IEEE International Conference on Robotics and Automation, pages 1470–1477. IEEE. find
  30. Kingma, D. P. and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. find
  31. Kolve, E., Mottaghi, R., Han, W., VanderBilt, E., Weihs, L., Herrasti, A., Gordon, D., Zhu, Y., Gupta, A., and Farhadi, A. (2017). Ai2-thor: An interactive 3d environment for visual ai. arXiv preprint arXiv:1712.05474. find
  32. Konidaris, G., Kaelbling, L. P., and Lozano-Perez, T. (2018). From skills to symbols: Learning symbolic representations for abstract high-level planning. Journal of Artificial Intelligence Research, 61:215–289. find
  33. Kulkarni, T. D., Whitney, W. F., Kohli, P., and Tenenbaum, J. (2015). Deep convolutional inverse graphics network. In Advances in neural information processing systems. find
  34. Küttler, H., Nardelli, N., Miller, A. H., Raileanu, R., Selvatici, M., Grefenstette, E., and Rocktäschel, T. (2020). The nethack learning environment. find
  35. Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L. (2014). Microsoft coco: Common objects in context. In European conference on computer vision. find our card
  36. Lu, J., Batra, D., Parikh, D., and Lee, S. (2019). Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. In Advances in Neural Information Processing Systems. find our card
  37. MacMahon, M., Stankiewicz, B., and Kuipers, B. (2006). Walk the talk: Connecting language, knowledge, and action in route instructions. In Proceedings of the 21st National Conference on Artificial Intelligence (AAAI-2006). find
  38. Marzoev, A., Madden, S., Kaashoek, M. F., Cafarella, M., and Andreas, J. (2020). Unnatural language processing: Bridging the gap between synthetic and natural language data. arXiv preprint arXiv:2004.13645. — [arXiv:2004.13645](https://arxiv.org/abs/2004.13645) find
  39. McDermott, D., Ghallab, M., Howe, A., Knoblock, C., Ram, A., Veloso, M., Weld, D., and Wilkins, D. (1998). Pddl-the planning domain definition language. find
  40. Narasimhan, K., Barzilay, R., and Jaakkola, T. (2018). Grounding language for transfer in deep reinforcement learning. JAIR, 63(1):849–874. find
  41. Press, O. and Wolf, L. (2016). Using the output embedding to improve language models. arXiv preprint arXiv:1608.05859. find our card
  42. Reddy, D. R. et al. (1977). Speech understanding systems: A summary of results of the five-year research effort. Department of Computer Science. Camegie-Mell University, Pittsburgh, PA, 17. find
  43. Ross, S., Gordon, G., and Bagnell, D. (2011). A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics. find
  44. Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2019). Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. find
  45. Schwartz, E., Tennenholtz, G., Tessler, C., and Mannor, S. (2019). Language is power: Representing states using natural language in reinforcement learning. find
  46. Sharma, S., Asri, L. E., Schulz, H., and Zumer, J. (2017). Relevance of unsupervised metrics in task-oriented dialogue for evaluating natural language generation. arXiv preprint arXiv:1706.09799. — [arXiv:1706.09799](https://arxiv.org/abs/1706.09799) find
  47. Shridhar, M., Thomason, J., Gordon, D., Bisk, Y., Han, W., Mottaghi, R., Zettlemoyer, L., and Fox, D. (2020). Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10740–10749. find our card
  48. Sun, C., Myers, A., Vondrick, C., Murphy, K., and Schmid, C. (2019). Videobert: A joint model for video and language representation learning. In Proceedings of the IEEE International Conference on Computer Vision. find our card
  49. Tang, K., Niu, Y., Huang, J., Shi, J., and Zhang, H. (2020). Unbiased scene graph generation from biased training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. find
  50. Todorov, E., Erez, T., and Tassa, Y. (2012). Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. find
  51. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems 30. find
  52. Wang, X., Huang, Q., Celikyilmaz, A., Gao, J., Shen, D., Wang, Y.-F., Wang, W. Y., and Zhang, L. (2019). Reinforced cross-modal matching and self-supervised imitation learning for vision-language navigation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. find
  53. Wu, J., Lu, E., Kohli, P., Freeman, B., and Tenenbaum, J. (2017). Learning to see physics via visual de-animation. In Advances in Neural Information Processing Systems. find
  54. Yu, A. W., Dohan, D., Le, Q., Luong, T., Zhao, R., and Chen, K. (2018a). Fast and accurate reading comprehension by combining self-attention and convolution. In International Conference on Learning Representations. find
  55. Yu, L., Lin, Z., Shen, X., Yang, J., Lu, X., Bansal, M., and Berg, T. L. (2018b). Mattnet: Modular attention network for referring expression comprehension. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. find
  56. Yuan, X., Côté, M.-A., Sordoni, A., Laroche, R., Combes, R. T. d., Hausknecht, M., and Trischler, A. (2018). Counting to explore and generalize in text-based games. arXiv preprint arXiv:1806.11525. — [arXiv:1806.11525](https://arxiv.org/abs/1806.11525) find
  57. Zhong, V., Rocktäschel, T., and Grefenstette, E. (2020). RTFM: Generalising to novel environment dynamics via reading. In ICLR. find
  58. Zhu, Y., Gordon, D., Kolve, E., Fox, D., Fei-Fei, L., Gupta, A., Mottaghi, R., and Farhadi, A. (2017). Visual semantic planning using deep successor representations. In IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017. find