Elamur Research⌕
☰
home / papers / 2023 / Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory

Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory

Paper · 2023 · Xizhou Zhu, Yuntao Chen, Hao Tian et al.

Summary

Earlier Minecraft agents rely mostly on reinforcement learning and target a single objective, ObtainDiamond, which involves under 5% of the game's item types. They reach roughly 20% success there and need enormous numbers of training steps. The authors propose Ghost in the Minecraft (GITM), which replaces direct goal-to-keyboard mapping with a hierarchy of three LLM-driven components:

  • LLM Decomposer: splits a goal into a sub-goal tree, using knowledge gathered from the web.
  • LLM Planner: turns each sub-goal into a list of structured actions, using execution feedback. It also stores successful plans in a text-based memory.
  • LLM Interface: converts those actions into low-level controls.

GITM improves ObtainDiamond success by 47.5 percentage points over prior methods. It is also the first agent to obtain every item in the Minecraft Overworld technology tree. Training needs no GPU, only a single 32-core CPU node. This suggests that LLM planning with explicit knowledge and memory can be a far cheaper and more general alternative to RL for long-horizon tasks.

Key points

  • GITM unlocked all 262 items in the Overworld tech tree. Earlier agents (VPT, DreamerV3, DEPS) reached only 78 items, about 30%.

  • Structured actions: the action set has 13 entries, including explore, mine, craft, smelt and dig_down. Sub-goals follow an (Object, Count, Material, Tool, Info) format.

  • Diamond success: on diamonds, GITM is about 3.5× more successful than competing methods.

  • Cost: it uses over 10,000× fewer environment steps and trains in about 2 days on one CPU node. VPT needs 6,480 GPU days and DreamerV3 needs 17.

  • Ablation: each setting ran 40 games, with a 12,000-step limit and up to 30 LLM queries per sub-goal. Diamond success rose with each added component:

    ConfigurationDiamond success
    Goal decomposition alone0%
    + Feedback5%
    + External knowledge35%
    + Text-based memory67.5%

    Iron pickaxe success reached 95% with all components.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China415 works · 94,028 citations🎓University of Science and Technology of ChinaHefei · China145 works · 114,409 citations🎓Chinese University of Hong KongHong Kong · China159 works · 16,154 citations🏛️Chinese Academy of SciencesBeijing · China246 works · 24,952 citations🔬Shanghai Artificial Intelligence LaboratoryShanghai · China152 works · 13,164 citations🔬Institute of AutomationBeijing · China87 works · 14,883 citations🏢SensetimeBeijing · China13 works · 2,161 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

Cite

Cite as Xizhou Zhu, Yuntao Chen, Hao Tian, Chenxin Tao, Weijie Su, Chenyu Yang, Gao Huang, Bin Li, Lewei Lu, Xiaogang Wang, Yu Qiao, Zhaoxiang Zhang and Jifeng Dai. 2023. Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory. arXiv:2305.17144.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2305.17144 · arXiv 2305.17144

@misc{zhu2023ghost,
  title = {{Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory}},
  author = {Xizhou Zhu, Yuntao Chen, Hao Tian, Chenxin Tao, Weijie Su, Chenyu Yang, Gao Huang, Bin Li, Lewei Lu, Xiaogang Wang, Yu Qiao, Zhaoxiang Zhang and Jifeng Dai},
  year = {2023},
  doi = {10.48550/arxiv.2305.17144},
  eprint = {2305.17144},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2305.17144}
}

References

28 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. Amiranashvili, N. Dorka, W. Burgard, V. Koltun, and T. Brox. Scaling imitation learning in minecraft. arXiv preprint arXiv:2007.02701, 2020. — [arXiv:2007.02701](https://arxiv.org/abs/2007.02701) find
  2. B. Baker, I. Akkaya, P. Zhokov, J. Huizinga, J. Tang, A. Ecoffet, B. Houghton, R. Sampedro, and J. Clune. Video pretraining (vpt): Learning to act by watching unlabeled online videos. Advances in Neural Information Processing Systems, 35:24639–24654, 2022. find
  3. S. Cai, Z. Wang, X. Ma, A. Liu, and Y. Liang. Open-world multi-task control through goal-aware representation learning and adaptive horizon prediction. arXiv preprint arXiv:2301.10034, 2023. find
  4. D. Driess, F. Xia, M. S. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu, et al. Palm-e: An embodied multimodal language model. arXiv preprint arXiv:2303.03378, 2023. find
  5. L. Fan, G. Wang, Y. Jiang, A. Mandlekar, Y. Yang, H. Zhu, A. Tang, D.-A. Huang, Y. Zhu, and A. Anandkumar. Minedojo: Building open-ended embodied agents with internet-scale knowledge. arXiv preprint arXiv:2206.08853, 2022.
  6. W. H. Guss, S. Milani, N. Topin, B. Houghton, S. Mohanty, A. Melnik, A. Harter, B. Buschmaas, B. Jaster, C. Berganski, et al. Towards robust and domain agnostic reinforcement learning competitions: Minerl 2020. In NeurIPS 2020 Competition and Demonstration Track, pages 233–252. PMLR, 2021. find
  7. D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap. Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104, 2023. find
  8. W. Huang, P. Abbeel, D. Pathak, and I. Mordatch. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In International Conference on Machine Learning, pages 9118–9147. PMLR, 2022.
  9. W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y. Chebotar, et al. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608, 2022. find
  10. A. Kanervisto, S. Milani, K. Ramanauskas, N. Topin, Z. Lin, J. Li, J. Shi, D. Ye, Q. Fu, W. Yang, et al. Minerl diamond 2021 competition: Overview, results, and lessons learned. NeurIPS 2021 Competitions and Demonstrations Track, pages 13–28, 2022. find
  11. I. Kanitscheider, J. Huizinga, D. Farhi, W. H. Guss, B. Houghton, R. Sampedro, P. Zhokhov, B. Baker, A. Ecoffet, J. Tang, et al. Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraft. arXiv preprint arXiv:2106.14876, 2021. — [arXiv:2106.14876](https://arxiv.org/abs/2106.14876) find
  12. M. Li, F. Song, B. Yu, H. Yu, Z. Li, F. Huang, and Y. Li. Api-bank: A benchmark for tool-augmented llms. arXiv preprint arXiv:2304.08244, 2023. — [arXiv:2304.08244](https://arxiv.org/abs/2304.08244) find
  13. J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng. Code as policies: Language model programs for embodied control. arXiv preprint arXiv:2209.07753, 2022. find
  14. Z. Lin, J. Li, J. Shi, D. Ye, Q. Fu, and W. Yang. Juewu-mc: Playing minecraft with sample-efficient hierarchical reinforcement learning. arXiv preprint arXiv:2112.04907, 2021. — [arXiv:2112.04907](https://arxiv.org/abs/2112.04907) find
  15. H. Mao, C. Wang, X. Hao, Y. Mao, Y. Lu, C. Wu, J. Hao, D. Li, and P. Tang. Seihai: A sample-efficient hierarchical ai for the minerl competition. In Distributed Artificial Intelligence: Third International Conference, DAI 2021, Shanghai, China, December 17–18, 2021, Proceedings 3, pages 38–51. Sprin… find
  16. Y. Matsuo, Y. LeCun, M. Sahani, D. Precup, D. Silver, M. Sugiyama, E. Uchibe, and J. Morimoto. Deep learning, reinforcement learning, and world models. Neural Networks, 2022. find
  17. S. Milani, N. Topin, B. Houghton, W. H. Guss, S. P. Mohanty, K. Nakata, O. Vinyals, and N. S. Kuno. Retrospective analysis of the 2019 minerl competition on sample efficient reinforcement learning. In NeurIPS 2019 Competition and Demonstration Track, pages 203–214. PMLR, 2020. find
  18. S. Milani, A. Kanervisto, K. Ramanauskas, S. Schulhoff, B. Houghton, S. Mohanty, B. Galbraith, K. Chen, Y. Song, T. Zhou, et al. Towards solving fuzzy tasks with human feedback: A retrospective of the minerl basalt 2022 competition. arXiv preprint arXiv:2303.13512, 2023. — [arXiv:2303.13512](https://arxiv.org/abs/2303.13512) find
  19. T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, L. Zettlemoyer, N. Cancedda, and T. Scialom. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761, 2023.
  20. Y. Shen, K. Song, X. Tan, D. Li, W. Lu, and Y. Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface. arXiv preprint arXiv:2303.17580, 2023.
  21. I. Singh, V. Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay, D. Fox, J. Thomason, and A. Garg. Progprompt: Generating situated robot task plans using large language models. arXiv preprint arXiv:2209.11302, 2022.
  22. A. Skrynnik, A. Staroverov, E. Aitygulov, K. Aksenov, V. Davydov, and A. I. Panov. Hierarchical deep q-network from imperfect demonstrations in minecraft. Cognitive Systems Research, 65:74–78, 2021. find
  23. O. E. L. Team, A. Stooke, A. Mahajan, C. Barros, C. Deck, J. Bauer, J. Sygnowski, M. Trebacz, M. Jaderberg, M. Mathieu, et al. Open-ended learning leads to generally capable agents. arXiv preprint arXiv:2107.12808, 2021. — [arXiv:2107.12808](https://arxiv.org/abs/2107.12808) find
  24. C. Tessler, S. Givony, T. Zahavy, D. Mankowitz, and S. Mannor. A deep hierarchical approach to lifelong learning in minecraft. In Proceedings of the AAAI conference on artificial intelligence, 2017. find
  25. Z. Wang, S. Cai, A. Liu, X. Ma, and Y. Liang. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. arXiv preprint arXiv:2302.01560, 2023.
  26. J. Wei, X. Wang, D. Schuurmans, M. Bosma, E. Chi, Q. Le, and D. Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903, 2022.
  27. Wikipedia contributors. Minecraft — Wikipedia, the free encyclopedia. https://en.wikipedia.org/w/index.php?title=Minecraft&oldid=1155148900, 2023. ↗
  28. H. Yuan, C. Zhang, H. Wang, F. Xie, P. Cai, H. Dong, and Z. Lu. Plan4mc: Skill reinforcement learning and planning for open-world minecraft tasks. arXiv preprint arXiv:2303.16563, 2023. — [arXiv:2303.16563](https://arxiv.org/abs/2303.16563) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhu-2023-ghost-minecraft-generally-capable-agents/