Elamur Research⌕
☰
home / papers / 2023 / Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction

Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction

Paper · 2023 · Shaofei Cai, Zihao Wang, Xiaojian Ma et al.

Summary

The paper tackles training a single goal-conditioned agent for many tasks in Minecraft. The authors identify two obstacles. First, the game's scenes are so varied that observations reveal little about which task is being pursued. Second, partial observability makes the dynamics non-stationary, so the agent cannot easily judge how far it is from its goal.

They respond with two components:

  • Goal-Sensitive Backbone (GSB): injects goal information into the visual encoder at several feature levels, rather than just concatenating goal and image features.
  • Adaptive horizon prediction: conditions the policy on the estimated number of steps left until the goal. Because this distance is unknown at test time, a learned module supplies it.

Training uses behavior cloning on successful trajectories generated by VPT, with evaluation in the MineDoJo benchmark. Across 20 tasks in three biomes, the method clearly beats the strongest earlier baseline and roughly doubles its results on many tasks. It also transfers zero-shot to a biome it never saw during training. This matters because it shows that architecture and progress-awareness can substantially advance open-world multi-task control.

Key points

  • Evaluated on 20 Minecraft tasks (e.g., harvesting, combating mobs) across three common biomes in MineDoJo. Demonstrations are filtered successful VPT rollouts, and policies are trained with behavior cloning.
  • Feeding predicted distance-to-goal into the horizon-conditioned policy raises the average multi-task success rate by 27%.
  • Replacing a standard Impala CNN with GSB improves both success rate and precision in every environment. In the Flat biome the gains are 26% (success) and 22% (precision).
  • Subtracting a small constant from the predicted horizon at inference pushes the agent toward quicker task completion and improves overall performance.
  • An agent trained only on Plains reaches success rates close to an agent trained directly on Flat, even with unfamiliar distracting mobs present. Code is available at github.com/CraftJarvis/MC-Controller.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China335 works · 43,758 citations🎓University of California, Los AngelesLos Angeles · United States110 works · 58,035 citations🏛️Beijing Institute for General Artificial IntelligenceBeijing · China14 works · 692 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 works

Cite

Cite as Shaofei Cai, Zihao Wang, Xiaojian Ma, Anji Liu and Yitao Liang. 2023. Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction. arXiv:2301.10034.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2301.10034 · arXiv 2301.10034

@misc{cai2023open,
  title = {{Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction}},
  author = {Shaofei Cai, Zihao Wang, Xiaojian Ma, Anji Liu and Yitao Liang},
  year = {2023},
  doi = {10.48550/arxiv.2301.10034},
  eprint = {2301.10034},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2301.10034}
}

References

11 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Abramson, Arun Ahuja, Arthur Brussee, Federico Carnevale, Mary Cassin, Stephen R. Clark, Andrew Dudzik, Petko Georgiev, Aurelia Guy, Tim Harley, Felix Hill, Alden Hung, Zachary Kenton, Jessica Landon, Timothy P. Lillicrap, Kory Wallace Mathewson, Alistair Muldal, Adam Santoro, Nikolay Savinov,… find
  2. Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, et al. Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691, 2022. 1, 2
  3. Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. arXiv preprint arXiv:2204.14198, 2022. 1
  4. Bowen Baker, Ilge Akkaya, Peter Zhokhov, Joost Huizinga, Jie Tang, Adrien Ecoffet, Brandon Houghton, Raul Sampedro, and Jeff Clune. Video pretraining (VPT): learning to act by watching unlabeled online videos. CoRR, abs/2206.11795, 2022. 3, 6, 7, 8, 12 — [arXiv:2206.11795](https://arxiv.org/abs/2206.11795) find
  5. Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817, 2022. 4 find
  6. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020. 1
  7. Han Cai, Chuang Gan, Ligeng Zhu, and Song Han. Tinytl: Reduce memory, not parameters for efficient on-device learning. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 11285–11297. Curran Associates, Inc.,… find
  8. Tianshi Cao, Jingkang Wang, Yining Zhang, and Sivabalan Manivasagam. Babyai++: Towards grounded-language learning beyond memorization. CoRR, abs/2004.07200, 2020. 8 — [arXiv:2004.07200](https://arxiv.org/abs/2004.07200) find
  9. Maxime Chevalier-Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen, and Yoshua Bengio. Babyai: A platform to study the sample efficiency of grounded language learning. Learning, 2018. 8
  10. Yiming Ding, Carlos Florensa, Pieter Abbeel, and Mariano Phielipp. Goal-conditioned imitation learning. Advances in neural information processing systems, 32, 2019. 2 find
  11. Shibhansh Dohare, A Rupam Mahmood, and Richard S Sutton. Continual backprop: Stochastic gradient descent with persistent randomness. arXiv preprint arXiv:2108.06325, 2021. 6 general conditioning layer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018. 4 [35] Scott Re… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/cai-2023-open-world-multi-task-control/