Elamur Research⌕
☰
home / papers / 2026 / Model-Based Imaginative Planning for Embodied Agents

Model-Based Imaginative Planning for Embodied Agents

Paper · 2026 · Junru Song, Hengzhe Jin, Yucong Huang et al.

Summary

Large language models plan well in symbolic domains such as math and code, where transition rules are already absorbed during pretraining. Embodied tasks differ: an agent sees only sparse, partial visual evidence, and each environment has its own dynamics and affordances that language priors do not cover. IMPLEMENT addresses this by attaching a lightweight, self-supervised world model to a frozen LLM.

The world model reads raw pixels and produces object-centric symbolic states: agent location, inventory, and object or receptacle attributes and relations. It then forecasts how those states change under hypothetical actions. Temperature sampling yields several possible futures, which handles uncertainty from partial observability. Meta In-Context Learning lets the model refine its predictions from interaction history, including failed actions. At inference, the LLM proposes actions, the world model simulates them, and the LLM revises its choice. This loop acts as policy iteration without any parameter updates.

On ALFWorld, the method outperforms vanilla VLMs, finetuning-based methods and test-time scaling baselines. It is especially strong when interaction budgets are small. The work suggests a path for grounding language agents in visual environments without retraining them.

Key points

  • A single VLM-based world model runs in two modes. Perception mode records what is currently observable, and prediction mode forecasts the effects of a given action.
  • Baselines include ReAct, Reflexion, Self-consistency and SimuRA. Policy backbones include Gemini-2.5-flash and Qwen2.5-VL-72b, and the ablations use GPT-4.1-mini.
  • The co-reasoning loop allows up to 5 iterations per step, but the LLM usually stops early, averaging 2.4 iterations.
  • Ablations cover MetaICL, imaginative planning, sampling temperature and termination criteria. They use 12 repeated trials and two-tailed Mann-Whitney U tests.
  • Limitations include reliance on privileged supervised transitions and per-step latency of tens of seconds. Code and data are released at github.com/WoodySJR/IMPLEMENT.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 works

Cite

Cite as Junru Song, Hengzhe Jin, Yucong Huang, Tingsong Jiang, Weien Zhou, Feifei Wang, Yang Yang, Ying Wen and Wen Yao. 2026. Model-Based Imaginative Planning for Embodied Agents. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.827

@inproceedings{song2026model,
  title = {{Model-Based Imaginative Planning for Embodied Agents}},
  author = {Junru Song, Hengzhe Jin, Yucong Huang, Tingsong Jiang, Weien Zhou, Feifei Wang, Yang Yang, Ying Wen and Wen Yao},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.827},
  url = {https://doi.org/10.18653/v1/2026.acl-long.827}
}

References

50 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923. — Qwen2.5-VL Technical Report
  3. Siwei Chen, Anxing Xiao, and David Hsu. 2023. Llm-state: Open world state representation for long-horizon task planning with large language model. arXiv preprint arXiv:2311.17406. — [arXiv:2311.17406](https://arxiv.org/abs/2311.17406) find
  4. Zixuan Chen, Jing Huo, Yangtao Chen, and Yang Gao. 2025. Robohorizon: An llm-assisted multi-view world model for long-horizon robotic manipulation. arXiv preprint arXiv:2501.06605. — [arXiv:2501.06605](https://arxiv.org/abs/2501.06605) find
  5. Wei Chow, Jiageng Mao, Boyi Li, Daniel Seita, Vitor Guizilini, and Yue Wang. 2025. Physbench: Benchmarking and enhancing vision-language models for physical world understanding. arXiv preprint arXiv:2501.16411. — [arXiv:2501.16411](https://arxiv.org/abs/2501.16411) find
  6. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arX…
  7. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. 2023. Instructblip: Towards general-purpose vision-language models with instruction tuning. Advances in neural information processing systems, 36:49250–49267.
  8. Mingkai Deng, Jinyu Hou, Zhiting Hu, and Eric Xing. 2025. Simura: A world-model-driven simulative reasoning architecture for general goal-oriented agents. arXiv preprint arXiv:2507.23773. — [arXiv:2507.23773](https://arxiv.org/abs/2507.23773) find
  9. Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, and 1 others. 2023. Palm-e: an embodied multimodal language model. In Proceedings of the 40th International Conference on Machine Learning, pages 8469–8488. find
  10. Peng Gao, Jiaming Han, Renrui Zhang, Ziyi Lin, Shijie Geng, Aojun Zhou, Wei Zhang, Pan Lu, Conghui He, Xiangyu Yue, and 1 others. 2023. Llama-adapter v2: Parameter-efficient visual instruction model. arXiv preprint arXiv:2304.15010. find
  11. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  12. David Ha and Jürgen Schmidhuber. 2018. World models. arXiv preprint arXiv:1803.10122, 2(3). find
  13. Peter Jansen. 2022. A systematic survey of text worlds as embodied natural language environments. In Proceedings of the 3rd Wordplay: When Language Meets Games Workshop (Wordplay 2022), pages 1–15. find
  14. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730–19742. PMLR.
  15. Wenlong Liang, Rui Zhou, Yang Ma, Bing Zhang, Songlin Li, Yijia Liao, and Ping Kuang. 2025. Large model empowered embodied ai: A survey on decision-making and embodied learning. arXiv preprint arXiv:2508.10399. — [arXiv:2508.10399](https://arxiv.org/abs/2508.10399) find
  16. Bingqian Lin, Yunshuang Nie, Khun Loun Zai, Ziming Wei, Mingfei Han, Rongtao Xu, Minzhe Niu, Jianhua Han, Liang Lin, Cewu Lu, and 1 others. 2025. Evolvenav: Self-improving embodied reasoning for llm-based vision-language navigation. arXiv e-prints, pages arXiv–2506. find
  17. Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. 2023a. Libero: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems, 36:44776–44791. find
  18. Zhihan Liu, Hao Hu, Shenao Zhang, Hongyi Guo, Shuqi Ke, Boyi Liu, and Zhaoran Wang. 2023b. Reason for future, act for now: A principled framework for autonomous llm agents with provable sample efficiency. arXiv preprint arXiv:2309.17382. find
  19. Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. 2025. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. IEEE Transactions on Audio, Speech and Language Processing. find
  20. Asher J Hancock, Xindi Wu, Lihan Zha, Olga Russakovsky, and Anirudha Majumdar. 2025. Actions as language: Fine-tuning vlms into vlas without catastrophic forgetting. arXiv preprint arXiv:2509.22195. — [arXiv:2509.22195](https://arxiv.org/abs/2509.22195) find
  21. Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2022. Metaicl: Learning to learn in context. In Proceedings of the 2022 conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2791–2809. find
  22. Shibo Hao, Yi Gu, Haodi Ma, Joshua Hong, Zhen Wang, Daisy Wang, and Zhiting Hu. 2023. Reasoning with language model is planning with world model. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8154–8173. find
  23. Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Mian. 2025. A comprehensive overview of large language models. ACM Transactions on Intelligent Systems and Technology, 16(5):1–72.
  24. Mark K Ho, David Abel, Carlos G Correa, Michael L Littman, Jonathan D Cohen, and Thomas L Griffiths. 2022. People construct simplified mental representations to plan. Nature, 606(7912):129–136. find
  25. Lerrel Pinto, Marcin Andrychowicz, Peter Welinder, Wojciech Zaremba, and Pieter Abbeel. 2017. Asymmetric actor critic for image-based robot learning. arXiv preprint arXiv:1710.06542. find
  26. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3.
  27. Rizwan Qureshi, Ranjan Sapkota, Abbas Shah, Amgad Muneer, Anas Zafar, Ashmal Vayani, Maged Shoman, Abdelrahman Eldaly, Kai Zhang, Ferhat Sadak, and 1 others. 2025. Thinking beyond tokens: From brain-inspired intelligence to cognitive foundations for artificial general intelligence and its societal i… — [arXiv:2507.00951](https://arxiv.org/abs/2507.00951) find
  28. Neil Rabinowitz, Frank Perbet, Francis Song, Chiyuan Zhang, S. M. Ali Eslami, and Matthew Botvinick. 2018. Machine theory of mind. In International Conference on Machine Learning, pages 4218–4227. PMLR. find
  29. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634–8652.
  30. Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. 2020. Alfworld: Aligning text and embodied environments for interactive learning. arXiv preprint arXiv:2010.03768.
  31. David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, and 1 others. 2016. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484–489. find
  32. Austin Stone, Oscar Ramirez, Kurt Konolige, and Rico Jonschkowski. 2021. The distracting control suite–a challenging benchmark for reinforcement learning from pixels. arXiv preprint arXiv:2101.02722. — [arXiv:2101.02722](https://arxiv.org/abs/2101.02722) find
  33. Richard S Sutton, Andrew G Barto, and 1 others. 1998. Reinforcement learning: An introduction, volume 1. MIT press Cambridge. find
  34. Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. 2017. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 23–30. IEEE. find
  35. Andrew Wang, Andrew C Li, Toryn Q Klassen, Rodrigo Toro Icarte, and Sheila A McIlraith. 2023a. Learning belief representations for partially observable deep rl. In International Conference on Machine Learning, pages 35970–35988. PMLR. find
  36. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023b. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291. find
  37. Haonan Wang, Junfeng Sun, Xingdi Yuan, Ruoyao Wang, and Ziang Xiao. 2025a. Bytesized32refactored: Towards an extensible interactive text games corpus for llm world modeling and evaluation. arXiv preprint arXiv:2509.23979. — [arXiv:2509.23979](https://arxiv.org/abs/2509.23979) find
  38. Jun Wang, Meng Fang, Ziyu Wan, Muning Wen, Jiachen Zhu, Anjie Liu, Ziqin Gong, Yan Song, Lei Chen, Lionel M Ni, and 1 others. 2024a. Openr: An open source framework for advanced reasoning with large language models. arXiv preprint arXiv:2410.09671. find
  39. Junqiao Wang, Zhongliang Yu, Dong Zhou, Jiaqi Shi, and Runran Deng. 2024b. Vision-based deep reinforcement learning of uav autonomous navigation using privileged information. arXiv preprint arXiv:2412.06313. — [arXiv:2412.06313](https://arxiv.org/abs/2412.06313) find
  40. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171.
  41. Yucen Wang, Rui Yu, Shenghua Wan, Le Gan, and De-Chuan Zhan. 2025b. Founder: Grounding foundation models in world models for open-ended embodied decision making. In Forty-second International Conference on Machine Learning. find
  42. Zihao Wang, Shaofei Cai, Guanzhou Chen, Anji Liu, Xiaojian Ma, and Yitao Liang. 2023c. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. arXiv preprint arXiv:2302.01560.
  43. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  44. Jun Yamada, Marc Rigter, Jack Collins, and Ingmar Posner. 2024. Twist: Teacher-student world model distillation for efficient sim-to-real transfer. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 9190–9196. IEEE. find
  45. Chang Yang, Xinrun Wang, Junzhe Jiang, Qinggang Zhang, and Xiao Huang. 2024a. Evaluating world models with llm for decision making. arXiv preprint arXiv:2411.08794. — [arXiv:2411.08794](https://arxiv.org/abs/2411.08794) find
  46. Yijun Yang, Tianyi Zhou, Kanxue Li, Dapeng Tao, Lu-song Li, Li Shen, Xiaodong He, Jing Jiang, and Yuhui Shi. 2024b. Embodied multi-modal agent trained by an llm from a parallel textworld. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26275–26285. find
  47. Yue Yang, Fan-Yun Sun, Luca Weihs, Eli Vander-Bilt, Alvaro Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Krishna, Lingjie Liu, and 1 others. 2024c. Holodeck: Language guided generation of 3d embodied ai environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco… find
  48. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. In The eleventh international conference on learning representations.
  49. Ning Zhang, Yongjia Zhao, Minghao Yang, and Shuling Dai. 2025a. Llms augmented hierarchical reinforcement learning with action primitives for long-horizon manipulation tasks. Scientific Reports, 15(1):36779. find
  50. Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Wenyue Hua, Haolun Wu, Zhihan Guo, Yufei Wang, Niklas Muennighoff, and 1 others. 2025b. A survey on test-time scaling in large language models: What, how, where, and how well? arXiv preprint arXiv:2503.24235. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/song-2026-model-based-imaginative-planning-embodied/