Elamur Research⌕
☰
home / papers / 2026 / Current Agents Fail to Leverage World Model as Tool for Foresight

Current Agents Fail to Leverage World Model as Tool for Foresight

Paper · 2026 · Cheng Qian, Emre Can Acikgoz, Bingxuan Li et al.

Summary

The paper asks whether today's vision-language agents can treat a generative world model as an optional external tool: a simulator they consult to preview the consequences of an action before committing to it in the real environment. The authors build an evaluation protocol in which, at every step, the agent chooses either to simulate an action in the world model or to execute it in the environment. Three prompting regimes are compared. In Normal mode, simulation is optional. In WM Invisible mode, the agent does not know the simulator exists. In WM Force mode, the agent must simulate before acting. The testbed covers agentic tasks, which use a ground-truth simulator, and visual question answering, which uses WAN2.1 rollouts. The results are mostly negative. Agents seldom call the simulator, often misread its predictions, and sometimes perform worse when simulation is offered or required. Attribution analysis places the bottleneck in the agents' judgment rather than in access to foresight. The weak points are deciding when to simulate, interpreting the predicted outcomes, and folding them into later decisions. This matters for anyone building agents that are expected to plan with anticipation.

Key points

  • Headline numbers: some agents invoke simulation in under 1% of cases and misuse rollouts roughly 15% of the time. Offering or mandating simulation can lower performance by as much as 5%.
  • Model differences: reluctance to use the world model grows with model scale. LLaMA variants are relatively willing to simulate, while GPT and Qwen variants often decline across sizes. Gemini and Claude were excluded because of cost.
  • Failure taxonomy: most regressions trace to three governance breakdowns:
    • poorly formulated simulator inputs;
    • ambiguous interpretation of simulated outcomes;
    • unstable integration of foresight into actions.
  • Training ablation: limited training on interaction trajectories (FrozenLake, Navigate, SAT, Spatial) raised WM usage by about 13%. However, accuracy barely moved: FrozenLake 0.45 → 0.48, SAT 0.65 → 0.69, and out-of-domain 3DSRBench 0.54 → 0.53.
  • Proposed remedies:
    • a structured Decider / Reflector / Memory loop;
    • multi-turn RL with rewards for appropriate, diverse world-model queries;
    • SFT cold-start data;
    • information-gain objectives.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Cheng Qian, Emre Can Acikgoz, Bingxuan Li, Xiusi Chen, Yuji Zhang, Bingxiang He, Qinyu Luo, Gokhan Tur, Dilek Hakkani-Tür, Yunzhu Li and Heng Ji. 2026. Current Agents Fail to Leverage World Model as Tool for Foresight. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.623

@inproceedings{qian2026current,
  title = {{Current Agents Fail to Leverage World Model as Tool for Foresight}},
  author = {Cheng Qian, Emre Can Acikgoz, Bingxuan Li, Xiusi Chen, Yuji Zhang, Bingxiang He, Qinyu Luo, Gokhan Tur, Dilek Hakkani-Tür, Yunzhu Li and Heng Ji},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.623},
  url = {https://doi.org/10.18653/v1/2026.acl-long.623}
}

References

38 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Saaket Agashe, Kyle Wong, Vincent Tu, Jiachen Yang, Ang Li, and Xin Eric Wang. 2025. Agent s2: A compositional generalist-specialist framework for computer use agents. arXiv preprint arXiv:2504.00906. — [arXiv:2504.00906](https://arxiv.org/abs/2504.00906) find
  2. Meta AI. 2025. The llama 4 herd: The beginning of a new era of natively multimodal ai. https://ai.meta.com/blog/ llama-4-multimodal-intelligence/. ↗
  3. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923. — Qwen2.5-VL Technical Report
  4. Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. 2024a. Video generation models as world simulators. find
  5. Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, et al. 2024b. Video generation models as world simulators. OpenAI Blog, 1(8):1. find
  6. Quentin Carbonneaux, Gal Cohen, Jonas Gehring, Jacob Kahn, Jannik Kossen, Felix Kreuk, Emily McMilin, Michel Meyer, Yuxiang Wei, David Zhang, et al. 2025. Cwm: An open-weights llm for research on code generation with world models. arXiv preprint arXiv:2510.02387. — [arXiv:2510.02387](https://arxiv.org/abs/2510.02387) find
  7. Hyungjoo Chae, Namyoung Kim, Kai Tzu iunn Ong, Minju Gwak, Gwanwoo Song, Jihoon Kim, Sunghwan Kim, Dongha Lee, and Jinyoung Yeo. 2024. Web agents with world models: Learning and leveraging environment dynamics in web navigation. Preprint, arXiv:2410.13232. — [arXiv:2410.13232](https://arxiv.org/abs/2410.13232) find
  8. ICLR 2025 submission (arXiv preprint). find
  9. Mingkai Deng, Jinyu Hou, Zhiting Hu, and Eric Xing. 2025a. Simura: A world-model-driven simula-tive reasoning architecture for general goal-oriented agents. arXiv preprint arXiv:2507.23773. — [arXiv:2507.23773](https://arxiv.org/abs/2507.23773) find
  10. Xiang Deng, Jeff Da, Edwin Pan, Yannis Yiming He, Charles Ide, Kanak Garg, Niklas Lauffer, Andrew Park, Nitin Pasari, Chetan Rane, Karmini Sampath, Maya Krishnan, Srivatsa Kundurthy, Sean Hendryx, Zifan Wang, Vijay Bharadwaj, Jeff Holm, Raja Aluri, Chen Bo Calvin Zhang, Noah Jacobson, Bing Liu, and… — [arXiv:2509.16941](https://arxiv.org/abs/2509.16941) find
  11. Lutfi Eren Erdogan, Nicholas Lee, Sehoon Kim, Suhong Moon, Hiroki Furuta, Gopala Anumanchipalli, Kurt Keutzer, and Amir Gholami. 2025. Plan-and-act: Improving planning of agents for long-horizon tasks. arXiv preprint arXiv:2503.09572. — [arXiv:2503.09572](https://arxiv.org/abs/2503.09572) find
  12. Yiguo Fan, Pengxiang Ding, Shuanghao Bai, Xinyang Tong, Yuyang Zhu, Hongchao Lu, Fengqi Dai, Wei Zhao, Yang Liu, Siteng Huang, Zhaoxin Fan, Badong Chen, and Donglin Wang. 2025. Long-vla: Unleashing long-horizon capability of vision language action model for robot manipulation. arXiv preprint arXiv:2… — [arXiv:2508.19958](https://arxiv.org/abs/2508.19958) find
  13. Yunhai Feng, Jiaming Han, Zhuoran Yang, Xiangyu Yue, Sergey Levine, and Jianlan Luo. 2025. Reflective planning: Vision-language models for multi-stage long-horizon robotic manipulation. arXiv preprint arXiv:2502.16707. — [arXiv:2502.16707](https://arxiv.org/abs/2502.16707) find
  14. Divyansh Garg, Shaun VanWeelden, Diego Caples, Andis Draguns, Nikil Ravi, Pranav Putta, Naman Garg, Tomas Abraham, Michael Lara, Federico Lopez, James Liu, Atharva Gundawar, Prannay Hebbar, Youngchul Joo, Jindong Gu, Charles London, Christian Schroeder de Witt, and Sumeet Motwani. 2025. Real: Benchm… — [arXiv:2504.11543](https://arxiv.org/abs/2504.11543) find
  15. Longling Geng and Edward Y. Chang. 2025. Realm-bench: A benchmark for evaluating multi-agent systems on real-world, dynamic planning and scheduling tasks. arXiv preprint arXiv:2502.18836. — [arXiv:2502.18836](https://arxiv.org/abs/2502.18836) find
  16. Daniel T. Gilbert and Timothy D. Wilson. 2007. Prospection: Experiencing the future. Science, 317(5843):1351–1354. find
  17. Alexi Gladstone, Ganesh Nanduru, Md Mofijul Islam, Peixuan Han, Hyeonjeong Ha, Aman Chadha, Yilun Du, Heng Ji, Jundong Li, and Tariq Iqbal. 2025. Energy-based transformers are scalable learners and thinkers. In arxiv. find
  18. Shibo Hao, Yi Gu, Haodi Ma, Joshua Hong, Zhen Wang, Daisy Wang, and Zhiting Hu. 2023. Reasoning with language model is planning with world model. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8154–8173. find
  19. Haoran He, Yang Zhang, Liang Lin, Zhongwen Xu, and Ling Pan. 2025. Pre-trained video generative models as world simulators. In ICLR 2025 Workshop on World Models: Understanding, Modelling and Scaling. find
  20. Wenlong Huang, Chen Wang, Yunzhu Li, Ruohan Zhang, and Li Fei-Fei. 2024. Rekep: Spatiotemporal reasoning of relational keypoint constraints for robotic manipulation. arXiv preprint arXiv:2409.01652. — [arXiv:2409.01652](https://arxiv.org/abs/2409.01652) find
  21. PN Johnson-Laird. 1983. Mental models: Towards a cognitive science of language, inference, and consciousness. Harvard University Press. find
  22. Wufei Ma, Haoyu Chen, Guofeng Zhang, Yu-Cheng Chou, Jieneng Chen, Celso de Melo, and Alan Yuille. 2025. 3dsrbench: A comprehensive 3d spatial reasoning benchmark. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6924–6934. find
  23. Siwei Meng, Yawei Luo, and Ping Liu. 2025. Magic: Motion-aware generative inference via confidence-guided llm. arXiv preprint arXiv:2505.16456. — [arXiv:2505.16456](https://arxiv.org/abs/2505.16456) find
  24. OpenAI. 2025. Gpt-5 system card. https://cdn.openai.com/gpt-5-system-card.pdf. ↗
  25. David Premack and Guy Woodruff. 1978. Does the chimpanzee have a theory of mind? Behavioral and Brain Sciences, 1(4):515–526. find
  26. Cheng Qian, Peixuan Han, Qinyu Luo, Bingxiang He, Xiusi Chen, Yuji Zhang, Hongyi Du, Jiarui Yao, Xiaocheng Yang, Denghui Zhang, et al. 2024. Es-capebench: Pushing language models to think outside the box. arXiv preprint arXiv:2412.13549. — [arXiv:2412.13549](https://arxiv.org/abs/2412.13549) find
  27. Arijit Ray, Jiafei Duan, Ellis Brown, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan A Plummer, Ranjay Krishna, et al. 2024. Sat: Dynamic spatial aptitude training for multimodal language models. arXiv preprint arXiv:2412.07755. find
  28. Daniel L. Schacter, Donna Rose Addis, and Randy L. Buckner. 2007. Remembering the past to imagine the future: the prospective brain. Nature Reviews Neuroscience, 8:657–661. find
  29. Fatemeh Shiri, Xiao-Yu Guo, Mona Golestan Far, Xin Yu, Reza Haf, and Yuan-Fang Li. 2024. An empirical analysis on spatial reasoning capabilities of large multimodal models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 21440–21455. find
  30. Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. 2025. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314. find
  31. Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, and Li Fei-Fei. 2023. Voxposer: Composable 3d value maps for robotic manipulation with language models. arXiv preprint arXiv:2307.05973. find
  32. Huacan Wang, Ziyi Ni, Shuo Zhang, Shuo Lu, Sen Hu, Ziyang He, Chen Hu, Jiaye Lin, Yifu Guo, Ronghao Chen, Xin Li, Daxin Jiang, Yuntao Du, and Pin Lyu. 2025a. Repomaster: Autonomous exploration and understanding of github repositories for complex task solving. arXiv preprint arXiv:2505.21577. — [arXiv:2505.21577](https://arxiv.org/abs/2505.21577) find
  33. Emily Jin, Zhuoyi Huang, Jan-Philipp Fränken, Weiyu Liu, Hannah Cha, Erik Brockbank, Sarah Wu, Ruohan Zhang, Jiajun Wu, and Tobias Gerstenberg. 2024. Marple: A benchmark for long-horizon inference. Preprint, arXiv:2410.01926. Preprint. Under review. — [arXiv:2410.01926](https://arxiv.org/abs/2410.01926) find
  34. Kangrui Wang, Pingyue Zhang, Zihan Wang, Yaning Gao, Linjie Li, Qineng Wang, Hanyang Chen, Chi Wan, Yiping Lu, Zhengyuan Yang, et al. 2025b. Vagen: Reinforcing world model reasoning for multi-turn vlm agents. arXiv preprint arXiv:2510.16907. find
  35. Zhenhailong Wang, Haiyang Xu, Junyang Wang, Xi Zhang, Ming Yang, Ji Zhang, Fei Huang, and Heng Ji. 2025c. Mobile-agent-e: Self-evolving mobile assistant for complex real-world tasks. In Proc. NeurIPS2025 Workshop on Scaling Environments for Agents. find
  36. Sihan Yang, Runsen Xu, Yiman Xie, Sizhe Yang, Mo Li, Jingli Lin, Chenming Zhu, Xiaochen Chen, Haodong Duan, Xiangyu Yue, et al. 2025. Mmsi-bench: A benchmark for multi-image spatial intelligence. arXiv preprint arXiv:2505.23764. — [arXiv:2505.23764](https://arxiv.org/abs/2505.23764) find
  37. Jiahan Zhang, Muqing Jiang, Nanru Dai, Taiming Lu, Arda Uzunoglu, Shunchi Zhang, Yana Wei, Jiahao Wang, Vishal M Patel, Paul Pu Liang, et al. 2025a. World-in-world: World models in a closed-loop world. arXiv preprint arXiv:2510.18135. — [arXiv:2510.18135](https://arxiv.org/abs/2510.18135) find
  38. Kevin Zhang, Kuangzhi Ge, Xiaowei Chi, Renrui Zhang, Shaojun Shi, Zhen Dong, Sirui Han, and Shanghang Zhang. 2025b. Can world models benefit vlms for world dynamics? arXiv preprint arXiv:2510.00855. — [arXiv:2510.00855](https://arxiv.org/abs/2510.00855) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/qian-2026-current-agents-fail-leverage-world/