Elamur Research⌕
☰
home / papers / 2026 / Ascending the Infinite Ladder: Benchmarking Spatial Deformation Reasoning in Vision-Language Models

Ascending the Infinite Ladder: Benchmarking Spatial Deformation Reasoning in Vision-Language Models

Paper · 2026 · Jiahuan Zhang, Shunwen Bai, Tianheng Wang et al.

Summary

Vision-language models are increasingly marketed as spatially capable, but existing spatial benchmarks mostly test static relations such as relative height or path finding. They rarely test whether a model can follow an object's shape as it is cut, rotated, stacked or recolored. The authors present Inf-Bench, which targets this gap. It covers three levels: 2D and 2.5D tasks based on the factory game Shapez, and 3D tasks built on the Rubik's Cube. Each level has forward questions (predict the result of given operations) and inverse questions (recover the operations linking two states).

An automatic generator produces unlimited new items, which avoids data leakage. Evaluation uses an "Infinite Ladder" protocol: models start with one-step tasks and move to longer deformation chains as they succeed. Their score is the highest rung they reach, so the benchmark should not saturate.

Humans beat every tested model, and the gap is widest in 3D, where even OpenAI o3 fails basic items. Fine-tuning and prompting-based reasoning aids help little. The authors conclude that deformation reasoning remains an open weakness of current VLMs.

Key points

  • Task design:
    • 2D shapes have four quadrants and six operations (cutting, rotating, coloring and similar).
    • 2.5D adds up to four stacked layers and uses seven operations.
    • 3D uses a 27-unit cube with 54 visible faces and all 54 basic rotations.
  • Main result: In the extracted Table 4 numbers, GPT-5 reaches mean top ranks of roughly 8–10 on 2D/2.5D over 10 ladder runs. It drops to about 1 on 3D forward and 0 on 3D inverse. Llama 4-scout stays below 1 on every task.
  • Model strategies: The authors identify two approaches, "direct pairing" (intuitive pattern matching) and step-by-step encoding of the figure. Both hit clear performance limits.
  • Fine-tuning: Supervised fine-tuning helps models generalize on low-dimensional tasks. On 3D forward tasks, models stay below about seven steps even when trained on sequences of up to 10 steps. Training on forward tasks transfers to inverse tasks better than the reverse.
  • Reasoning enhancements: CoT, few-shot prompting, self-reflection, tool use and ReAct give marginal or negative changes. For example, self-reflection lowers GPT-5's 2D forward score by about 5.3 ranks.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,685 worksVision & language · 811 works

Cite

Cite as Jiahuan Zhang, Shunwen Bai, Tianheng Wang, KaiWen Guo, Zijia Song, Hanqing WU, Guozheng Rao, Kai Han and Kaicheng Yu. 2026. Ascending the Infinite Ladder: Benchmarking Spatial Deformation Reasoning in Vision-Language Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1119

@inproceedings{zhang2026ascending,
  title = {{Ascending the Infinite Ladder: Benchmarking Spatial Deformation Reasoning in Vision-Language Models}},
  author = {Jiahuan Zhang, Shunwen Bai, Tianheng Wang, KaiWen Guo, Zijia Song, Hanqing WU, Guozheng Rao, Kai Han and Kaicheng Yu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1119},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1119}
}

References

47 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anthropic. 2025. Introducing claude opus 4.5. https://www.anthropic.com/news/claude-opus-4-5. ↗
  2. Wenxiao Cai, Iaroslav Ponomarenko, Jianhao Yuan, Xiaoqi Li, Wankou Yang, Hao Dong, and Bo Zhao. 2024. Spatialbot: Precise spatial understanding with vision language models. arXiv preprint arXiv:2406.13642. — [arXiv:2406.13642](https://arxiv.org/abs/2406.13642) find
  3. Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. 2024. Spatialvlm: Endowing vision-language models with spatial reasoning capabilities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14455–14465. find
  4. An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. 2024a. Spatialrgpt: Grounded spatial reasoning in vision language models. arXiv preprint arXiv:2406.01584. — [arXiv:2406.01584](https://arxiv.org/abs/2406.01584) find
  5. Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, and 1 others. 2024b. Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms. arXiv preprint arXiv:2406.07476.
  6. James Gain and Dominique Bechmann. 2008. A survey of spatial deformation from a user-centered perspective. ACM Transactions on Graphics (TOG), 27(4):1–21. find
  7. Chuang Gan, Jeremy Schwartz, Seth Alter, Damian Mrowca, Martin Schrimpf, James Traer, Julian De Freitas, Jonas Kubilius, Abhishek Bhandwaldar, Nick Haber, Megumi Sano, Kuno Kim, Elias Wang, Michael Lingelbach, Aidan Curtis, Kevin Feigelis, Daniel M. Bear, Dan Gutfreund, David Cox, and 5 others. 2021… — [arXiv:2007.04954](https://arxiv.org/abs/2007.04954) find
  8. Akash Ghosh, Arkadeep Acharya, Sriparna Saha, Vinija Jain, and Aman Chadha. 2024. Exploring the frontier of vision-language models: A survey of current methodologies and future directions. arXiv preprint arXiv:2404.07214. — [arXiv:2404.07214](https://arxiv.org/abs/2404.07214) find
  9. Google DeepMind. 2025a. Gemini 3 flash. https://deepmind.google/models/gemini/flash/. ↗
  10. Google DeepMind. 2025b. Gemini 3 pro. https://deepmind.google/models/gemini/pro/. ↗
  11. Yifan Hou, Buse Giledereli, Yilei Tu, and Mrinmaya Sachan. 2024. Do vision-language models really understand visual language? arXiv preprint arXiv:2410.00193. — [arXiv:2410.00193](https://arxiv.org/abs/2410.00193) find
  12. OpenAI. 2024a. Openai o1 system card. OpenAI Website. Https://cdn.openai.com/o1-system-card-20241205.pdf, Accessed: 2025-05-14.
  13. OpenAI. 2024b. Openai o3 and o4-mini system card. OpenAI Website. Https://openai.com/index/o3-o4-mini-system-card/, Accessed: 2025-05-14. find
  14. Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and 1 others. 2024. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326.
  15. OpenAI. 2025a. Introducing gpt-5. https://openai.com/zh-Hans-CN/index/introducing-gpt-5/. ↗
  16. Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Yike Yuan, Wangbo Zhao, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin. 2024. MMBench: Is your multi-modal model an all-around player?
  17. Santhosh Kumar Ramakrishnan, Erik Wijmans, Philipp Kraehenbuehl, and Vladlen Koltun. 2025. Does spatial cognition emerge in frontier models? In The Thirteenth International Conference on Learning Representations. find
  18. Yuecheng Liu, Dafeng Chi, Shiguang Wu, Zhanguang Zhang, Yaochen Hu, Lingfeng Zhang, Yingxue Zhang, Shuang Wu, Tongtong Cao, Guowei Huang, and 1 others. 2025. Spatialcot: Advancing spatial reasoning through coordinate alignment and chain-of-thought for embodied task planning. arXiv preprint arXiv:250… — [arXiv:2501.10074](https://arxiv.org/abs/2501.10074) find
  19. Arijit Ray, Jiafei Duan, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan A Plummer, Ranjay Krishna, Kuo-Hao Zeng, and 1 others. 2024. Sat: Spatial aptitude training for multimodal language models. arXiv preprint arXiv:2412.07755. find
  20. Enhui Ma, Jiahuan Zhang, Guantian Zheng, Tao Tang, Shengbo Eben Li, Yuhang Lu, Xia Zhou, Xueyang Zhang, Yifei Zhan, Kun Zhan, and 1 others. 2026a. Drivecombo: Benchmarking compositional traffic rule reasoning in autonomous driving. arXiv preprint arXiv:2603.01637. — [arXiv:2603.01637](https://arxiv.org/abs/2603.01637) find
  21. Enhui Ma, Lijun Zhou, Tao Tang, Jiahuan Zhang, Junpeng Jiang, Zhan Zhang, Dong Han, Kun Zhan, Xueyang Zhang, Xianpeng Lang, and 1 others. 2026b. Correctad: A self-correcting agentic system to improve end-to-end planning in autonomous driving. In Proceedings of the AAAI Conference on Artificial Intel… find
  22. Julius Mayer, Mohamad Ballout, Serwan Jassim, Farbod Nosrat Nezami, and Elia Bruni. 2025. ivispar–an interactive visual-spatial reasoning benchmark for vlms. arXiv preprint arXiv:2502.03214. find
  23. AI Meta. 2025. The llama 4 herd: The beginning of a new era of natively multimodal ai innovation, april 2025. find
  24. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393.
  25. Bryan L. M. de Oliveira, Murilo L. da Luz, Bruno Brandão, Luana G. B. Martins, Telma W. de L. Soares, and Luckeciano C. Melo. 2025. Sliding puzzles gym: a scalable benchmark for state representation in visual reinforcement learning. arXiv preprint. ArXiv:2410.14038 [cs]. — [arXiv:2410.14038](https://arxiv.org/abs/2410.14038) find
  26. OpenAI. 2025b. Introducing gpt-5.2. https://openai.com/index/introducing-gpt-5-2/. ↗
  27. Parshin Shojaee, Ngoc-Hieu Nguyen, Kazem Meidani, Amir Barati Farimani, Khoa D. Doan, and Chandan K. Reddy. 2025. LLM-SRBench: a new benchmark for scientific equation discovery with large language models. arXiv preprint. ArXiv:2504.10415 [cs]. find
  28. Tobias Springer. 2020. Shapez. https://store.steampowered.com/app/1318690/shapez/. PC, Released June 7, 2020. ↗
  29. Kexian Tang, Junyao Gao, Yanhong Zeng, Haodong Duan, Yanan Sun, Zhening Xing, Wenran Liu, Kaifeng Lyu, and Kai Chen. 2025a. Lego-puzzles: How good are mllms at multi-step spatial reasoning? arXiv preprint arXiv:2503.19990. — [arXiv:2503.19990](https://arxiv.org/abs/2503.19990) find
  30. Yihong Tang, Ao Qu, Zhaokai Wang, Dingyi Zhuang, Zhaofeng Wu, Wei Ma, Shenhao Wang, Yunhan Zheng, Zhan Zhao, and Jinhua Zhao. 2025b. Sparkle: mastering basic spatial capabilities in vision language models elicits generalization to spatial reasoning. arXiv preprint. ArXiv:2410.16162 [cs]. find
  31. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penche… find
  32. Trieu H Trinh, Yuhuai Wu, Quoc V Le, He He, and Thang Luong. 2024. Solving olympiad geometry without human demonstrations. Nature, 625(7995):476–482.
  33. Ziyu Wan, Xidong Feng, Muning Wen, Stephen Marcus McAleer, Ying Wen, Weinan Zhang, and Jun Wang. 2024. Alphazero-like tree-search can guide large language model decoding and training. In Forty-first International Conference on Machine Learning. find
  34. Xingrui Wang, Wufei Ma, Tiezheng Zhang, Celso M de Melo, Jieneng Chen, and Alan Yuille. 2025. Pulsecheck457: A diagnostic benchmark for 6d spatial reasoning of large multimodal models. arXiv e-prints, pages arXiv–2502. find
  35. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837.
  36. Jinyang Wu, Mingkuan Feng, Shuai Zhang, Ruihan Jin, Feihu Che, Zengqi Wen, and Jianhua Tao. 2025. Boosting multimodal reasoning with mcts-automated structured thinking. arXiv preprint arXiv:2502.02339. — [arXiv:2502.02339](https://arxiv.org/abs/2502.02339) find
  37. Wenshan Wu, Shaoguang Mao, Yadong Zhang, Yan Xia, Li Dong, Lei Cui, and Furu Wei. 2024. Mind’s eye of llms: Visualization-of-thought elicits spatial reasoning in large language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. xAI. 2025. grok-4-1. find
  38. Grok 4.1. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR).
  39. Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. 2025. Limo: Less is more for reasoning. arXiv preprint arXiv:2502.03387.
  40. Weichen Zhan, Zile Zhou, Zhiheng Zheng, Chen Gao, Jinqiang Cui, Yong Li, Xinlei Chen, and Xiao-Ping Zhang. 2025. Open3dvqa: A benchmark for comprehensive spatial reasoning with multimodal large language model in open space. arXiv preprint arXiv:2503.11094. — [arXiv:2503.11094](https://arxiv.org/abs/2503.11094) find
  41. Jiahuan Zhang, Tianheng Wang, Ziyi Huang, Yulong Wu, Hanqing Wu, DongbaiChen DongbaiChen, Linfeng Song, Yue Zhang, Guozheng Rao, and Kaicheng Yu. 2025. Sr-llm: Rethinking the structured representation in large language model. In Proceedings of the 63rd Annual Meeting of the Association for Computati…
  42. Ruohong Zhang, Bowen Zhang, Yanghao Li, Haotian Zhang, Zhiqing Sun, Zhe Gan, Yinfei Yang, Ruoming Pang, and Yiming Yang. 2024. Improve vision language model chain-of-thought reasoning. arXiv preprint arXiv:2410.16198. https://x.ai/news/ ↗
  43. Renqiu Xia, Mingsheng Li, Hancheng Ye, Wenjie Wu, Hongbin Zhou, Jiakang Yuan, Tianshuo Peng, Xinyu Cai, Xiangchao Yan, Bin Wang, Conghui He, Botian Shi, Tao Chen, Junchi Yan, and Bo Zhang. 2025. Geox: Geometric problem solving through unified formalized vision-language pre-training. In The Thirteent… find
  44. Weiye Xu, Jiahao Wang, Weiyun Wang, Zhe Chen, Wengang Zhou, Aijun Yang, Lewei Lu, Houqiang Li, Xiaohua Wang, Xizhou Zhu, and 1 others. 2025. Visulogic: A benchmark for evaluating visual reasoning in multi-modal large language models. arXiv preprint arXiv:2504.15279. — [arXiv:2504.15279](https://arxiv.org/abs/2504.15279) find
  45. Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. 2024. Thinking in space: How multimodal large language models see, remember, and recall spaces. arXiv preprint arXiv:2412.14171.
  46. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik R Narasimhan. 2023a. Tree of thoughts: Deliberate problem solving with large language models. In Thirty-seventh Conference on Neural Information Processing Systems. find
  47. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023b. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-ascending-infinite-ladder-benchmarking-spatial/