Elamur Research⌕
☰
home / papers / 2026 / From Pixels to BFS: High Maze Accuracy Does Not Imply Visual Planning

From Pixels to BFS: High Maze Accuracy Does Not Imply Visual Planning

Paper · 2026 · Rodriguez Salgado, Alberto Gonzalo

Summary

The paper asks whether multimodal LLMs that solve visual mazes actually plan spatially, or whether they reach answers through exhaustive search in text. The author introduces MazeBench, a set of 110 procedurally generated 1024×1024 pixel-art mazes. They are split into nine controlled groups that vary grid size (5×5 to 20×20), wall density, traps, border walls, visual palette and reachability, with BFS-computed ground truth. A maze counts as solved only if the model returns the exact shortest sequence of moves; there is no partial credit. Nineteen configurations from OpenAI, Anthropic, Google and Alibaba are evaluated across different reasoning budgets, and the study records tokens consumed per solve. The best systems score highly: GPT-5.5 reaches 96% and GPT-5.4 91% on the 100-maze core set. Their reasoning traces, however, follow a two-step routine. They first transcribe the image into a text grid, then enumerate paths in prose, essentially running BFS verbally. The work matters because it shows that strong benchmark accuracy can conceal an expensive strategy that looks nothing like human visual reasoning.

Key points

  • Brute-force search: Token cost per solve ranges from 1,096 to 22,818, a 14× gap between GPT-5.5 low and Gemini 3 Flash.
  • Reasoning budget and maze size: With no thinking or the lowest budget, all 19 configurations solve only 2–15%. On 20×20 ultra-hard mazes, GPT-5.5 medium manages just 4/10, typically running out of tokens and answering "unreachable."
  • Text-grid ablation: When given the correct text grid instead of the image, Claude Sonnet 4.6 rises from 6% to 80% and Claude Opus 4.7 from 31% to 90%. This identifies image-to-grid extraction as Claude's main weakness.
  • Generation matters more than size: Within the Claude 4.6 generation, Opus 4.6 scores the same 4% as Haiku 4.5. Gains come from newer model generations rather than larger models.
  • Prompting does not change the strategy: A prompt that forbids grids and graph search does not help. GPT-5.4 low scores 17/20 with it versus 19/20 with the standard prompt, and Sonnet 4.6 quietly reverts to grid enumeration anyway.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 works

Cite

Cite as Rodriguez Salgado, Alberto Gonzalo. 2026. From Pixels to BFS: High Maze Accuracy Does Not Imply Visual Planning. Proceedings of the 4th Workshop on Advances in Language and Vision Research (ALVR).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.alvr-main.13

@inproceedings{gonzalo2026pixels,
  title = {{From Pixels to BFS: High Maze Accuracy Does Not Imply Visual Planning}},
  author = {Rodriguez Salgado, Alberto Gonzalo},
  booktitle = {Proceedings of the 4th Workshop on Advances in Language and Vision Research (ALVR)},
  year = {2026},
  doi = {10.18653/v1/2026.alvr-main.13},
  url = {https://doi.org/10.18653/v1/2026.alvr-main.13}
}

References

20 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aradhye Agarwal, Ayan Sengupta, and Tanmoy Chakraborty. 2025. The art of scaling test-time compute for large language models. arXiv preprint arXiv:2512.02008. — [arXiv:2512.02008](https://arxiv.org/abs/2512.02008) find
  2. Boyuan Chen, Zhuo Xu, Sean Kirmani, Brian Ichter, Danny Driess, Pete Florence, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. 2024a. SpatialVLM: Endowing vision-language models with spatial reasoning capabilities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVP… find
  3. Liang Chen, Weichu Xie, Yiyan Liang, Hongfeng He, Hans Zhao, Zhibo Yang, and Ziqi Huang. 2026. BabyVision: Visual reasoning beyond language. arXiv preprint arXiv:2601.06521. — [arXiv:2601.06521](https://arxiv.org/abs/2601.06521) find
  4. Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, and Feng Zhao. 2024b. Are we on the right way for evaluating large vision-language models? Advances in Neural Information Processing Systems, 37.
  5. An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. 2024. SpatialRGPT: Grounded spatial reasoning in vision-language models. In Advances in Neural Information Processing Systems (NeurIPS). find
  6. Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, Rongrong Ji, Caifeng Shan, and Ran He. 2024a. MME: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306.13394.
  7. Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A. Smith, Wei-Chiu Ma, and Ranjay Krishna. 2024b. BLINK: Multimodal large language models can see but not perceive. In European Conference on Computer Vision (ECCV), pages 148–166. find
  8. Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, and Yu Cheng. 2025. Can MLLMs reason in multimodality? EMMA: An enhanced multimodal reasoning benchmark. In International Conference on Machine Learning (ICML). find
  9. Jinsheng Huang, Liang Chen, Taian Guo, Fu Zeng, Yusheng Zhao, Bohan Wu, Ye Yuan, Haozhe Zhao, Zhihui Guo, Yichi Zhang, Jingyang Yuan, Wei Ju, Luchen Liu, Tianyu Liu, Baobao Chang, and Ming Zhang. 2025. MMEvalPro: Calibrating multimodal benchmarks towards trustworthy and efficient evaluation. In Proc… find
  10. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2024. MathVista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representations (ICLR).
  11. Yiming Qin, Bomin Wei, Jiaxin Ge, Konstantinos Kallidromitis, Stephanie Fu, Trevor Darrell, and Xudong Wang. 2025. Chain-of-visual-thought: Teaching VLMs to see and think better with continuous visual tokens. arXiv preprint arXiv:2511.19418. find
  12. Navid Rajabi and Jana Kosecka. 2024. GSR-Bench: A benchmark for grounded spatial reasoning evaluation via multimodal LLMs. In NeurIPS 2024 Workshop on Compositional Learning. find
  13. Yufan Ren, Konstantinos Tertikas, Shalini Maiti, Junlin Han, Tong Zhang, Sabine Süsstrunk, and Filippos Kokkinos. 2025. VGRP-Bench: Visual grid reasoning puzzle benchmark for large vision-language models. arXiv preprint arXiv:2503.23064. — [arXiv:2503.23064](https://arxiv.org/abs/2503.23064) find
  14. Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2025. Scaling LLM test-time compute optimally can be more effective than scaling model parameters. In International Conference on Learning Representations (ICLR).
  15. Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. 2024. Eyes wide shut? Exploring the visual shortcomings of multimodal LLMs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  16. Shengbang Tong, Yuexiang Zhai, Lingjie Liu, Yann Le-Cun, Yi Ma, and Saining Xie. 2026. SpatiaLab: Can vision-language models perform spatial reasoning in the wild? In International Conference on Learning Representations (ICLR). find
  17. Yaoting Wang, Shengqiong Wu, Yuecheng Zhang, Shuicheng Yan, Ziwei Liu, Jiebo Luo, and Hao Fei. 2025. Multimodal chain-of-thought reasoning: A comprehensive survey. arXiv preprint arXiv:2503.12605. — [arXiv:2503.12605](https://arxiv.org/abs/2503.12605) find
  18. Peiran Xu, Sudong Wang, Yao Zhu, Jianing Li, and Yunjian Zhang. 2025. SpatialBench: Benchmarking multimodal large language models for spatial cognition. arXiv preprint arXiv:2511.21471. — [arXiv:2511.21471](https://arxiv.org/abs/2511.21471) find
  19. Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, and 3 others. 2024. MMMU: A massive multi-discipline multimodal u…
  20. Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Peng Gao, and Hongsheng Li. 2024. MathVerse: Does your multi-modal LLM truly see find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/salgado-2026-pixels-bfs-high-maze-accuracy/