Elamur Research⌕
☰
home / papers / 2026 / Does RLVR Extend Reasoning Boundaries? Investigating Capability Expansion in Vision-Language Models

Does RLVR Extend Reasoning Boundaries? Investigating Capability Expansion in Vision-Language Models

Paper · 2026 · Minghe Shen, Zhuo Zhi, Chonghan Liu et al.

Summary

The paper asks whether reinforcement learning with verifiable rewards (RLVR) gives vision-language models genuinely new abilities or only makes them more efficient at sampling answers they could already produce. Earlier work on language-only tasks such as math supports the second view. The authors argue that this conclusion is domain-dependent, because visual spatial planning is poorly covered by pre-training. To test this, they built Ariadne, a synthetic grid-maze environment adapted from AlphaMaze. The model reads a maze image and outputs a sequence of moves. Difficulty is set by path length and number of turns, and every step can be checked automatically.

They trained Qwen-family VLMs with GRPO, using two additions:

  • a dense prefix-matching reward, so the model gets a learning signal before it can fully solve a maze;
  • a curriculum that over-samples the easiest and hardest training levels.

After training, the models solved longer mazes on which the base model scored 0% even with large pass@k budgets. The models also improved zero-shot on the real-map benchmarks MapBench and ReasonMap. The authors conclude that RLVR can extend reasoning boundaries in visual-spatial tasks, not merely sharpen existing behavior.

Key points

  • Train/test split: training uses only short mazes (path length ≤ 5, ≤ 2 turns). Evaluation uses unseen, longer mazes (length ≤ 10, ≤ 4 turns), where the base policy fails at every pass@k budget tested.
  • Curriculum: optimal path length s ∈ {1..5} is sampled from an inverted Gaussian-shaped distribution (µ = 3, σ = 2). This places more weight on easy "anchor" levels and frontier levels than on medium ones.
  • ReasonMap, 7B backbone:
    • Long-question weighted accuracy: 6.00% (base) and 4.10% (SFT) vs. 7.47% (Ariadne).
    • Weighted map score on long questions: 4.51 → 5.15.
    • On the hard tier, accuracy is about 21% for Ariadne vs. about 12% for the base model.
  • Gains across model sizes:
    • Qwen2.5-VL-3B-Instruct, ReasonMap long-question accuracy: 3.48% → 4.32%.
    • Qwen3-VL-8B-Instruct, ReasonMap long-question accuracy: 6.24% → 7.73%.
    • On MapBench, scores fall in every scene category (e.g., Mall, Zoo, Campus). The authors say gains are largest for the smaller models.
  • Limitations: the exact difficulty boundary found on synthetic mazes does not carry over directly to real maps. Experiments cover only smaller Qwen2.5-VL and Qwen3-VL models. Code is released on GitHub (MingheShen/Ariadne).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 worksEngineering · 133 worksControl and Systems Engineering · 45 works

🧭 Topics 3Research subjects and languages the work is about.Language models & pretraining · 3,320 worksReasoning · 1,426 worksVision & language · 657 works

Cite

Cite as Minghe Shen, Zhuo Zhi, Chonghan Liu, Shuo Xing, Zhengzhong Tu and Che Liu. 2026. Does RLVR Extend Reasoning Boundaries? Investigating Capability Expansion in Vision-Language Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.2102

@inproceedings{shen2026does,
  title = {{Does RLVR Extend Reasoning Boundaries? Investigating Capability Expansion in Vision-Language Models}},
  author = {Minghe Shen, Zhuo Zhi, Chonghan Liu, Shuo Xing, Zhengzhong Tu and Che Liu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.2102},
  url = {https://doi.org/10.18653/v1/2026.acl-long.2102}
}

References

33 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Essential AI, Darsh J Shah, Peter Rushton, Somanshu Singla, Mohit Parmar, Kurt Smith, Yash Vanjani, Ashish Vaswani, Adarsh Chaluvaraju, Andrew Hojel, Andrew Ma, Anil Thomas, Anthony Polloreno, Ashish Tanwer, Burhan Drak Sibai, Divya S Mans-ingka, Divya Shivaprasad, Ishaan Shah, Karl Stratos, and 9 o… — Rethinking Reflection in Pre-Training
  2. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, and 45 others. 2025a. Qwen3-vl technical report.
  3. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, and 8 others. 2025b. Qwen2.5-vl technical report. arXiv preprint arXiv:2…
  4. Alan Dao and Dinh Bach Vu. 2025. Alphamaze: Enhancing large language models’ spatial intelligence via grpo. arXiv preprint arXiv:2502.14669. — [arXiv:2502.14669](https://arxiv.org/abs/2502.14669) find
  5. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 81 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llm…
  6. Wenhao Deng, Long Wei, Chenglei Yu, and Tailin Wu. 2025. Unlocking reasoning capabilities in llms via reinforcement learning exploration. arXiv preprint arXiv:2510.03865. find
  7. Sicheng Feng, Song Wang, Shuyi Ouyang, Lingdong Kong, Zikai Song, Jianke Zhu, Huan Wang, and Xinchao Wang. 2025. Can mllms guide me home? a benchmark study on fine-grained visual reasoning from transit maps. arXiv preprint arXiv:2505.18675. — [arXiv:2505.18675](https://arxiv.org/abs/2505.18675) find
  8. Yifan Hou, Buse Giledereli, Yilei Tu, and Mrinmaya Sachan. 2025. Do vision-language models really understand visual language? In Proceedings of International Conference on Machine Learning. find
  9. Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. 2025. Vision-r1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749. find
  10. Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, Kai Dang, Yang Fan, Yichang Zhang, An Yang, Rui Men, Fei Huang, Bo Zheng, Yibo Miao, Shanghaoran Quan, and 5 others. 2024. Qwen2.5-coder technical report. arXiv preprint arXiv:2409.1…
  11. Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James Validad Miranda, Alisa Liu, Nouha Dziri, Xinxi Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Christopher Wilhelm, Luca Soldaini,… — Tulu 3: Pushing Frontiers in Open Language Model Post-Training
  12. Shanchao Liang, Spandan Garg, and Roshanak Zilouchian Moghaddam. 2025. The swe-bench illusion: When state-of-the-art llms remember instead of reason. arXiv preprint arXiv:2506.12286. find
  13. Che Liu, Haozhe Wang, Jiazhen Pan, Zhongwei Wan, Yong Dai, Fangzhen Lin, Wenjia Bai, Daniel Rueckert, and Rossella Arcucci. 2025a. Beyond distillation: Pushing the limits of medical llm reasoning with minimalist rule-based rl. arXiv preprint arXiv:2505.17952.
  14. Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. 2025b. Understanding r1-zero-like training: A critical perspective. In Proceedings of Conference on Language Modeling.
  15. Alexandre Matton, Tom Sherborne, Dennis Aumiller, Elena Tommasone, Milad Alizadeh, Jingyi He, Raymond Ma, Maxime Voisin, Ellen Gilsenan-McMahon, and Matthias Gallé. 2024. On leakage of code generation evaluation datasets. In Findings of the Association for Computational Linguistics: EMNLP 2024, page… find
  16. Seyed Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. 2025. GSM-symbolic: Understanding the limitations of mathematical reasoning in large language models. In Proceedings of The International Conference on Learning Representations.
  17. OpenAI, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, and 243 others. 2024. O… — OpenAI o1 System Card
  18. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Train…
  19. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. find
  20. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  21. Zhongwei Wan, Zhihao Dou, Che Liu, Yu Zhang, Dongfei Cui, Qinjian Zhao, Hui Shen, Jing Xiong, Yi Xin, Yifan Jiang, Chaofan Tao, Yangfan He, Mi Zhang, and Shen Yan. 2025. Srpo: Enhancing multimodal llm reasoning via reflection-aware reinforcement learning. arXiv preprint arXiv:2506.01713. — [arXiv:2506.01713](https://arxiv.org/abs/2506.01713) find
  22. Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. 2025a. Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning. arXiv preprint arXiv:2504.08837. find
  23. Haozhe Wang, Qixin Xu, Che Liu, Junhong Wu, Fangzhen Lin, and Wenhu Chen. 2025b. Emergent hierarchical reasoning in llms through reinforcement learning. arXiv preprint arXiv:2509.03646. — [arXiv:2509.03646](https://arxiv.org/abs/2509.03646) find
  24. Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. 2023. How far can camels go? exploring the state of instruction tuning on open resources. In Proceedings of the Internati…
  25. Mingqi Wu, Zhihao Zhang, Qiaole Dong, Zhiheng Xi, Jun Zhao, Senjie Jin, Xiaoran Fan, Yuhao Zhou, Huijie Lv, Ming Zhang, Yanwei Fu, Qin Liu, Songyang Zhang, and Qi Zhang. 2026. Reasoning or memorization? unreliable results of reinforcement learning due to data contamination. In Proceedings of the AAA… find
  26. Shuo Xing, Zezhou Sun, Shuangyu Xie, Kaiyuan Chen, Yanjia Huang, Yuping Wang, Jiachen Li, Dezhen Song, and Zhengzhong Tu. 2025. Can large vision language models read maps like a human? arXiv preprint arXiv:2503.14607. — [arXiv:2503.14607](https://arxiv.org/abs/2503.14607) find
  27. Sen Xu, Yi Zhou, Wei Wang, Jixin Min, Zhibin Yin, Yingwei Dai, Shixi Liu, Lianyu Pang, Yirong Chen, and Junlin Zhang. 2025. Tiny model, big logic: Diversity-driven optimization elicits large-model reasoning ability in vibethinker-1.5b. find
  28. An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. 2024. Qwen2.5-math technical report: Toward mathematical expert model via self-improvement. a…
  29. Guanghao Ye, Khiem Duc Pham, Xinzhi Zhang, Sivakanth Gopi, Baolin Peng, Beibin Li, Janardhan Kulkarni, and Huseyin A Inan. 2025. On the emergence of thinking in llms i: Searching for the right intuition. arXiv preprint arXiv:2502.06773. — [arXiv:2502.06773](https://arxiv.org/abs/2502.06773) find
  30. Huaiyuan Ying, Shuo Zhang, Linyang Li, Zhejian Zhou, Yunfan Shao, Zhaoye Fei, Yichuan Ma, Jiawei Hong, Kuikun Liu, Ziyi Wang, Yudong Wang, Zijian Wu, Shuaibin Li, Fengzhe Zhou, Hongwei Liu, Songyang Zhang, Wenwei Zhang, Hang Yan, Xipeng Qiu, and 3 others. 2024. Internlm-math: Open math large languag… find
  31. Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. 2025. Does reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model? In Proceedings of the International Conference on Neural Information Processing Systems.
  32. Duzhen Zhang, Zhong-Zhi Li, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Xiuyi Chen, Yingying Zhang, Fei Yin, Jiahua Dong, Zhijiang Guo, Le Song, and Cheng-Lin Liu. 2025. From System 1 to System 2: A Survey of Reasoning Large Language Models . IEEE Transactions…
  33. Rosie Zhao, Alexandru Meterez, Sham Kakade, Cengiz Pehlevan, Samy Jelassi, and Eran Malach. 2025. Echo chamber: Rl post-training amplifies behaviors learned in pretraining. arXiv preprint arXiv:2504.07912. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shen-2026-does-rlvr-extend-reasoning-boundaries/