Elamur Research⌕
☰
home / papers / 2025 / VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model

VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model

Paper · 2025 · Haozhan Shen, Peng Liu, Jingcheng Li et al.

Summary

The paper asks whether DeepSeek R1-style reinforcement learning with rule-based rewards can help vision-language models (VLMs) on perception tasks. Many such tasks have exact ground-truth labels, and VLMs often trail much smaller specialist detectors on them. The authors build VLM-R1, an open framework on top of open-r1. It supports GRPO, LoRA, multi-node and multi-image training, mixed image-text and text-only data, and the QwenVL and InternVL model families. They apply it to two tasks that both output bounding boxes: comprehending a referring expression to locate the described object (REC), and detecting objects drawn from an open vocabulary (OVD). The training data contains no reasoning annotations, and rewards are computed directly from box quality (e.g., IoU). Compared with SFT, the RL models do better and transfer especially well to hard out-of-domain benchmarks. The ablations show three things: a naive mAP reward invites reward hacking, data with richer semantics helps, and larger base models gain more from RL. These findings offer practical advice for reward and data design in multimodal RL.

Key points

  • Base models such as Qwen2.5-VL-3B (3.75B parameters) underperform Grounding DINO (341M), for example 37.5 vs 55.0 on ODinW. This gap motivates the RL approach.
  • On REC, the 3B RL model reaches 63.16 on the out-of-domain LISA-Grounding benchmark, compared with 54.82 for SFT.
  • On OVD, the 3B RL model gets 21.1 AP on COCO (17.8 for SFT, 14.2 for the 7B baseline). It also sets a new SOTA of 31.01 nms-AP on OVDEval (26.50 for SFT, 29.08 for the 7B model).
  • Reward hacking and the fix: the plain mAP reward pushes the model to output many redundant boxes. The proposed odLength reward penalizes this behavior. With it, the model shows an "OD aha moment": it first reasons about whether the object is present and only then predicts boxes.
  • Data and scale: training on D³ beats training on COCO, even on the in-domain COCO filtered set. Across model sizes, RL clearly improves the Relation sub-task (e.g., 20.6→27.0 for 32B). On Negation, however, the 3B model drops slightly (38.7→37.7). Code and weights are released at github.com/om-ai-lab/VLM-R1.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksVision & language · 811 works

Cite

Cite as Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, Ruochen Xu and Tiancheng Zhao. 2025. VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model. arXiv:2504.07615.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2504.07615 · arXiv 2504.07615

@misc{shen2025vlm,
  title = {{VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model}},
  author = {Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, Ruochen Xu and Tiancheng Zhao},
  year = {2025},
  doi = {10.48550/arxiv.2504.07615},
  eprint = {2504.07615},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2504.07615}
}

References

13 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Easyr1: An efficient, scalable, multi-modality rl training framework. https://github.com/hiyouga/ EasyR1, 2025. 4 ↗
  2. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. 3
  3. Arash Ahmadian, Chris Cremer, Matthias Gallé, Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet Üstün, and Sara Hooker. Back to basics: Revisiting reinforce style optimization for learning from human feedback in llms. arXiv preprint arXiv:2402.14740, 2024. 3 find
  4. Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716–23736, 2022. 3
  5. Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in ai safety. arXiv preprint arXiv:1606.06565, 2016. 9
  6. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025. 2, 3
  7. Kezhen Chen, Rahul Thapa, Rahul Chalamala, Ben Athiwaratkun, Shuaiwen Leon Song, and James Zou. Dragonfly: Multi-resolution zoom supercharges large visual-language model. arXiv preprint arXiv:2406.00977, 2024. 3 — [arXiv:2406.00977](https://arxiv.org/abs/2406.00977) find
  8. Lin Chen, Jisong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. arXiv preprint arXiv:2311.12793, 2023. 3
  9. Liang Chen, Lei Li, Haozhe Zhao, Yifan Song, and Vinci. R1-v: Reinforcing super generalization ability in vision-language models with less than $3. https://github.com/Deep-Agent/R1-V, 2025. Accessed: 2025-02-02. 3, 4 ↗
  10. Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024. 2, 3
  11. Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. arXiv preprint arXiv:2404.16821, 2024. 2, 3
  12. Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. Sft memorizes, rl generalizes: A comparative study of foundation model post-training. arXiv preprint arXiv:2501.17161, 2025. 11
  13. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shen-2025-vlm-r1-stable-generalizable-r1/