Elamur Research⌕
☰
home / papers / 2025 / Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought

Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought

Paper · 2025 · Yi Peng, Peiyu Wang, Xiaokun Wang et al.

Summary

Vision-language models describe images well but fall behind text-only reasoning LLMs on multi-step tasks such as geometry proofs. Aligning a vision backbone with a reasoning LLM usually needs large amounts of costly R1-style multimodal reasoning data. Skywork R1V, a 38B open-weight model from Skywork AI, avoids this by separating visual alignment from reasoning.

First, an MLP projector learns to connect a ViT encoder with Qwen2.5-32B-Instruct, a non-reasoning model with the same architecture, using ordinary multimodal data. The adapter is then moved onto DeepSeek-R1-distill-Qwen2.5-32B. From that point only the adapter is trained, while the encoder and LLM stay frozen. Training alternates rounds of SFT on reward-filtered and previously failed samples, followed by GRPO. Training data come from an Adaptive-Length CoT distillation pipeline that rates samples with GPT-4o and adjusts chain length to limit overthinking.

The model scores 69.0 on MMMU and 67.5 on MathVista while keeping strong text-math results. This points to an inexpensive route to multimodal reasoning.

Key points

  • Architecture and release: The model combines ViT, an MLP adapter and DeepSeek-R1-distill-Qwen2.5-32B. Weights are public on Hugging Face as Skywork/Skywork-R1V-38B.

  • Adapter pretraining: The adapter goes through three SFT passes with a frozen encoder and LLM:

    • 2M samples;
    • 200K samples selected with GPT-4;
    • 40K CoT samples.

    The preliminary Qwen-based model reaches 64.0 on MMMU. After the swap to the R1-distilled LLM, with no extra tuning, it still scores 60.2.

  • Iterative SFT: Four rounds (T=4) use a reward model scoring 0–5, with the threshold rising from 2 to 5, plus a set of earlier errors. MMMU improves from 62.5 to 65.6 across these rounds.

  • GRPO stage: Training uses accuracy and format rewards, 8 generations per prompt and 8k-token completions. It lifts MMMU to 69.0 and makes outputs longer.

  • Benchmark results:

    • MMMU: 69.0, close to GPT-4o (69.1) and above Claude 3.5 Sonnet (66.4) and InternVL2.5-38B (63.9).
    • MathVista: 67.5, above Claude 3.5 Sonnet (65.3) but below InternVL2.5-38B (71.9).
    • Text math: 72.0 on AIME 2024 and 94.0 on MATH500.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,685 worksVision & language · 811 works

Cite

Cite as Yi Peng, Peiyu Wang, Xiaokun Wang, Yichen Wei, Jiangbo Pei, Weijie Qiu, Ai Jian, Yunzhuo Hao, Jiachun Pan, Tianyidan Xie, Li Ge, Rongxian Zhuang, Xuchen Song, Yang Liu and Yahui Zhou. 2025. Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought. arXiv:2504.05599.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2504.05599 · arXiv 2504.05599

@misc{peng2025skywork,
  title = {{Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought}},
  author = {Yi Peng, Peiyu Wang, Xiaokun Wang, Yichen Wei, Jiangbo Pei, Weijie Qiu, Ai Jian, Yunzhuo Hao, Jiachun Pan, Tianyidan Xie, Li Ge, Rongxian Zhuang, Xuchen Song, Yang Liu and Yahui Zhou},
  year = {2025},
  doi = {10.48550/arxiv.2504.05599},
  eprint = {2504.05599},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2504.05599}
}

References

23 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anthropic. Claude-3.5, 2024. 1, 7 find
  2. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025. 7
  3. Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024. 7
  4. DeepSeek-AI. Deepseek-v3 technical report, 2024. 7 find
  5. DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. 1, 3, 7
  6. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 3
  7. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021. 6
  8. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024. 7
  9. Ji Lin, Hongxu Yin, Wei Ping, Yao Lu, Pavlo Molchanov, Andrew Tao, Huizi Mao, Jan Kautz, Mohammad Shoeybi, and Song Han. Vila: On pre-training for visual language models, 2023. 7
  10. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255, 2023. 6
  11. OpenAI. Gpt-4 technical report, 2023. 1
  12. OpenAI. Gpt-4o system card, 2024. 7
  13. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024. 6
  14. Gemini Team. Gemini: A family of highly capable multimodal models, 2024. 2
  15. Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599, 2025. 2, 7
  16. Qwen Team. Qwen2.5: A party of foundation models, September 2024. 3 find
  17. Qwen Team. Qwq: Reflect deeply on the boundaries of the unknown, November 2024. 7 find
  18. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision-language model’s perception of the world a… — Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
  19. Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, Jiazheng Xu, Bin Xu, Juanzi Li, Yuxiao Dong, Ming Ding, and Jie Tang. Cogvlm: Visual expert for pretrained language models, 2023. 1
  20. Wenhai Wang, Zhe Chen, Xiaokang Chen, Jiannan Wu, Xizhou Zhu, Gang Zeng, Ping Luo, Tong Lu, Jie Zhou, Yu Qiao, and Jifeng Dai. VisionLLM: Large language model is also an open-ended decoder for vision-centric tasks. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. 1 find
  21. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022. 2
  22. Rui Yang, Lin Song, Yanwei Li, Sijie Zhao, Yixiao Ge, Xiu Li, and Ying Shan. Gpt4tools: Teaching large language model to use tools via self-instruction. arXiv preprint arXiv:2305.18752, 2023. 1
  23. Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patt…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/peng-2025-skywork-r1v-pioneering-multimodal/