Elamur Research⌕
☰
home / papers / 2026 / MVP: Enhancing Video Large Language Models via Self-supervised Masked Video Prediction

MVP: Enhancing Video Large Language Models via Self-supervised Masked Video Prediction

Paper · 2026 · Xiaokun Sun, Zezhong Wu, Zewen Ding et al.

Summary

RL-based post-training of Video Large Language Models (VideoLLMs) usually relies on tasks such as captioning, VideoQA and grounding. These tasks reward overall content understanding but give little direct signal about temporal order or how frames depend on each other. The authors propose Masked Video Prediction (MVP), a self-supervised "visual cloze" objective. A continuous stretch of a video is hidden, and the model must pick the missing key frames from a shuffled pool that also contains hard distractors taken from the same video. It must then put the chosen frames back in their original order.

A synthesis pipeline turns any video collection into MVP samples, so training data is effectively unlimited. Training uses GRPO with a graded reward that credits correct frame selection and correct ordering separately.

Across several base models and six benchmarks, MVP improves video reasoning, and gains grow steadily with more training data. The work matters because it offers cheap, label-free supervision aimed specifically at temporal and causal understanding.

Key points

  • Task and data pipeline: MVP is posed as multiple choice over K masked key frames mixed with intra-video distractors. The pipeline samples frames at 1 FPS and uses CLIP similarity with a threshold κ to discard near-duplicate frames, which keeps samples informative.
  • Reward design: The GRPO reward has three parts:
    • token-level credit, with α/K for a correct frame in the correct position and a smaller γ/K for a correct frame in the wrong position;
    • a sequence-level bonus for preserved ordered sub-runs, detected via common substrings;
    • a format reward.
  • Why RL instead of SFT: The authors argue that supervised fine-tuning on such out-of-distribution data could erode the model's prior knowledge, so they chose RL.
  • Evaluation: The six benchmarks are VideoMME, LongVideoBench, LVBench, MLVU, Video-Holmes and TempCompass. The authors report consistent gains and a monotonic improvement as data scale increases.
  • Limitations: There is no process-level supervision of the reasoning chain. Training data and duration were limited, so performance had not converged.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Science and Technology of ChinaHefei · China108 works · 112,698 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Xiaokun Sun, Zezhong Wu, Zewen Ding and Linli Xu. 2026. MVP: Enhancing Video Large Language Models via Self-supervised Masked Video Prediction. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.891

@inproceedings{sun2026mvp,
  title = {{MVP: Enhancing Video Large Language Models via Self-supervised Masked Video Prediction}},
  author = {Xiaokun Sun, Zezhong Wu, Zewen Ding and Linli Xu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.891},
  url = {https://doi.org/10.18653/v1/2026.acl-long.891}
}

References

42 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Xiang An, Yin Xie, Kaicheng Yang, Wenkang Zhang, Xiuwei Zhao, Zheng Cheng, Yirui Wang, Songcen Xu, Changrui Chen, Chunsheng Wu, and 1 others. 2025. Llava-onevision-1.5: Fully open framework for democratized multimodal training. arXiv preprint arXiv:2509.23661. — [arXiv:2509.23661](https://arxiv.org/abs/2509.23661) find
  2. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, and 45 others. 2025a. Qwen3-vl technical report. Preprint, a…
  3. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, and 8 others. 2025b. Qwen2.5-vl technical report. Preprint, arXiv:2502.1…
  4. Haoyu Cao, Changcun Bao, Chaohu Liu, Huang Chen, Kun Yin, Hao Liu, Yinsong Liu, Deqiang Jiang, and Xing Sun. 2023. Attention where it matters: Rethinking visual document understanding with selective region concentration. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pag… find
  5. Junhao Cheng, Yuying Ge, Teng Wang, Yixiao Ge, Jing Liao, and Ying Shan. 2025. Video-holmes: Can mllm think like holmes for complex video reasoning? arXiv preprint arXiv:2505.21374. — [arXiv:2505.21374](https://arxiv.org/abs/2505.21374) find
  6. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. Deepseek-r1: Incentivizing reasoning capability in ll…
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,…
  8. Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Junfei Wu, Xiaoying Zhang, Benyou Wang, and Xiangyu Yue. 2025. Video-r1: Reinforcing video reasoning in mllms. arXiv preprint arXiv:2503.21776. — [arXiv:2503.21776](https://arxiv.org/abs/2503.21776) find
  9. Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, and 1 others. 2025a. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. In Proceedings of the Computer Vision and Pattern Rec…
  10. Shenghao Fu, Qize Yang, Yuan-Ming Li, Xihan Wei, Xiaohua Xie, and Wei-Shi Zheng. 2025b. Love-r1: Advancing long video understanding with an adaptive zoom-in mechanism via multi-step reasoning. arXiv preprint arXiv:2509.24786. — [arXiv:2509.24786](https://arxiv.org/abs/2509.24786) find
  11. Jiyang Gao, Chen Sun, Zhenheng Yang, and Ram Nevatia. 2017. Tall: Temporal activity localization via language query. In Proceedings of the IEEE international conference on computer vision, pages 5267– 5275. find
  12. Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng, Guang Shi, Haobin Chen, Haoqi Fan, Jian Wang, Jianyu Jiang, Jiawei Wang, Jingji Chen, Jingjia Huang, Kang Lei, Liping Yuan, Lishu Luo, Pengfei Liu, Qinghao Ye, Rui Qian, Shen Yan, and 178 others. 2025. Seed1.5-vl technical report. Preprint, arXiv:2505.070…
  13. Zefeng He, Xiaoye Qu, Yafu Li, Siyuan Huang, Daizong Liu, and Yu Cheng. 2025. Videossr: Video self-supervised reinforcement learning. Preprint, arXiv:2511.06281. — [arXiv:2511.06281](https://arxiv.org/abs/2511.06281) find
  14. Jie Lei, Tamara L Berg, and Mohit Bansal. 2021. Detecting moments and highlights in videos via natural language queries. Advances in Neural Information Processing Systems, 34:11846–11858. find
  15. Chaohu Liu, Haoyu Cao, YongXiang Hua, and Linli Xu. 2026. Multimodal table understanding with difficulty-aware reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 755–763. find
  16. Chaohu Liu, Kun Yin, Haoyu Cao, Xinghua Jiang, Xin Li, Yinsong Liu, Deqiang Jiang, Xing Sun, and Linli Xu. 2024a. Hrvda: High-resolution visual document assistant. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15534–15545. find
  17. Shuming Liu, Chen Zhao, Tianqi Xu, and Bernard Ghanem. 2025. Bolt: Boost large vision-language model without training for long-form video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 3318–3327. find
  18. Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. 2024b. Tempcompass: Do video llms really understand videos? In Findings of the Association for Computational Linguistics: ACL 2024, pages 8731–8772. find
  19. OpenAI, :, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, and 244 others. 2024. Openai o1 sy…
  20. Jinyoung Park, Jeehye Na, Jinyoung Kim, and Hyunwoo J Kim. 2025. Deepvideo-r1: Video reinforcement fine-tuning via difficulty-aware regressive grpo. arXiv preprint arXiv:2506.07464. — [arXiv:2506.07464](https://arxiv.org/abs/2506.07464) find
  21. Kairui Hu, Penghao Wu, Fanyi Pu, Wang Xiao, Yuanhan Zhang, Xiang Yue, Bo Li, and Ziwei Liu. 2025a. Video-mmmu: Evaluating knowledge acquisition from multi-discipline professional videos. Preprint, arXiv:2501.13826. — [arXiv:2501.13826](https://arxiv.org/abs/2501.13826) find
  22. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8…
  23. Kairui Hu, Penghao Wu, Fanyi Pu, Wang Xiao, Yuanhan Zhang, Xiang Yue, Bo Li, and Ziwei Liu. 2025b. Video-mmmu: Evaluating knowledge acquisition from multi-discipline professional videos. arXiv preprint arXiv:2501.13826. — [arXiv:2501.13826](https://arxiv.org/abs/2501.13826) find
  24. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728–53741.
  25. YongXiang Hua, Haoyu Cao, Zhou Tao, Bocheng Li, Zihao Wu, Chaohu Liu, and Linli Xu. 2025. Input domain aware moe: Decoupling routing decisions from task optimization in mixture of experts. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 5110–5119. find
  26. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https://arxiv.org/abs/2402.03300, 2(3):5. ↗
  27. Canhui Tang, Zifan Han, Hongbo Sun, Sanping Zhou, Xuchong Zhang, Xin Wei, Ye Yuan, Huayu Zhang, Jinglin Xu, and Hao Sun. 2026. Tspo: Temporal sampling policy optimization for long-form video language understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 93… find
  28. Zhou Tao, Shida Wang, Yongxiang Hua, Haoyu Cao, and Linli Xu. 2025. Dig: Differential grounding for enhancing fine-grained perception in multimodal large language model. Preprint, arXiv:2512.12633. — [arXiv:2512.12633](https://arxiv.org/abs/2512.12633) find
  29. V Team, Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, Lihang Pan, Shuaiqi Duan, Weihan Wang, Yan Wang, Yean Cheng, Zehai He, Zhe Su, Zhen Yang, Ziyang Pan, and 69 others. 2025. Glm-4.5v and glm-4.1v-thinking: Towards versatile multimodal reas…
  30. Haonan Wang, Hongfu Liu, Xiangyan Liu, Chao Du, Kenji Kawaguchi, Ye Wang, and Tianyu Pang. 2025a. Fostering video reasoning via next-event prediction. arXiv preprint arXiv:2505.22457. — [arXiv:2505.22457](https://arxiv.org/abs/2505.22457) find
  31. Haozhe Wang, Alex Su, Weiming Ren, Fangzhen Lin, and Wenhu Chen. 2025b. Pixel reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning. Preprint, arXiv:2505.15966. — [arXiv:2505.15966](https://arxiv.org/abs/2505.15966) find
  32. Shida Wang, YongXiang Hua, Zhou Tao, Haoyu Cao, and Linli Xu. 2026. Dynamic token compression for efficient video understanding through reinforcement learning. Preprint, arXiv:2603.26365. — [arXiv:2603.26365](https://arxiv.org/abs/2603.26365) find
  33. Weihan Wang, Zehai He, Wenyi Hong, Yean Cheng, Xiaohan Zhang, Ji Qi, Ming Ding, Xiaotao Gu, Shiyu Huang, Bin Xu, and 1 others. 2025c. Lvbench: An extreme long video understanding benchmark. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22958–22967. find
  34. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, and 56 others. 2025d. Internvl3.5: Advancing open-source multimodal…
  35. Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. 2024. Longvideobench: A benchmark for long-context interleaved video-language understanding. Advances in Neural Information Processing Systems, 37:28828–28857. and Dahua Lin. 2025. Caprl: Stimulating dense image caption capabilities via reinforcement l… find
  36. Mingze Xu, Mingfei Gao, Shiyu Li, Jiasen Lu, Zhe Gan, Zhengfeng Lai, Meng Cao, Kai Kang, Yinfei Yang, and Afshin Dehghan. 2025. Slowfast-llava-1.5: A family of token-efficient video large language models for long-form video understanding. arXiv preprint arXiv:2503.18943. — [arXiv:2503.18943](https://arxiv.org/abs/2503.18943) find
  37. Ziang Yan, Xinhao Li, Yinan He, Zhengrong Yue, Xiangyu Zeng, Yali Wang, Yu Qiao, Limin Wang, and Yi Wang. 2025. Videochat-r1. 5: Visual test-time scaling to reinforce multimodal reasoning by iterative perception. arXiv preprint arXiv:2509.21100. — [arXiv:2509.21100](https://arxiv.org/abs/2509.21100) find
  38. Biao Yang, Bin Wen, Boyang Ding, Changyi Liu, Chenglong Chu, Chengru Song, Chongling Rao, Chuan Yi, Da Li, Dunju Zang, and 1 others. 2025. Kwai keye-vl 1.5 technical report. arXiv preprint arXiv:2509.01563. — [arXiv:2509.01563](https://arxiv.org/abs/2509.01563) find
  39. Shi-Xue Zhang, Hongfa Wang, Duojun Huang, Xin Li, Xiaobin Zhu, and Xu-Cheng Yin. 2026. Vcaps-bench: A large-scale fine-grained benchmark for video caption quality evaluation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 12726–12734. find
  40. Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. 2025. Llava-video: Video instruction tuning with synthetic data. Preprint, arXiv:2410.02713. find
  41. Yilun Zhao, Haowei Zhang, Lujing Xie, Tongyan Hu, Guo Gan, Yitao Long, Zhiyuan Hu, Weiyuan Chen, Chuhan Li, Zhijian Xu, and 1 others. 2025. Mmvu: Measuring expert-level multi-discipline video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 8475–8489. find
  42. Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. 2024. Mlvu: A comprehensive benchmark for multi-task long video understanding. arXiv preprint arXiv:2406.04264, 2(5):6. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sun-2026-mvp-enhancing-video-large-language/