Elamur Research⌕
☰
home / papers / 2026 / Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Models

Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Models

Paper · 2026 · Hoang Phan, Xianjun Yang, Yuanshun Yao et al.

Summary

Reinforcement learning with verifiable rewards (RLVR) improves math and multimodal reasoning, but it tends to wear away skills a model already had, such as perception, OCR and robustness. The authors audit open reasoning-tuned VLMs built on Qwen2.5-VL. These models usually score below their base versions on non-reasoning benchmarks. MiMo-VL-7B-RL, trained with mixed on-policy RL, is an exception and stays close to its SFT version. The authors propose RECAP (Replay-Enhanced CApability Preservation). It mixes general-domain data (detection, segmentation, OCR) back into RLVR training alongside reasoning tasks such as geometry, algebra, charts and diagrams. It also adjusts each objective's weight online, based on its recent convergence speed and instability. Saturated signals like format rewards lose priority, and harder, noisier ones gain it. The scheduler needs no auxiliary models, is magnitude-agnostic and adds little overhead. On Qwen2.5-VL-3B and Qwen2.5-VL-7B, RECAP preserves or improves general abilities while matching or beating reasoning-only finetuning. This matters for adding reasoning to a model without sacrificing its broader competence.

Key points

  • The forgetting audit covered A-OKVQA, AesBench, VStar, VisOnly, OCRBench and R-Bench-Dis. Reasoning-tuned Qwen2.5-VL variants consistently fell behind on VisOnlyQA.
  • Training Qwen2.5-VL-7B on reasoning rewards alone reduced perception performance by roughly 7%. On out-of-domain LISA segmentation, RECAP raised IoU by about 2%, while the reasoning-only model fell below the base after around 100 iterations.
  • Reward signals converge at very different rates, with some up to roughly three times faster than others. This makes fixed or hand-tuned weights suboptimal.
  • The ablation compared RECAP with uniform weighting (λ_k = 1/K). The uniform baseline gains format reward faster at first, but RECAP overtakes it near step 40 and steadily widens its lead on accuracy.
  • With RECAP, average thinking length on the reasoning task dropped from about 67 to about 27 words (around 60%) with no accuracy loss. When a single thinking reward was applied to every domain, the model abandoned chain-of-thought on segmentation, so thinking rewards are reserved for reasoning tasks.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Johns Hopkins UniversityBaltimore · United States240 works · 96,487 citations🎓New York UniversityNew York · United States201 works · 100,180 citations🏢Meta (United Kingdom)London · United Kingdom46 works · 18,033 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Hoang Phan, Xianjun Yang, Yuanshun Yao, Jingyu Zhang, Shengjie Bi, Xiaocheng Tang, Madian Khabsa, Lijuan Liu and Deren Lei. 2026. Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1717

@inproceedings{phan2026beyond,
  title = {{Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Models}},
  author = {Hoang Phan, Xianjun Yang, Yuanshun Yao, Jingyu Zhang, Shengjie Bi, Xiaocheng Tang, Madian Khabsa, Lijuan Liu and Deren Lei},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1717},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1717}
}

References

120 references, 35 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. 2021. HME100K: A dataset for handwritten mathematical expressions. https://github.com/Phymond/ HME100K. Accessed October 2025. ↗
  2. Josh Achiam, Steven Adler, Sandhini Agarwal, and et al. 2023. GPT-4 technical report. arXiv:2303.08774.
  3. Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. 2018. Memory aware synapses: Learning what (not) to forget. In ECCV, pages 139–154. find
  4. Thomas Borsani, Andrea Rosani, Giuseppe Nicosia, and Giuseppe Di Fatta. 2025. Gradient similarity surgery in multi-task deep learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 95–111. Springer. find
  5. Ralph Allan Bradley and Milton E Terry. 1952. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324– 345. find
  6. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  7. Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. 2020. Dark experience for general continual learning: A strong, simple baseline. In NeurIPS, volume 33, pages 15920– 15930. find
  8. Nikhil Chandak, Shashwat Goel, and Ameya Prabhu. 2025. Incorrect baseline evaluations call into question recent llm-rl claims. https://safe-lip-9a8.notion.site/ Incorrect-Baseline-Evaluations-Call-into-Question-Recent pvs=4. Notion Blog. ↗
  9. Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. 2019. Efficient lifelong learning with A-GEM. In ICLR. find
  10. Jiaze Chen and 1 others. 2025a. Seed1.5-thinking: Advancing superb reasoning models with reinforcement learning.
  11. Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, and 1 others. 2025b. Reasoning models don’t always say what they think. arXiv preprint arXiv:2505.05410. find
  12. Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich. 2018. GradNorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In Proceedings of the 35th International Conference on Machine Learning, volume 80, pages 794–803. find
  13. Zhao Chen, Jiquan Ngiam, Yanping Huang, Thang Luong, Henrik Kretzschmar, Yuning Chai, and Dragomir Anguelov. 2020. Just pick a sign: Optimizing deep multitask models with gradient sign dropout. In Advances in Neural Information Processing Systems, volume 33, pages 2039–2050. find
  14. Xu Chu, Xinrong Chen, Guanyu Wang, Zhijie Tan, Kui Huang, Wenyu Lv, Tong Mo, and Weiping Li. 2025. Qwen look again: Guiding vision-language reasoning models to re-attention visual information. arXiv:2505.23558. Introduces BRPO with reflection and visual re-attention to reduce hallucinations. — [arXiv:2505.23558](https://arxiv.org/abs/2505.23558) find
  15. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, and 1 others. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53. find
  16. Roberto Cipolla, Yarin Gal, and Alex Kendall. 2018. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7482–7491. find
  17. Xingyu Dang, Christina Baek, J Zico Kolter, and Aditi Raghunathan. 2025. Assessing diversity collapse in reasoning. In Scaling Self-Improving Foundation Models without Human Supervision. find
  18. Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022. Flashattention: Fast and memory-efficient exact attention with io-awareness. In Advances in Neural Information Processing Systems (NeurIPS). find
  19. Daxiang Dong, Hua Wu, Wei He, Dianhai Yu, and Haifeng Wang. 2015. Multi-task learning for multiple language translation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Lon…
  20. Haodong Duan, Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang, Pan Zhang, Jiaqi Wang, and 1 others. 2024. Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 11… find
  21. K. Ethayarajh. 2024. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306. find
  22. Robert M. French. 1999. Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences, 3(4):128–135. find
  23. Ling Fu, Biao Yang, Zhebin Kuang, Jiajun Song, Yuzhe Li, Linghao Zhu, Qidi Luo, Xinyu Wang, Hao Lu, Mingxin Huang, Zhang Li, Guozhi Tang, Bin Shan, Chunhui Lin, Qi Liu, Binghong Wu, Hao Feng, Hao Liu, Can Huang, and 5 others. 2024. Ocrbench v2: An improved benchmark for evaluating large multimodal m… — [arXiv:2501.00321](https://arxiv.org/abs/2501.00321) find
  24. Yuqian Fu, Tinghong Chen, Jiajun Chai, Xihuai Wang, Songjun Tu, Guojun Yin, Wei Lin, Qichao Zhang, Yuanheng Zhu, and Dongbin Zhao. 2025. Srft: A single-stage method with supervised and reinforcement fine-tuning for reasoning. arXiv preprint arXiv:2506.19767. — [arXiv:2506.19767](https://arxiv.org/abs/2506.19767) find
  25. Shivank Garg, Ayush Singh, Shweta Singh, and Paras Chopra. 2025. Ipo: Your language model is secretly a preference classifier. arXiv preprint arXiv:2502.16182. find
  26. Danna Gurari, Qing Li, Abigale J. Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P. Bigham. 2018. Vizwiz grand challenge: Answering visual questions from blind people. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). find
  27. Zhezheng Hao, Hong Wang, Haoyang Liu, Jian Luo, Jiarui Yu, Hande Dong, Qiang Lin, Can Wang, and Jiawei Chen. 2025. Rethinking entropy interventions in rlvr: An entropy change perspective. arXiv preprint arXiv:2510.10150.
  28. Peter Henderson, Riashat Islam, Philip Bachman, Joelle Pineau, Doina Precup, and David Meger. 2018. Deep reinforcement learning that matters. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32. find
  29. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv:1503.02531.
  30. Jiwoo Hong, Noah Lee, and James Thorne. 2024. Orpo: Monolithic preference optimization without reference model. arXiv preprint arXiv:2403.07691. find
  31. Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146. find
  32. Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. 2025a. Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model. arXiv preprint arXiv:2503.24290.
  33. Zhiyuan Hu, Yibo Wang, Hanze Dong, Yuhui Xu, Amrita Saha, Caiming Xiong, Bryan Hooi, and Junnan Li. 2025b. Beyond’aha!’: Toward systematic meta-abilities alignment in large reasoning models. arXiv preprint arXiv:2505.10554. — [arXiv:2505.10554](https://arxiv.org/abs/2505.10554) find
  34. Yipo Huang, Quan Yuan, Xiangfei Sheng, Zhichao Yang, Haoning Wu, Pengfei Chen, Yuzhe Yang, Leida Li, and Weisi Lin. 2024. Aesbench: An expert benchmark for multimodal large language models on image aesthetics perception. arXiv preprint arXiv:2401.08276. — [arXiv:2401.08276](https://arxiv.org/abs/2401.08276) find
  35. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720.
  36. Daya Guo and DeepSeek-AI. 2025. Deepseek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature. find
  37. Yunjie Ji, Xiaoyu Tian, Sitong Zhao, Haotian Wang, Shuaiting Chen, Yiping Peng, Han Zhao, and Xiangang Li. 2025. Am-thinking-v1: Advancing the frontier of reasoning at 32b scale. arXiv preprint arXiv:2505.08311. — [arXiv:2505.08311](https://arxiv.org/abs/2505.08311) find
  38. Yihang Guo, Tianyuan Yu, Liang Bai, Yanming Guo, Yirun Ruan, William Li, and Weishi Zheng. 2025. Revisit the imbalance optimization in multi-task learning: An experimental analysis. arXiv preprint arXiv:2509.23915. — [arXiv:2509.23915](https://arxiv.org/abs/2509.23915) find
  39. Ryo Kamoi, Yusen Zhang, Sarkar Snigdha Sarathi Das, Ranran Haoran Zhang, and Rui Zhang. 2025. Visonlyqa: Large vision language models still struggle with visual perception of geometric information. In Conference on Language Modeling (COLM). find
  40. Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg. 2014. ReferItGame: Referring to objects in photographs of natural scenes. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 787– 798, Doha, Qatar. Association for Computational Lin…
  41. Aniruddha Kembhavi, Michael Salvato, Bardia Fang, Alaaeldin El-Nouby, Ludovic Schmidt, and 1 others. 2016. A diagram is worth a dozen images: Developing a diagram taxonomy for diagram understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Dataset page…
  42. Yiqing Liang, Jielin Qiu, Wenhao Ding, Zuxin Liu, James Tompkin, Mengdi Xu, Mengzhou Xia, Zhengzhong Tu, Laixi Shi, and Jiacheng Zhu. 2025. Modomodo: Multi-domain data mixtures for multimodal llm reinforcement learning. Preprint, arXiv:2505.24871. — [arXiv:2505.24871](https://arxiv.org/abs/2505.24871) find
  43. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023a. Let’s verify step by step: Improving mathematical reasoning with process supervision. In OpenAI Technical Report. Introduces PRM800K and shows advanta…
  44. Diederik P Kingma. 2015. Adam: A method for stochastic optimization. The third International Conference on Learning Representations.
  45. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023b. Let’s verify step by step. In The Twelfth International Conference on Learning Representations.
  46. James Kirkpatrick, Razvan Pascanu, and Neil et al. Rabinowitz. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13):3521–3526. find
  47. Baijiong Lin, Feiyang Ye, Yu Zhang, and Ivor W Tsang. 2022. Reasonable effectiveness of random weighting: A litmus test for multi-task learning. Transactions on Machine Learning Research, pages 2835–8856. find
  48. Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. 2023. Lisa: Reasoning segmentation via large language model. find
  49. Bo Liu, Yihao Feng, Peter Stone, and Qiang Liu. 2023. Famo: Fast adaptive multitask optimization. In Advances in Neural Information Processing Systems, volume 36, pages 57226–57243. find
  50. Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. 2024. Lisa: Reasoning segmentation via large language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). find
  51. Nathan Lambert. 2025. Reinforcement learning from human feedback. arXiv preprint arXiv:2504.12501. — [arXiv:2504.12501](https://arxiv.org/abs/2504.12501) find
  52. Bo Li, Kaichen Zhang, and Andrés Marafioti. 2025a. Multimodal open r1. https://github.com/ EvolvingLMMs-Lab/open-r1-multimodal. Accessed: 2025-02-08. ↗
  53. Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326.
  54. Chunyi Li, Jianbo Zhang, Zicheng Zhang, Haoning Wu, Yuan Tian, Wei Sun, Guo Lu, Xiongkuo Min, Xiaohong Liu, Weisi Lin, and 1 others. 2025b. R-bench: Are your large multimodal model robust to real-world corruptions? IEEE Journal of Selected Topics in Signal Processing. find
  55. Zhizhong Li and Derek Hoiem. 2016. Learning without forgetting. In ECCV, pages 614–629. find
  56. Zhizhong Li and Derek Hoiem. 2017. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935–2947. find
  57. Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. 2021a. Conflict-averse gradient descent for multi-task learning. In Advances in Neural Information Processing Systems, volume 34, pages 18878– 18890. find
  58. Haotian Liu and 1 others. 2024a. Mmbench: Is your multi-modal model an all-around player? In European Conference on Computer Vision (ECCV).
  59. Liyang Liu, Yi Li, Zhanghui Kuang, Jing-Hao Xue, Yimin Chen, Wenming Yang, Qingmin Liao, and Wayne Zhang. 2021b. Towards impartial multi-task learning. In International Conference on Learning Representations. find
  60. Mingjie Liu, Shizhe Diao, Ximing Lu, Jian Hu, Xin Dong, Yejin Choi, Jan Kautz, and Yi Dong. 2025a. Prorl: Prolonged reinforcement learning expands reasoning boundaries in large language models. arXiv preprint arXiv:2505.24864. find
  61. Shikun Liu, Edward Johns, and Andrew J. Davison. 2019. End-to-end multi-task learning with attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1871–1880. find
  62. Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu-Cheng Yin, Cheng-Lin Liu, Lianwen Jin, and Xiang Bai. 2024b. Ocrbench: on the hidden mystery of ocr in large multimodal models. Science China Information Sciences, 67(12):220102.
  63. Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. 2025b. Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783.
  64. Zifan Liu and 1 others. 2024c. Measuring multimodal mathematical reasoning with the math-vision dataset. In Advances in Neural Information Processing Systems (NeurIPS). find
  65. David Lopez-Paz and Marc’Aurelio Ranzato. 2017. Gradient episodic memory for continual learning. In NeurIPS, pages 6467–6476. find
  66. Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. find
  67. Xinyue Lou, You Li, Jinan Xu, Xiangyu Shi, Chi Chen, and Kaiyu Huang. 2025. Think in safety: Unveiling and mitigating safety alignment collapse in multimodal large reasoning model. arXiv preprint arXiv:2505.06538. find
  68. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2023. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255.
  69. Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022a. Learn to explain: Multimodal reasoning via thought chains for science question answering. In The 36th Conference on Neural Information Processing Systems (NeurIPS). find
  70. Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022b. Learn to explain: Multimodal reasoning via thought chains for science question answering. In NeurIPS. find
  71. U.-V. Marti and H. Bunke. 2002. The IAM-database: an english sentence database for offline handwriting recognition. International Journal on Document Analysis and Recognition, 5(1):39–46. find
  72. Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. In Findings of ACL.
  73. Minesh Mathew, Viraj Bagal, Rubèn Pérez Tito, Dimosthenis Karatzas, Ernest Valveny, and C. V. Jawahar. 2022. Infographicvqa. In WACV. find
  74. Michael McCloskey and Neal J. Cohen. 1989. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of Learning and Motivation, volume 24, pages 109–165. Academic Press. find
  75. Paulius Micikevicius, Sharan Narang, Jonah Alben, Greg Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, and Hao Wu. 2018. Mixed precision training. In International Conference on Learning Representations (ICLR). find
  76. Ankush Mishra, Karteek Alahari, and C. V. Jawahar. 2012. Scene text recognition using higher order language priors. In British Machine Vision Conference (BMVC). find
  77. Aviv Navon, Aviv Shamsian, Idan Achituve, Haggai Maron, Kenji Kawaguchi, Gal Chechik, and Ethan Fetaya. 2022. Multi-task learning as a bargaining game. In Proceedings of the 39th International Conference on Machine Learning (ICML). find
  78. Long Ouyang, Jeff Wu, Xu Jiang, and et al. 2022. Training language models to follow instructions with human feedback. In NeurIPS.
  79. Hao Peng, Yunjia Qi, Xiaozhi Wang, Bin Xu, Lei Hou, and Juanzi Li. 2025. Verif: Verification engineering for reinforcement learning in instruction following. arXiv preprint arXiv:2506.09942.
  80. Ivo Petrov, Jasper Dekoninck, Lyuben Baltadzhiev, Maria Drencheva, Kristian Minchev, Mislav Balunović, Nikola Jovanović, and Martin Vechev. 2025. Proof or bluff? evaluating llms on 2025 usa math olympiad. arXiv preprint arXiv:2503.21934. find
  81. Hoang Phan, Lam Tran, Quyen Tran, Ngoc Tran, Tuan Truong, Qi Lei, Nhat Ho, Dinh Phung, and Trung Le. 2025. Beyond losses reweighting: Empowering multi-task learning via the generalization perspective. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2440–2450. find
  82. Mihir Prabhudesai, Lili Chen, Alex Ippoliti, Katerina Fragkiadaki, Hao Liu, and Deepak Pathak. 2025. Maximizing confidence alone improves reasoning. arXiv preprint arXiv:2505.22660. — [arXiv:2505.22660](https://arxiv.org/abs/2505.22660) find
  83. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728–53741.
  84. Arijit Ray, Jiafei Duan, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan A. Plummer, Ranjay Krishna, Kuo-Hao Zeng, and Kate Saenko. 2024a. Sat: Spatial aptitude training for multimodal language models. find
  85. Arijit Ray, Jiafei Duan, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan A. Plummer, Ranjay Krishna, Kuo-Hao Zeng, and Kate Saenko. 2024b. Sat: Spatial aptitude training for multimodal language models. arXiv preprint arXiv:2412.07755. find
  86. Sylvestre Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. 2017. iCaRL: Incremental classifier and representation learning. In CVPR, pages 5533–5542. find
  87. David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy P. Lillicrap, and Greg Wayne. 2019. Experience replay for continual learning. In NeurIPS, pages 350– 360. find
  88. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. find
  89. Dmitry Senushkin, Nikolay Patakin, Arseny Kuznetsov, and Anton Konushin. 2023. Independent component alignment for multi-task learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20083–20093. find
  90. Amrith Setlur, Chirag Nagpal, Adam Fisch, Xinyang Geng, Jacob Eisenstein, Rishabh Agarwal, Alekh Agarwal, Jonathan Berant, and Aviral Kumar. 2024. Rewarding progress: Scaling automated process verifiers for llm reasoning. arXiv preprint arXiv:2410.08146. find
  91. Amrith Setlur, Chirag Nagpal, Adam Fisch, Xinyang Geng, Jacob Eisenstein, Rishabh Agarwal, Alekh Agarwal, Jonathan Berant, and Aviral Kumar. 2025. Rewarding progress: Scaling automated process verifiers for LLM reasoning. In The Thirteenth International Conference on Learning Representations. find
  92. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  93. Parshin Shojaee, Iman Mirzadeh, Keivan Alizadeh, Maxwell Horton, Samy Bengio, and Mehrdad Farajtabar. 2025. The illusion of thinking: Understanding the strengths and limitations of reasoning models via the lens of problem complexity. arXiv preprint arXiv:2506.06941. find
  94. Core Team, Zihao Yue, Zhenru Lin, Yifan Song, Weikun Wang, Shuhuai Ren, Shuhao Gu, Shicheng Li, Peidian Li, Liang Zhao, Lei Li, Kainan Bao, Hao Tian, Hailin Zhang, Gang Wang, Dawei Zhu, Cici, Chenhong He, Bowen Ye, and 55 others. 2025. Mimo-vl technical report. Preprint, arXiv:2506.03569.
  95. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  96. Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. 2025a. Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning. arXiv preprint arXiv:2504.08837. find
  97. Shibo Wang and Przemyslaw Kanwar. 2019. Bfloat16: The secret to high performance on cloud tpus. Google Cloud Blog. find
  98. Teng Wang, Zhangyi Jiang, Zhenqi He, Shenyang Tong, Wenhan Yang, Yanan Zheng, Zeyu Li, Zifan He, and Hailei Gong. 2025b. Towards hierarchical multi-step reward models for enhanced reasoning in large language models. arXiv preprint arXiv:2503.13551.
  99. Xiyao Wang, Zhengyuan Yang, Chao Feng, Hongjin Lu, Linjie Li, Chung-Ching Lin, Kevin Lin, Furong Huang, and Lijuan Wang. 2025c. Sota with less: Mcts-guided sample selection for data-efficient visual reasoning self-improvement. arXiv preprint arXiv:2504.07934. — [arXiv:2504.07934](https://arxiv.org/abs/2504.07934) find
  100. Zhenhailong Wang, Xuehang Guo, Sofia Stoica, Haiyang Xu, Hongru Wang, Hyeonjeong Ha, Xiusi Chen, Yangyi Chen, Ming Yan, Fei Huang, and 1 others. 2025d. Perception-aware policy optimization for multimodal reasoning. arXiv preprint arXiv:2507.06448. — [arXiv:2507.06448](https://arxiv.org/abs/2507.06448) find
  101. Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652.
  102. Oleksii Sidorov, Ronghang Hu, Marcus Rohrbach, and Amanpreet Singh. 2020. Textcaps: A dataset for image captioning with reading comprehension. In European Conference on Computer Vision (ECCV). find
  103. Penghao Wu and Saining Xie. 2023. V*: Guided visual search as a core mechanism in multimodal llms. arXiv preprint arXiv:2312.14135. find
  104. A. Singh and 1 others. 2021. TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text. arXiv preprint arXiv:2105.05486. — [arXiv:2105.05486](https://arxiv.org/abs/2105.05486) find
  105. Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. 2025. Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2502.14768.
  106. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020. Learning to summarize with human feedback. Advances in neural information processing systems, 33:3008– 3021. find
  107. Zhengyuan Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. 2023. The dawn of lmms: Preliminary explorations with gpt-4v (ision). arXiv preprint arXiv:2309.17421.
  108. Yang Yao, Xuan Tong, Ruofan Wang, Yixu Wang, Lu-jundong Li, Liang Liu, Yan Teng, and Yingchun Wang. 2025a. A mousetrap: Fooling large reasoning models for jailbreak with chain of iterative chaos. arXiv preprint arXiv:2502.15806. find
  109. Daniel M. Ziegler, Nisan Stiennon, Jeff Wu, and et al. 2019. Fine-tuning language models from human preferences. arXiv:1909.08593. find
  110. Zijun Yao, Yantao Liu, Yanxu Chen, Jianhui Chen, Junfeng Fang, Lei Hou, Juanzi Li, and Tat-Seng Chua. 2025b. Are reasoning models more prone to hallucination? arXiv preprint arXiv:2505.23646. — [arXiv:2505.23646](https://arxiv.org/abs/2505.23646) find
  111. En Yu, Kangheng Lin, Liang Zhao, Jisheng Yin, Yana Wei, Yuang Peng, Haoran Wei, Jianjian Sun, Chunrui Han, Zheng Ge, and 1 others. 2025. Perception-r1: Pioneering perception policy with reinforcement learning. arXiv preprint arXiv:2504.07954. — [arXiv:2504.07954](https://arxiv.org/abs/2504.07954) find
  112. Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. 2020. Gradient surgery for multi-task learning. In Advances in Neural Information Processing Systems, volume 33, pages 5824–5836.
  113. Xin Yue and 1 others. 2023. Mmmu: A massive multi-discipline multimodal understanding benchmark for expert agi. arXiv preprint arXiv:2311.16502.
  114. Xin Yue and 1 others. 2024. Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark. arXiv preprint arXiv:2409.02813. find
  115. Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. 2025. Does reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model? In 2nd AI for Math Workshop @ ICML 2025.
  116. Friedemann Zenke, Ben Poole, and Surya Ganguli. 2017. Continual learning through synaptic intelligence. Proceedings of the National Academy of Sciences, 114(13):3521–3526. Supplementary/alternate venue listings exist. find
  117. Hongzhi Zhang, Jia Fu, Jingyuan Zhang, Kai Fu, Qi Wang, Fuzheng Zhang, and Guorui Zhou. 2025. Rlep: Reinforcement learning with experience replay for llm reasoning. arXiv:2507.07451. Experience replay improves stability and final accuracy on AIME/AMC. — [arXiv:2507.07451](https://arxiv.org/abs/2507.07451) find
  118. Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, and Ziwei Liu. 2024a. Lmms-eval: Reality check on the evaluation of large multimodal models. Preprint, arXiv:2407.12772.
  119. Rui Zhang and 1 others. 2024b. Mathverse: Does your multi-modal llm truly see the diagrams? arXiv preprint arXiv:2403.14624.
  120. Ruxin Zheng, Yifei Li, Xiang Li, Chong Chen, and 1 others. 2023. Secrets of rlhf in large language models part i: Ppo. arXiv:2307.04964. Analyzes PPO variants and shows PPO-ptx mitigates knowledge decline.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/phan-2026-beyond-reasoning-gains-mitigating-general/