Elamur Research⌕
☰
home / papers / 2026 / A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions

A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions

Paper · 2026 · Zhiyin Yu, Yuchen Mou, Juncheng Yan et al.

Summary

RL-based post-training is now a leading way to strengthen reasoning in LLMs, but it is held back by data scarcity in two forms. External scarcity is the shortage of costly supervision such as human feedback, preference labels, expert annotations and step-level reasoning traces. Internal scarcity comes from limits on what the model can generate itself, including rollout counts, trajectory lengths and exploration budgets. The authors argue that adding more data or compute gives diminishing returns, and they present what they call the first systematic review of this area. They organize the literature into a bottom-up, three-level taxonomy:

  • data-centric methods that prune, synthesize or compress training data;
  • training-centric methods that make rollouts, reward signals and policy updates more efficient;
  • framework-centric methods in which a single model, an asymmetric pair of agents, or a group of agents generates its own tasks and feedback.

For each category the survey describes representative works and weighs their strengths and weaknesses. It concludes that handling scarcity requires all three levels together and gives researchers a map of the design space and its open problems.

Key points

  • Data-centric level: covers offline, online and fine-grained pruning (e.g., LIMR, LearnAlign, EAS, LSPO, MoPPS, BOTS). It also covers static, dynamic and hard data synthesis, and compression at the token, step, trajectory or dataset level (e.g., One-shot RLVR).
  • Training-centric level: has three parts.
    • Trajectory generation: guided exploration and selective rollouts.
    • Reward engineering: consistency- or entropy-based and process-based internal rewards (e.g., TTRL, RENT, Intuitor, CoVo).
    • Policy optimization: sample-efficient objectives and replay mechanisms.
  • Framework-centric level:
    • Self-evolving systems (e.g., SeRL, AZR).
    • Asymmetric co-evolution, either collaborative or adversarial (e.g., SPAG, SPC, GAR, SSR).
    • Multi-agent evolution through competitive games or cooperating roles (e.g., SPIRAL, Vision-Zero with "Who is the Spy", SPELL, MAE).
  • Open challenges:
    • Internal rewards can be noisy and are prone to reward hacking or model collapse.
    • Most methods are limited to verifiable domains such as math and code, so unverifiable and open-ended tasks remain largely unaddressed.
    • Self-play carries safety risks, such as amplifying biases present in the seed data.
  • Publication details: ACL 2026 (Long Papers), pp. 22823–22846. The curated paper list is on GitHub at YuZhiyin/Data-Efficient-RL.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China315 works · 43,267 citations🎓National University of SingaporeSingapore · Singapore207 works · 39,576 citations🎓Shanghai Jiao Tong UniversityShanghai · China212 works · 19,544 citations🔬Shanghai Artificial Intelligence LaboratoryShanghai · China152 works · 13,166 citations🎓University of Wisconsin–MadisonMadison · United States54 works · 49,089 citations🎓Nanjing UniversityNanjing · China99 works · 8,512 citations🎓Tongji UniversityShanghai · China33 works · 2,993 citations🎓Nankai UniversityTianjin · China36 works · 762 citations🎓University of WollongongWollongong · Australia3 works · 1 citation

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Zhiyin Yu, Yuchen Mou, Juncheng Yan, Junyu Luo, Chunchun Chen, Xing Wei, Yunhui Liu, Hongru Sun, Yuxing Zhang, Jun Xu, Yatao Bian, Ming Zhang, Wei Ye, Tieke He, Jie Yang, Guanjie Zheng, Zhonghai Wu, Bo Zhang, Lei Bai and Xiao Luo. 2026. A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1045

@inproceedings{yu2026survey,
  title = {{A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions}},
  author = {Zhiyin Yu, Yuchen Mou, Juncheng Yan, Junyu Luo, Chunchun Chen, Xing Wei, Yunhui Liu, Hongru Sun, Yuxing Zhang, Jun Xu, Yatao Bian, Ming Zhang, Wei Ye, Tieke He, Jie Yang, Guanjie Zheng, Zhonghai Wu, Bo Zhang, Lei Bai and Xiao Luo},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1045},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1045}
}

References

154 references, 38 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, and Hao Peng. 2025. The unreasonable effectiveness of entropy minimization in llm reasoning. arXiv preprint arXiv:2505.15134. find
  2. Huan ang Gao, Jiayi Geng, Wenyue Hua, Mengkang Hu, Xinzhe Juan, Hongzhang Liu, Shilong Liu, Jiahao Qiu, Xuan Qi, Yiran Wu, Hongru Wang, Han Xiao, Yuhang Zhou, Shaokun Zhang, Jiayi Zhang, Jinyu Xiang, Yixiong Fang, Qiwen Zhao, Dongrui Liu, and 8 others. 2026. A survey of self-evolving agents: What, w… find
  3. Samee Arif, Sualeha Farid, Abdul Hameed Azeemi, Awais Athar, and Agha Ali Raza. 2025. The fellowship of the LLMs: Multi-model workflows for synthetic preference optimization dataset generation. In Proceedings of the Fourth Workshop on Generation, Evaluation and Metrics (GEM²), pages 30–45, Vienna, A… find
  4. Dilip Arumugam and Thomas L Griffiths. 2025. Toward efficient exploration by large language model agents. In The Exploration in AI Today Workshop at ICML 2025. find
  5. Sanghwan Bae, Jiwoo Hong, Min Young Lee, Hanbyul Kim, JeongYeon Nam, and Donghyun Kwak. 2025. Online difficulty filtering for reasoning oriented reinforcement learning. CoRR, abs/2504.03380.
  6. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, and 32 others. 20… — Constitutional AI: Harmlessness from AI Feedback
  7. Guoxin Chen, Minpeng Liao, Chengxi Li, and Kai Fan. 2024. Alphamath almost zero: process supervision without process. Advances in Neural Information Processing Systems, 37:27689–27724.
  8. Jiangjie Chen, Qianyu He, Siyu Yuan, Aili Chen, Zhicheng Cai, Weinan Dai, Hongli Yu, Qiying Yu, Xuefeng Li, Jiaze Chen, Hao Zhou, and Mingxuan Wang. 2025a. Enigmata: Scaling logical reasoning in large language models with synthetic verifiable puzzles. CoRR, abs/2505.19914. — [arXiv:2505.19914](https://arxiv.org/abs/2505.19914) find
  9. Jiaqi Chen, Bang Zhang, Ruotian Ma, Peisong Wang, Xiaodan Liang, Zhaopeng Tu, Xiaolong Li, and Kwan-Yee K. Wong. 2025b. SPC: Evolving self-play critic via adversarial games for LLM reasoning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  10. Lili Chen, Mihir Prabhudesai, Katerina Fragkiadaki, Hao Liu, and Deepak Pathak. 2025c. Self-questioning language models. Preprint, arXiv:2508.03682. — [arXiv:2508.03682](https://arxiv.org/abs/2508.03682) find
  11. Weizhe Chen, Sven Koenig, and Bistra Dilkina. 2025d. LSPO: length-aware dynamic sampling for policy optimization in LLM reasoning. CoRR, abs/2510.01459. find
  12. Xingwu Chen, Tianle Li, and Difan Zou. 2025e. Reshaping reasoning in llms: A theoretical analysis of rl training dynamics through pattern selection. Preprint, arXiv:2506.04695. find
  13. Yixing Chen, Yiding Wang, Siqi Zhu, Haofei Yu, Tao Feng, Muhan Zhang, Mostofa Patwary, and Jiaxuan You. 2025f. Multi-agent evolve: Llm self-improve through co-evolution. Preprint, arXiv:2510.23595. — [arXiv:2510.23595](https://arxiv.org/abs/2510.23595) find
  14. Pengyu Cheng, Tianhao Hu, Han Xu, Zhisong Zhang, Yong Dai, Lei Han, nan du, and Xiaolong Li. 2024. Self-playing adversarial language game enhances LLM reasoning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. find
  15. Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guotong Xie, Ruobing Xie, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2024. ULTRAFEEDBACK: boosting language models with scaled AI feedback. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, A… find
  16. Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Yuchen Zhang, Jiacheng Chen, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, Jiarui Yuan, Huayu Chen, Kaiyan Zhang, Xingtai Lv, Shuo Wang, Yuan Yao, Xu Han, and 6 others. 2025. Process reinforcement through implicit rewards. Prepri…
  17. Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. 2023. Safe rlhf: Safe reinforcement learning from human feedback. Preprint, arXiv:2310.12773.
  18. Muzhi Dai, Chenxu Yang, and Qingyi Si. 2025a. S-grpo: Early exit via reinforcement learning in reasoning models. arXiv preprint arXiv:2505.07686. — [arXiv:2505.07686](https://arxiv.org/abs/2505.07686) find
  19. Runpeng Dai, Linfeng Song, Haolin Liu, Zhenwen Liang, Dian Yu, Haitao Mi, Zhaopeng Tu, Rui Liu, Tong Zheng, Hongtu Zhu, and 1 others. 2025b. Cde: Curiosity-driven exploration for efficient reinforcement learning in large language models. arXiv preprint arXiv:2509.09675. — [arXiv:2509.09675](https://arxiv.org/abs/2509.09675) find
  20. Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023. Enhancing chat language models by scaling high-quality instructional conversations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapo…
  21. He Du, Bowen Li, Aijun Yang, Siyang He, Qipeng Guo, and Dacheng Tao. 2025. Evosyn: Generalizable evolutionary data synthesis for verifiable learning. CoRR, abs/2510.17928. — [arXiv:2510.17928](https://arxiv.org/abs/2510.17928) find
  22. Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. 2024. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306. find
  23. Jinyuan Fang, Yanwen Peng, Xi Zhang, Yingxu Wang, Xinhao Yi, Guibin Zhang, Yi Xu, Bin Wu, Siwei Liu, Zihao Li, Zhaochun Ren, Nikos Aletras, Xi Wang, Han Zhou, and Zaiqiao Meng. 2025a. A comprehensive survey of self-evolving ai agents: A new paradigm bridging foundation models and lifelong agentic sy… — [arXiv:2508.07407](https://arxiv.org/abs/2508.07407) find
  24. Wenkai Fang, Shunyu Liu, Yang Zhou, Kongcheng Zhang, Tongya Zheng, Kaixuan Chen, Mingli Song, and Dacheng Tao. 2025b. SeRL: Self-play reinforcement learning for large language models with limited data. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  25. Daniel Fein and Gabriela Aranguiz-Dias. 2025. Influence functions for preference dataset pruning. CoRR, abs/2507.14344. — [arXiv:2507.14344](https://arxiv.org/abs/2507.14344) find
  26. Shuang Feng and Grace Feng. 2024. An extremely data-efficient and generative llm-based reinforcement learning agent for recommenders. Preprint, arXiv:2408.16032. find
  27. Xinyu Guan, Li Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. 2025. rstar-math: Small LLMs can master math reasoning with self-evolved deep thinking. In Forty-second International Conference on Machine Learning.
  28. Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, Wolfgang Macherey, Arnaud Doucet, Orhan Firat, and Nando de Freitas. 2023. Reinforced self-training (rest) for language modeling. Preprint, a…
  29. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, and 175 others. 2025. Deepseek-r1 incentivizes reasoning in llms through rein… find
  30. Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y. K. Li, Fuli Luo, Yingfei Xiong, and Wenfeng Liang. 2024. Deepseek-coder: When the large language model meets programming – the rise of code intelligence. Preprint, arXiv:2401.14196. find
  31. Li He, He Zhao, Stephen Wan, Dadong Wang, Lina Yao, and Tongliang Liu. 2025a. Direct advantage regression: Aligning llms with online AI reward. CoRR, abs/2504.14177. — [arXiv:2504.14177](https://arxiv.org/abs/2504.14177) find
  32. Qiang He and Setareh Maghsudi. 2026. Pareto multi-objective alignment forălanguage models. In Machine Learning and Knowledge Discovery in Databases. Research Track, pages 257–272, Cham. Springer Nature Switzerland. find
  33. Yicheng He, Chengsong Huang, Zongxia Li, Jiaxin Huang, and Yonghui Yang. 2025b. Visplay: Self-evolving vision-language models from images. Preprint, arXiv:2511.15661. — [arXiv:2511.15661](https://arxiv.org/abs/2511.15661) find
  34. Bairu Hou, Yang Zhang, Jiabao Ji, Yujian Liu, Kaizhi Qian, Jacob Andreas, and Shiyu Chang. 2025. Thinkprune: Pruning long chain-of-thought of llms via reinforcement learning. arXiv preprint arXiv:2504.01296.
  35. Sam Houliston, Alizée Pace, Alexander Immer, and Gunnar Rätsch. 2024. Uncertainty-penalized direct preference optimization. Preprint, arXiv:2410.20187. — [arXiv:2410.20187](https://arxiv.org/abs/2410.20187) find
  36. Tue Le, Nghi D. Q. Bui, Linh Ngo Van, and Trung Le. 2025. Sharpness-controlled group relative policy optimization with token-level probability shaping. Preprint, arXiv:2511.00066. — [arXiv:2511.00066](https://arxiv.org/abs/2511.00066) find
  37. Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. 2025. Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model. Preprint, arXiv:2503.24290.
  38. Bowen Li, Wenhan Wu, Ziwei Tang, Lin Shi, John Yang, Jinyang Li, Shunyu Yao, Chen Qian, Binyuan Hui, Qicheng Zhang, Zhiyin Yu, He Du, Ping Yang, Dahua Lin, Chao Peng, and Kai Chen. 2024. Prompting large language models to tackle the full software development lifecycle: A case study. Preprint, arXiv:…
  39. Chengsong Huang, Wenhao Yu, Xiaoyang Wang, Hongming Zhang, Zongxia Li, Ruosen Li, Jiaxin Huang, Haitao Mi, and Dong Yu. 2025. R-zero: Self-evolving reasoning llm from zero data. Preprint, arXiv:2508.05004. — [arXiv:2508.05004](https://arxiv.org/abs/2508.05004) find
  40. Dulhan Jayalath, Shashwat Goel, Thomas Foster, Parag Jain, Suchin Gururangan, Cheng Zhang, Anirudh Goyal, and Alan Schelten. 2025. Compute as teacher: Turning inference compute into reference-free supervision. arXiv preprint arXiv:2509.14234. — [arXiv:2509.14234](https://arxiv.org/abs/2509.14234) find
  41. Kaixuan Ji, Jiafan He, and Quanquan Gu. 2024. Reinforcement learning from human feedback with active queries. find
  42. Trans. Mach. Learn. Res., 2025. find
  43. Chunyang Jiang, Yonggang Zhang, Yiyang Cai, Chi-Min Chan, Yulong Liu, Mingming Chen, Wei Xue, and Yike Guo. 2025. Semantic voting: A self-evaluation-free approach for efficient llm self-improvement on unverifiable open-ended tasks. In The Thirty-ninth Annual Conference on Neural Information Processi… find
  44. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516.
  45. N Jones. 2024. The ai revolution is running out of data. What can researchers do. find
  46. Sahil Kale and Devendra Singh Dhami. 2025. Knowrl: Teaching language models to know what they know. arXiv preprint arXiv:2510.11407. — [arXiv:2510.11407](https://arxiv.org/abs/2510.11407) find
  47. Bryce Kan, Wei Yang, Emily Nguyen, Ganghui Yi, Bowen Yi, Chenxiao Yu, and Yan Liu. 2026. De-conflating preference and qualification: Constrained dual-perspective reasoning for job recommendation with large language models. Preprint, arXiv:2602.03097. find
  48. Pengyi Li, Matvey Skripkin, Alexander Zubrey, Andrey Kuznetsov, and Ivan Oseledets. 2025a. Confidence is all you need: Few-shot rl fine-tuning of language models. arXiv preprint arXiv:2506.06395. — [arXiv:2506.06395](https://arxiv.org/abs/2506.06395) find
  49. Shikun Li, Shipeng Li, Zhiqin Yang, Xinghua Zhang, Gaode Chen, Xiaobo Xia, Hengyu Liu, and Zhe Peng. 2025b. Learnalign: Reasoning data selection for reinforcement learning in large language models based on improved gradient alignment. CoRR, abs/2506.11480. — [arXiv:2506.11480](https://arxiv.org/abs/2506.11480) find
  50. Xuefeng Li, Haoyang Zou, and Pengfei Liu. 2025c. LIMR: less is more for RL scaling. CoRR, abs/2502.11886. find
  51. Yiyuan Li, Zhen Huang, Yanan Wu, Weixun Wang, Xuefeng Li, Yijia Luo, Pengfei Liu, Wenbo Su, and Bo Zheng. 2025d. One sample to rule them all: Extreme data efficiency in RL scaling. In NeurIPS 2025 Workshop on Efficient Reasoning. find
  52. Zeju Li, Jianyuan Zhong, Ziyang Zheng, Xiangyu Wen, Zhijian Xu, Yingying Cheng, Fan Zhang, and Qiang Xu. 2025e. Compressing chain-of-thought in llms via step entropy. Preprint, arXiv:2508.03346. — [arXiv:2508.03346](https://arxiv.org/abs/2508.03346) find
  53. Xiao Liang, Zhong-Zhi Li, Yeyun Gong, Yang Wang, Hengyuan Zhang, Yelong Shen, Ying Nian Wu, and Weizhu Chen. 2025. Sws: Self-aware weakness-driven problem synthesis in reinforcement learning for LLM reasoning. CoRR, abs/2506.08989. — [arXiv:2506.08989](https://arxiv.org/abs/2506.08989) find
  54. Zhihang Lin, Mingbao Lin, Yuan Xie, and Rongrong Ji. 2025. CPPO: accelerating the training of group relative policy optimization-based reasoning models. CoRR, abs/2503.22342. find
  55. Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, and 1 others. 2025. Training language models to self-correct via reinforcement learning. In The Thirteenth International Conference on Learning Representations…
  56. Bo Liu, Leon Guertler, Simon Yu, Zichen Liu, Penghui Qi, Daniel Balcells, Mickel Liu, Cheston Tan, Weiyan Shi, Min Lin, Wee Sun Lee, and Natasha Jaques. 2025a. Spiral: Self-play on zero-sum games incentivizes reasoning via multi-agent multi-turn reinforcement learning. Preprint, arXiv:2506.24119. — [arXiv:2506.24119](https://arxiv.org/abs/2506.24119) find
  57. Zhejian Lai, Xiang Geng, Zhijun Wang, Yang Bai, Jiahuan Li, Rongxiang Weng, Jingang Wang, Xuezhi Cao, Xunliang Cai, and Shujian Huang. 2025. Making mathematical reasoning adaptive. CoRR, abs/2510.04617. — [arXiv:2510.04617](https://arxiv.org/abs/2510.04617) find
  58. Bo Liu, Chuanyang Jin, Seungone Kim, Weizhe Yuan, Wenting Zhao, Ilia Kulikov, Xian Li, Sainbayar Sukhbaatar, Jack Lanchantin, and Jason Weston. 2025b. Spice: Self-play in corpus environments improves reasoning. Preprint, arXiv:2510.24684. — [arXiv:2510.24684](https://arxiv.org/abs/2510.24684) find
  59. Junteng Liu, Yuanxiang Fan, Zhuo Jiang, Han Ding, Yongyi Hu, Chi Zhang, Yiqi Shi, Shitong Weng, Aili Chen, Shiqi Chen, Yunan Huang, Mozhi Zhang, Pengyu Zhao, Junjie Yan, and Junxian He. 2025c. Synlogic: Synthesizing verifiable reasoning data at scale for learning logical reasoning and beyond. CoRR,… — [arXiv:2505.19641](https://arxiv.org/abs/2505.19641) find
  60. Qihao Liu, Luoxin Ye, Wufei Ma, Yu-Cheng Chou, and Alan Yuille. 2025d. Generative adversarial reasoner: Enhancing llm reasoning with adversarial reinforcement learning. Preprint, arXiv:2512.16917. — [arXiv:2512.16917](https://arxiv.org/abs/2512.16917) find
  61. Zichuan Liu, Jinyu Wang, Lei Song, and Jiang Bian. 2025e. Sample-efficient llm optimization with reset replay. arXiv preprint arXiv:2508.06412. — [arXiv:2508.06412](https://arxiv.org/abs/2508.06412) find
  62. Hongliang Lu, Yuhang Wen, Pengyu Cheng, Ruijin Ding, Haotian Xu, Jiaqi Guo, Chutian Wang, Haonan Chen, Xiaoxi Jiang, and Guanjun Jiang. 2025. Search self-play: Pushing the frontier of agent capability without supervision. Preprint, arXiv:2510.18821. — [arXiv:2510.18821](https://arxiv.org/abs/2510.18821) find
  63. Junyu Luo, Xiao Luo, Xiusi Chen, Zhiping Xiao, Wei Ju, and Ming Zhang. 2025a. Semi-supervised fine-tuning for large language models. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 2795–2808. find
  64. Junyu Luo, Bohan Wu, Xiao Luo, Zhiping Xiao, Yiqiao Jin, Rong-Cheng Tu, Nan Yin, Yifan Wang, Jingyang Yuan, Wei Ju, and Ming Zhang. 2025b. A survey on efficient large language model training: From data-centric perspectives. In Proceedings of the 63rd Annual Meeting of the Association for Computation…
  65. Ruotian Ma, Peisong Wang, Cheng Liu, Xingyan Liu, Jiaqi Chen, Bang Zhang, Xin Zhou, Nan Du, and Jia Li. 2025. S 2 r: Teaching llms to self-verify and self-correct via reinforcement learning. Preprint, arXiv:2502.12853.
  66. Luckeciano C. Melo, Panagiotis Tigas, Alessandro Abate, and Yarin Gal. 2024. Deep bayesian active learning for preference modeling in large language models. In Advances in Neural Information Processing Systems, volume 37, pages 118052–118085. Curran Associates, Inc. find
  67. OpenAI. 2024. Openai o1 system card. Preprint, arXiv:2412.16720.
  68. OpenAI OpenAI, Matthias Plappert, Raul Sampedro, Tao Xu, Ilge Akkaya, Vineet Kosaraju, Peter Welinder, Ruben D’Sa, Arthur Petron, Henrique P. d. O. Pinto, Alex Paino, Hyeonwoo Noh, Lilian Weng, Qiming Yuan, Casey Chu, and Wojciech Zaremba. 2021. Asymmetric self-play for automatic goal discovery in r… find
  69. Mihir Prabhudesai, Lili Chen, Alex Ippoliti, Katerina Fragkiadaki, Hao Liu, and Deepak Pathak. 2025. find
  70. Maximizing confidence alone improves reasoning. arXiv preprint arXiv:2505.22660. — [arXiv:2505.22660](https://arxiv.org/abs/2505.22660) find
  71. Xuan Qi, Jiahao Qiu, Xinzhe Juan, Yue Wu, and Mengdi Wang. 2025. Shallow preference signals: Large language model aligns even better with truncated data? arXiv preprint arXiv:2505.17122. find
  72. Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Xinyue Yang, Jiadai Sun, Yu Yang, Shuntian Yao, Tianjie Zhang, and 1 others. 2024. Webrl: Training llm web agents via self-evolving online curriculum reinforcement learning. arXiv preprint arXiv:2411.02337. — [arXiv:2411.02337](https://arxiv.org/abs/2411.02337) find
  73. Yun Qu, Qi Wang, Yixiu Mao, Vincent Tao Hu, Björn Ommer, and Xiangyang Ji. 2025. Can prompt difficulty be online predicted for accelerating RL finetuning of reasoning models? CoRR, abs/2507.04632. — [arXiv:2507.04632](https://arxiv.org/abs/2507.04632) find
  74. Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, H. Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-S… find
  75. Sheikh Shafayat, Fahim Tajwar, Ruslan Salakhutdinov, Jeff Schneider, and Andrea Zanette. 2025. Can large reasoning models self-train? Preprint, arXiv:2505.21444. find
  76. Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang, Sewoong Oh, Simon Shaolei Du, Nathan Lambert, Sewon Min, Ranjay Krishna, and 1 others. 2025. Spurious rewards: Rethinking training signals in rlvr. arXiv preprint arXiv:2506.10947.
  77. Qianli Shen, Daoyuan Chen, Yilun Huang, Zhenqing Ling, Yaliang Li, Bolin Ding, and Jingren Zhou. 2025a. BOTS: A unified framework for bayesian online task selection in LLM reinforcement finetuning. CoRR, abs/2510.26374. — [arXiv:2510.26374](https://arxiv.org/abs/2510.26374) find
  78. Yi Shen, Jian Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang, Kai Wang, Zhaoxiang Liu, and Shiguo Lian. 2025b. Dast: Difficulty-adaptive slow-thinking for large reasoning models. arXiv preprint arXiv:2503.04472. find
  79. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. 2024. Ai models collapse when trained on recursively generated data. Nature, 631(8022):755–759.
  80. David Silver and Richard S. Sutton. 2025. Welcome to the era of experience. Technical report, Google AI. find
  81. Toby Simonds, Kevin Lopez, Akira Yoshiyama, and Dominique Garmier. 2025. Self rewarding self improving. arXiv preprint arXiv:2505.08827. — [arXiv:2505.08827](https://arxiv.org/abs/2505.08827) find
  82. Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. 2020. Learning to summarize from human feedback. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY, US… find
  83. Zafir Stojanovski, Oliver Stanley, Joe Sharratt, Richard Jones, Abdulhakeem Adefioye, Jean Kaddour, and Andreas Köpf. 2025. REASONING GYM: reasoning environments for reinforcement learning with verifiable rewards. CoRR, abs/2505.24760. — [arXiv:2505.24760](https://arxiv.org/abs/2505.24760) find
  84. Sainbayar Sukhbaatar, Zeming Lin, Ilya Kostrikov, Gabriel Synnaeve, Arthur Szlam, and Rob Fergus. 2018. Intrinsic motivation and automatic curricula via asymmetric self-play. In International Conference on Learning Representations. find
  85. Wangtao Sun, Xiang Cheng, Jialin Fan, Yao Xu, Xing Yu, Shizhu He, Jun Zhao, and Kang Liu. 2025a. Towards agentic self-learning llms in search environment. Preprint, arXiv:2510.14253. — [arXiv:2510.14253](https://arxiv.org/abs/2510.14253) find
  86. Yan Sun, Jia Guo, Stanley Kok, Zihao Wang, Zujie Wen, and Zhiqiang Zhang. 2025b. Efficient reinforcement learning for large language models with intrinsic exploration. CoRR, abs/2511.00794. — [arXiv:2511.00794](https://arxiv.org/abs/2511.00794) find
  87. Yifan Sun, Jingyan Shen, Yibin Wang, Tianyu Chen, Zhendong Wang, Mingyuan Zhou, and Huan Zhang. 2025c. Improving data efficiency for LLM reinforcement fine-tuning through difficulty-targeted online data selection and rollout replay. CoRR, abs/2506.05316. — [arXiv:2506.05316](https://arxiv.org/abs/2506.05316) find
  88. Chuyi Tan, Peiwen Yuan, Xinglin Wang, Yiwei Li, Shaoxiong Feng, Yueqi Zhang, Jiayi Shi, Ji Zhang, Boyuan Pan, Yao Hu, and Kan Li. 2025a. Diagnosing and mitigating system bias in self-rewarding rl. Preprint, arXiv:2510.08977. — [arXiv:2510.08977](https://arxiv.org/abs/2510.08977) find
  89. Zezhong Tan, Hang Gao, Xinhong Ma, Feng Zhang, and Ziqiang Dong. 2025b. Towards flash thinking via decoupled advantage policy optimization. arXiv preprint arXiv:2510.15374. — [arXiv:2510.15374](https://arxiv.org/abs/2510.15374) find
  90. Zhengwei Tao, Ting-En Lin, Xiancai Chen, Hangyu Li, Yuchuan Wu, Yongbin Li, Zhi Jin, Fei Huang, Dacheng Tao, and Jingren Zhou. 2024. A survey on self-evolution of large language models. Preprint, arXiv:2404.14387. — [arXiv:2404.14387](https://arxiv.org/abs/2404.14387) find
  91. InternAgent Team, Bo Zhang, Shiyang Feng, Xiangchao Yan, Jiakang Yuan, Runmin Ma, Yusong Hu, Zhiyin Yu, Xiaohan He, Songtao Huang, Shaowei Hou, Zheng Nie, Zhilong Wang, Jinyao Liu, Tianshuo Peng, Peng Ye, Dongzhan Zhou, Shufei Zhang, Xiaosong Wang, and 7 others. 2025a. Internagent: When agent become… find
  92. Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, Chuning Tang, Congcong Wang, Dehao Zhang, Enming Yuan, Enzhe Lu, Fengxiang Tang, Flood Sung, Guangda Wei, Guokun Lai, and 77 others. 2025b. Kimi k1.5: Scaling reinforcement…
  93. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models. CoRR, ab…
  94. Ziyu Wan, Xidong Feng, Muning Wen, Stephen Marcus Mcaleer, Ying Wen, Weinan Zhang, and Jun Wang. 2024. Alphazero-like tree-search can guide large language model decoding and training. In International Conference on Machine Learning, pages 49890–49920. PMLR. find
  95. Huanqian Wang, Yang Yue, Rui Lu, Jingxin Shi, Andrew Zhao, Shenzhi Wang, Shiji Song, and Gao Huang. 2025a. Model surgery: Modulating LLM’s behavior via simple parameter editing. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguist… find
  96. Jingyao Wang, Wenwen Qiang, Zeen Song, Changwen Zheng, and Hui Xiong. 2025b. Learning to think: Information-theoretic reinforcement fine-tuning for llms. arXiv preprint arXiv:2505.10425. — [arXiv:2505.10425](https://arxiv.org/abs/2505.10425) find
  97. Junqiao Wang, Zhaoyang Guan, Guanyu Liu, Tianze Xia, Xianzhi Li, Shuo Yin, Xinyuan Song, Chuhan Cheng, Tianyu Shi, and Alex Lee. 2025c. Meta-learning reinforcement learning for crypto-return prediction. Preprint, arXiv:2509.09751. — [arXiv:2509.09751](https://arxiv.org/abs/2509.09751) find
  98. Qinsi Wang, Jinghan Ke, Hancheng Ye, Yueqian Lin, Yuzhe Fu, Jianyi Zhang, Kurt Keutzer, Chenfeng Xu, and Yiran Chen. 2025d. Angles don’t lie: Unlocking training-efficient RL through the model’s own signals. CoRR, abs/2506.02281. — [arXiv:2506.02281](https://arxiv.org/abs/2506.02281) find
  99. Qinsi Wang, Bo Liu, Tianyi Zhou, Jing Shi, Yueqian Lin, Yiran Chen, Hai Helen Li, Kun Wan, and Wentian Zhao. 2025e. Vision-zero: Scalable vlm self-improvement via strategic gamified self-play. Preprint, arXiv:2509.25541. — [arXiv:2509.25541](https://arxiv.org/abs/2509.25541) find
  100. Shenzhi Wang, Chang Liu, Zilong Zheng, Siyuan Qi, Shuo Chen, Qisen Yang, Andrew Zhao, Chaofei Wang, Shiji Song, and Gao Huang. 2024. Boosting LLM agents with recursive contemplation for effective deception handling. In Findings of the Association for Computational Linguistics: ACL 2024, pages 9909–9… find
  101. Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xiong-Hui Chen, Jianxin Yang, Zhenru Zhang, Yuqiong Liu, An Yang, Andrew Zhao, Yang Yue, Shiji Song, Bowen Yu, Gao Huang, and Junyang Lin. 2025f. Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforce…
  102. Shuai Wang, Yaoming Yang, Bingdong Li, Hao Hao, and Aimin Zhou. 2026. Ib-grpo: Aligning llm-based learning path recommendation with educational objectives via indicator-based group relative policy optimization. Preprint, arXiv:2601.14686. — [arXiv:2601.14686](https://arxiv.org/abs/2601.14686) find
  103. Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Lucas Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, Simon Shaolei Du, and Yelong Shen. 2025g. Reinforcement learning for reasoning in large language models with one training example. arXiv preprint arXi…
  104. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long… find
  105. Lai Wei, Yuting Li, Chen Wang, Yue Wang, Linghe Kong, Weiran Huang, and Lichao Sun. 2025a. First sft, second rl, third upt: Continual improving multimodal llm reasoning via unsupervised post-training. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  106. Yongxian Wei, Yilin Zhao, Li Shen, Xinrui Chen, Runxi Cheng, Sinan Du, Hao Yu, Gang Liu, Jiahong Yan, Chun Yuan, and Dian Li. 2025b. Learning to pose problems: Reasoning-driven and solver-adaptive data synthesis for large reasoning models. Preprint, arXiv:2511.09907. — [arXiv:2511.09907](https://arxiv.org/abs/2511.09907) find
  107. Yuxiang Wei, Zhiqing Sun, Emily McMilin, Jonas Gehring, David Zhang, Gabriel Synnaeve, Daniel Fried, Lingming Zhang, and Sida Wang. 2025c. Toward training superintelligent software agents through self-play swe-rl. Preprint, arXiv:2512.18552. — [arXiv:2512.18552](https://arxiv.org/abs/2512.18552) find
  108. Martin Weyssow, Aton Kamanda, Xin Zhou, and Houari Sahraoui. 2025. Codeultrafeedback: An llm-as-a-judge dataset for aligning large language models to coding preferences. ACM Trans. Softw. Eng. Methodol. Just Accepted. find
  109. Alex Wilf, Pranjal Aggarwal, Bryan Parno, Daniel Fried, Louis-Philippe Morency, Paul Pu Liang, and Sean Welleck. 2025. Propose, solve, verify: Self-play through formal verification. Preprint, arXiv:2512.18160. find
  110. Yue Wu, Zhiqing Sun, Huizhuo Yuan, Kaixuan Ji, Yiming Yang, and Quanquan Gu. Self-play preference optimization for language model alignment. In The Thirteenth International Conference on Learning Representations. find
  111. Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Ammanabrolu, Noah A. Smith, Mari Ostendorf, and Hannaneh Hajishirzi. 2023a. Fine-grained human feedback gives better rewards for language model training. In Thirty-seventh Conference on Neural Information Processing Systems.
  112. Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Ammanabrolu, Noah A Smith, Mari Ostendorf, and Hannaneh Hajishirzi. 2023b. Fine-grained human feedback gives better rewards for language model training. In Advances in Neural Information Processing Systems, volume 36, pages 59008–59…
  113. Zijian Wu, Jinjie Ni, Xiangyan Liu, Zichen Liu, Hang Yan, and Michael Qizhe Shieh. 2025. Synthrl: Scaling visual reasoning with verifiable data synthesis. CoRR, abs/2506.02096.
  114. Yu Xia, Rui Wang, Xu Liu, Mingyan Li, Tong Yu, Xiang Chen, Julian McAuley, and Shuai Li. 2025. Beyond chain-of-thought: A survey of chain-of-X paradigms for LLMs. In Proceedings of the 31st International Conference on Computational Linguistics, pages 10795–10809, Abu Dhabi, UAE. Association for Comp…
  115. Roy Xie, David Qiu, Deepak Gopinath, Dong Lin, Yanchao Sun, Chong Wang, Saloni Potdar, and Bhuwan Dhingra. 2025. Interleaved reasoning for large language models via reinforcement learning. arXiv preprint arXiv:2505.19640. — [arXiv:2505.19640](https://arxiv.org/abs/2505.19640) find
  116. Rihui Xin, Han Liu, Zecheng Wang, Yupeng Zhang, Dianbo Sui, Xiaolin Hu, and Bingning Wang. 2025. Surrogate signals from format and length: Reinforcement learning for solving mathematical problems without ground truth answers. arXiv preprint arXiv:2505.19439. — [arXiv:2505.19439](https://arxiv.org/abs/2505.19439) find
  117. Wei Xiong, Hanning Zhang, Chenlu Ye, Lichang Chen, Nan Jiang, and Tong Zhang. 2025. Self-rewarding correction for mathematical reasoning. Preprint, arXiv:2502.19613. — [arXiv:2502.19613](https://arxiv.org/abs/2502.19613) find
  118. Ran Xu, Yuchen Zhuang, Zihan Dong, Ruiyu Wang, Yue Yu, Joyce C. Ho, Linjun Zhang, Haoyu Wang, Wenqi Shi, and Carl Yang. 2025a. Acesearcher: Bootstrapping reasoning and search for LLMs via reinforced self-play. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  119. Yixuan Even Xu, Yash Savani, Fei Fang, and Zico Kolter. 2025b. Not all rollouts are useful: Downsampling rollouts in LLM reinforcement learning. CoRR, abs/2504.13818. — [arXiv:2504.13818](https://arxiv.org/abs/2504.13818) find
  120. Xiangyuan Xue, Yifan Zhou, Guibin Zhang, Zaibin Zhang, Yijiang Li, Chen Zhang, Zhenfei Yin, Philip Torr, Wanli Ouyang, and Lei Bai. 2025. Comas: Co-evolving multi-agent systems via interaction rewards. arXiv preprint arXiv:2510.08529. — [arXiv:2510.08529](https://arxiv.org/abs/2510.08529) find
  121. Chenyu Yang, Shiqian Su, Shi Liu, Xuan Dong, Yue Yu, Weijie Su, Xuehui Wang, Zhaoyang Liu, Jinguo Zhu, Hao Li, Wenhai Wang, Yu Qiao, Xizhou Zhu, and Jifeng Dai. 2025a. Zerogui: Automating online gui learning at zero human cost. Preprint, arXiv:2505.23762. — [arXiv:2505.23762](https://arxiv.org/abs/2505.23762) find
  122. Wenjie Yang, Mao Zheng, Mingyang Song, Zheng Li, and Sitong Wang. 2025b. Ssr-zero: Simple self-rewarding reinforcement learning for machine translation. Preprint, arXiv:2505.16637.
  123. Zhihe Yang, Xufang Luo, Zilong Wang, Dongqi Han, Zhiyuan He, Dongsheng Li, and Yunjian Xu. 2025c. Do not let low-probability tokens over-dominate in rl for llms. find
  124. Ziyi Yang, Weizhou Shen, Ruijun Chen, Chenliang Li, Fanqi Wan, Ming Yan, Xiaojun Quan, and Fei Huang. 2025d. Spell: Self-play reinforcement learning for evolving long-context language models. Preprint, arXiv:2509.23863. — [arXiv:2509.23863](https://arxiv.org/abs/2509.23863) find
  125. Chenlu Ye, Zhou Yu, Ziji Zhang, Hao Chen, Narayanan Sadagopan, Jing Huang, Tong Zhang, and Anurag Beniwal. 2025a. Beyond correctness: Harmonizing process and outcome rewards through rl training. arXiv preprint arXiv:2509.03403. — [arXiv:2509.03403](https://arxiv.org/abs/2509.03403) find
  126. Ziyu Ye, Rishabh Agarwal, Tianqi Liu, Rishabh Joshi, Sarmishta Velury, Quoc V. Le, Qijun Tan, and Yuan Liu. 2025b. Reward-guided prompt evolving in reinforcement learning for llms. In Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025. OpenR… find
  127. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, and 1 others. 2025. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476.
  128. Zhiyin Yu, Bo Zhang, Qibin Hou, Zhonghai Wu, Xiao Luo, and Lei Bai. 2026. Easy samples are all you need: Self-evolving llms via data-efficient reinforcement learning. In Findings of the Association for Computational Linguistics: ACL 2026.
  129. Wenzhen Yuan, Shengji Tang, Weihao Lin, Jiacheng Ruan, Ganqu Cui, Bo Zhang, Tao Chen, Ting Liu, Yuzhuo Fu, Peng Ye, and 1 others. 2025. Wisdom of the crowd: Reinforcement learning from coevolutionary collective feedback. arXiv preprint arXiv:2508.12338. — [arXiv:2508.12338](https://arxiv.org/abs/2508.12338) find
  130. Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. 2022. Star: Bootstrapping reasoning with reasoning. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022.
  131. Sihang Zeng, Kai Tian, Kaiyan Zhang, Yuru Wang, Junqi Gao, Runze Liu, Sa Yang, Jingxuan Li, Xinwei Long, Jiaheng Ma, and 1 others. 2025a. Reviewrl: Towards automated scientific review with rl. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 16942–1695…
  132. Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun MA, and Junxian He. 2025b. SimpleRL-zoo: Investigating and taming zero reinforcement learning for open base models in the wild. In Second Conference on Language Modeling.
  133. Yuanzhao Zhai, Zhuo Zhang, Kele Xu, Hanyang Peng, Yue Yu, Dawei Feng, Cheng Yang, Bo Ding, and Huaimin Wang. 2024. Online self-preferring language models. CoRR, abs/2405.14103. — [arXiv:2405.14103](https://arxiv.org/abs/2405.14103) find
  134. Guibin Zhang, Hejia Geng, Xiaohang Yu, Zhenfei Yin, Zaibin Zhang, Zelin Tan, Heng Zhou, Zhongzhi Li, Xiangyuan Xue, Yijiang Li, Yifan Zhou, Yang Chen, Chen Zhang, Yutao Fan, Zihu Wang, Songtao Huang, Francisco Piedrahita-Velez, Yue Liao, Hongru Wang, and 6 others. 2025a. The landscape of agentic rei… — [arXiv:2509.02547](https://arxiv.org/abs/2509.02547) find
  135. Hangfan Zhang, Siyuan Xu, Zhimeng Guo, Huaisheng Zhu, Shicheng Liu, Xinrun Wang, Qiaosheng Zhang, Yang Chen, Peng Ye, Lei Bai, and Shuyue Hu. 2025b. The path of self-evolving large language models: Achieving data-efficient learning via intrinsic feedback. Preprint, arXiv:2510.02752. — [arXiv:2510.02752](https://arxiv.org/abs/2510.02752) find
  136. Kaiyan Zhang, Yuxin Zuo, Bingxiang He, Youbang Sun, Runze Liu, Che Jiang, Yuchen Fan, Kai Tian, Guoli Jia, Pengfei Li, Yu Fu, Xingtai Lv, Yuchen Zhang, Sihang Zeng, Shang Qu, Haozhan Li, Shijie Wang, Yuru Wang, Xinwei Long, and 20 others. 2025c. A survey of reinforcement learning for large reasoning… — [arXiv:2509.08827](https://arxiv.org/abs/2509.08827) find
  137. Kangning Zhang, Wenxiang Jiao, Kounianhua Du, Yuan Lu, Weiwen Liu, Weinan Zhang, Lei Zhang, and Yong Yu. 2025d. Looptool: Closing the data-training loop for robust llm tool calls. Preprint, arXiv:2511.09148.
  138. Kongcheng Zhang, Qi Yao, Shunyu Liu, Yingjie Wang, Baisheng Lai, Jieping Ye, Mingli Song, and Dacheng Tao. 2025e. Consistent paths lead to truth: Self-rewarding reinforcement learning for llm reasoning. Preprint, arXiv:2506.08745. — [arXiv:2506.08745](https://arxiv.org/abs/2506.08745) find
  139. Qingyang Zhang, Haitao Wu, Changqing Zhang, Peilin Zhao, and Yatao Bian. 2025f. Right question is already half the answer: Fully unsupervisedd LLM reasoning incentization. In Second Workshop on Test-Time Adaptation: Putting Updates to the Test! at ICML 2025. find
  140. Ruiqi Zhang, Daman Arora, Song Mei, and Andrea Zanette. 2025g. SPEED-RL: faster training of reasoning models via online curriculum learning. CoRR, abs/2506.09016. — [arXiv:2506.09016](https://arxiv.org/abs/2506.09016) find
  141. Yanzhi Zhang, Zhaoxi Zhang, Haoxiang Guan, Yilin Cheng, Yitong Duan, Chen Wang, Yue Wang, Shuxin Zheng, and Jiyan He. 2025h. No free lunch: Rethinking internal feedback for llm reasoning. arXiv preprint arXiv:2506.17219. — [arXiv:2506.17219](https://arxiv.org/abs/2506.17219) find
  142. Zhengxin Zhang, Chengyu Huang, Aochong Oliver Li, and Claire Cardie. 2025i. Better llm reasoning via dual-play. Preprint, arXiv:2511.11881.
  143. Zizhuo Zhang, Jianing Zhu, Xinmu Ge, Zihua Zhao, Zhanke Zhou, Xuan Li, Xiao Feng, Jiangchao Yao, and Bo Han. 2025j. Co-rewarding: Stable self-supervised rl for eliciting reasoning in large language models. arXiv preprint arXiv:2508.00410. — [arXiv:2508.00410](https://arxiv.org/abs/2508.00410) find
  144. Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Yang Yue, Matthieu Lin, Shenzhi Wang, Qingyun Wu, Zilong Zheng, and Gao Huang. 2025a. Absolute zero: Reinforced self-play reasoning with zero data. Preprint, arXiv:2505.03335.
  145. Yujun Zhou, Zhenwen Liang, Haolin Liu, Wenhao Yu, Kishan Panaganti, Linfeng Song, Dian Yu, Xiangliang Zhang, Haitao Mi, and Dong Yu. 2025b. Evolving language models without labels: Majority drives selection, novelty promotes variation. Preprint, arXiv:2509.15194. — [arXiv:2509.15194](https://arxiv.org/abs/2509.15194) find
  146. Yongfu Zhu, Lin Sun, Guangxiang Zhao, Weihong Lin, and Xiangzheng Zhang. 2025. Uncertainty under the curve: A sequence-level entropy area metric for reasoning LLM. CoRR, abs/2508.20384. — [arXiv:2508.20384](https://arxiv.org/abs/2508.20384) find
  147. Yuxin Zuo, Kaiyan Zhang, Li Sheng, Shang Qu, Ganqu Cui, Xuekai Zhu, Haozhan Li, Yuchen Zhang, Xinwei Long, Ermo Hua, Biqing Qi, Youbang Sun, Zhiyuan Ma, Lifan Yuan, Ning Ding, and Bowen Zhou. 2025. TTRL: Test-time reinforcement learning. In The Thirty-ninth Annual Conference on Neural Information Pr…
  148. Siyan Zhao, Mengchen Liu, Jing Huang, Miao Liu, Chenyu Wang, Bo Liu, Yuandong Tian, Guan Pang, Sean Bell, Aditya Grover, and 1 others. 2025b. Inpainting-guided policy optimization for diffusion large language models. arXiv preprint arXiv:2509.10396. — [arXiv:2509.10396](https://arxiv.org/abs/2509.10396) find
  149. Xuandong Zhao, Zhewei Kang, Aosong Feng, Sergey Levine, and Dawn Song. 2025c. Learning to reason without external rewards. arXiv preprint arXiv:2505.19590. find
  150. Yang Zhao, Hepeng Wang, Xiao Ding, Yangou Ouyang, Bibo Cai, Kai Xiong, Jinglong Gao, Zhouhao Sun, Li Du, Bing Qin, and Ting Liu. 2026. Maestro: Meta-learning adaptive estimation of scalarization trade-offs for reward optimization. Preprint, arXiv:2601.07208.
  151. Yang Zhao, Kai Xiong, Xiao Ding, Li Du, Yan-gouOuyang, Zhouhao Sun, Jiannan Guan, Wenbin Zhang, Bin Liu, Dong Hu, Bing Qin, and Ting Liu. 2025d. Ufo-rl: Uncertainty-focused optimization for efficient reinforcement learning data selection. Preprint, arXiv:2505.12457. — [arXiv:2505.12457](https://arxiv.org/abs/2505.12457) find
  152. Haizhong Zheng, Yang Zhou, Brian R. Bartoldson, Bhavya Kailkhura, Fan Lai, Jiawei Zhao, and Beidi Chen. 2025. Act only when it pays: Efficient reinforcement learning for LLM reasoning via selective rollouts. CoRR, abs/2506.02177. — [arXiv:2506.02177](https://arxiv.org/abs/2506.02177) find
  153. Rui Zheng, Shihan Dou, Songyang Gao, Yuan Hua, Wei Shen, Binghai Wang, Yan Liu, Senjie Jin, Qin Liu, Yuhao Zhou, Limao Xiong, Lu Chen, Zhiheng Xi, Nuo Xu, Wenbin Lai, Minghao Zhu, Cheng Chang, Zhangyue Yin, Rongxiang Weng, and 8 others. 2023. Secrets of rlhf in large language models part i: Ppo. Pre…
  154. Qi Zhou, Jie Zhang, Dongxia Wang, Qiang Liu, Tianlin Li, Jin Song Dong, Wenhai Wang, and Qing Guo. 2025a. Fair-pp: A synthetic dataset for aligning LLM with personalized preferences of social equity. CoRR, abs/2505.11861. — [arXiv:2505.11861](https://arxiv.org/abs/2505.11861) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yu-2026-survey-reinforcement-learning-large-language/