Elamur Research⌕
☰
home / papers / 2026 / SCALER: Synthetic Scalable Adaptive Learning Environment for Reasoning

SCALER: Synthetic Scalable Adaptive Learning Environment for Reasoning

Paper · 2026 · Caijun Xu, Changyi Xiao, Zhongyuan Peng et al.

Summary

RL post-training for LLM reasoning tends to stall for two reasons. Tasks can drift out of step with the model, becoming either trivially easy or hopelessly hard, so rewards turn sparse. Training can also keep cycling through a small set of problem patterns. SCALER tackles both issues. Its first part is a synthesis pipeline that automatically turns real-world programming problems into verifiable reasoning environments. Each environment has an input template, a randomized instance generator with explicit scale parameters that control difficulty, and a program-based verifier. Testcase generators are validated through breadth and depth checks, which allows effectively unlimited instance generation. Its second part is an adaptive multi-environment RL loop. A per-environment controller nudges difficulty toward a target success rate based on rollout accuracy. A curation mechanism retires environments whose learning signal has saturated and replaces them with fresh ones from the pool. On Qwen3 base models, SCALER beats dataset-based RL baselines across math and general reasoning benchmarks and trains more stably over long horizons. This suggests that synthesized, diverse environments can sustain RL beyond fixed datasets.

Key points

  • Environment pool: 2,739 synthesized environments, each formalized as a tuple E = (I, P, R) following RLVE. Performance rises consistently as the pool grows from 8 to 64, 512 and 2,739 environments.
  • Difficulty controller: difficulty follows the update d_{t+1} = clip(d_t + β·(acc_t − τ), 0, D). The real-valued level is approximated by mixing instances at adjacent integer levels.
  • Environment retirement: an environment is retired when its accuracy slope is non-positive, its accuracy stays at zero, or its difficulty sits at the maximum for a set window. Defaults are K_slope = 10 and K_zero = K_sat = 5, and results are fairly insensitive to these values.
  • Evaluation setup: models are Qwen3-4B-base and Qwen3-1.7B-base, compared against RL on MATH and DeepMath. Benchmarks are MATH-500, AMC23, AIME24, MMLU-Pro and BBEH; Qwen3-1.7B reaches about a 40.2 average with the default settings.
  • Ablations: removing either the difficulty controller or the curation mechanism lowers performance, so both components contribute.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Fudan UniversityShanghai · China202 works · 22,509 citations🎓Singapore Management UniversitySingapore · Singapore35 works · 2,457 citations🎓Shanghai Innovation InstituteShanghai · China34 works · 325 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksComputer Science Applications · 131 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,685 works

Cite

Cite as Caijun Xu, Changyi Xiao, Zhongyuan Peng, Xinrun Wang and Yixin Cao. 2026. SCALER: Synthetic Scalable Adaptive Learning Environment for Reasoning. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1596

@inproceedings{xu2026scaler,
  title = {{SCALER: Synthetic Scalable Adaptive Learning Environment for Reasoning}},
  author = {Caijun Xu, Changyi Xiao, Zhongyuan Peng, Xinrun Wang and Yixin Cao},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1596},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1596}
}

References

46 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Art of Problem Solving. 2023. Amc 12 (2023) problems. https://artofproblemsolving.com/ wiki/index.php/2023_AMC_12A_Problems. We use the 2023 AMC 12A/12B problems as hosted by AoPS; a processed version is available at https://huggingface.co/datasets/zwhe99/amc23. ↗
  2. Bytedance-Seed-Foundation-Code-Team, :, Yao Cheng, Jianfeng Chen, Jie Chen, Li Chen, Liyu Chen, Wentao Chen, Zhengyu Chen, Shijie Geng, Aoyan Li, Bo Li, Bowen Li, Linyi Li, Boyi Liu, Jiaheng Liu, Kaibo Liu, Qi Liu, Shukai Liu, and 37 others. 2025. Fullstack bench: Evaluating llms as full stack coder… — [arXiv:2412.00535](https://arxiv.org/abs/2412.00535) find
  3. Justin Yang Chae, Md Tanvirul Alam, and Nidhi Rastogi. 2025. Towards understanding self-play for llm reasoning. Preprint, arXiv:2510.27072. — [arXiv:2510.27072](https://arxiv.org/abs/2510.27072) find
  4. Jiaqi Chen, Bang Zhang, Ruotian Ma, Peisong Wang, Xiaodan Liang, Zhaopeng Tu, Xiaolong Li, and Kwan-Yee K. Wong. 2025a. Spc: Evolving self-play critic via adversarial games for llm reasoning. Preprint, arXiv:2504.19162. — [arXiv:2504.19162](https://arxiv.org/abs/2504.19162) find
  5. Xiaoyin Chen, Jiarui Lu, Minsu Kim, Dinghuai Zhang, Jian Tang, Alexandre Piché, Nicolas Gontier, Yoshua Bengio, and Ehsan Kamalloo. 2025b. Self-evolving curriculum for llm reasoning. arXiv preprint arXiv:2505.14970. find
  6. Xinjie Chen, Minpeng Liao, Guoxin Chen, Chengxi Li, Biao Fu, Kai Fan, and Xinggao Liu. 2025c. From data-centric to sample-centric: Enhancing llm reasoning via progressive optimization. Preprint, arXiv:2507.06573. find
  7. Zhuoyue Chen, Jihai Zhang, Ben Liu, Fangquan Lin, and Wotao Yin. 2025d. Scale down to speed up: Dynamic data selection for reinforcement learning. Training, 2500:3000. find
  8. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. Deepseek-r1: Incentivizing reasoning capability in ll…
  9. Wenkai Fang, Shunyu Liu, Yang Zhou, Kongcheng Zhang, Tongya Zheng, Kaixuan Chen, Mingli Song, and Dacheng Tao. 2025. Serl: Self-play reinforcement learning for large language models with limited data. Preprint, arXiv:2505.20347. — [arXiv:2505.20347](https://arxiv.org/abs/2505.20347) find
  10. Jia Fu, Xinyu Yang, Hongzhi Zhang, Yahui Liu, Jingyuan Zhang, Qi Wang, Fuzheng Zhang, and Guorui Zhou. 2025. Klear-codetest: Scalable test case generation for code reinforcement learning. Preprint, arXiv:2508.05710. — [arXiv:2508.05710](https://arxiv.org/abs/2508.05710) find
  11. Anna Goldie, Azalia Mirhoseini, Hao Zhou, Irene Cai, and Christopher D. Manning. 2025. Synthetic data generation & multi-step rl for reasoning & tool use. Preprint, arXiv:2504.04736. find
  12. Jiacheng Guo, Ling Yang, Peter Chen, Qixin Xiao, Yinjie Wang, Xinzhe Juan, Jiahao Qiu, Ke Shen, and Mengdi Wang. 2025a. Genenv: Difficulty-aligned co-evolution between llm agents and environment simulators. Preprint, arXiv:2512.19682. — [arXiv:2512.19682](https://arxiv.org/abs/2512.19682) find
  13. Yiduo Guo, Zhen Guo, Chuanwei Huang, Zi-Ang Wang, Zekai Zhang, Haofei Yu, Huishuai Zhang, and Yikang Shen. 2025b. Synthetic data rl: Task definition is all you need. Preprint, arXiv:2505.17063. — [arXiv:2505.17063](https://arxiv.org/abs/2505.17063) find
  14. Zhiwei He, Tian Liang, Jiahao Xu, Qiuzhi Liu, Xingyu Chen, Yue Wang, Linfeng Song, Dian Yu, Zhenwen Liang, Wenxuan Wang, and 1 others. 2025. Deepmath-103k: A large-scale, challenging, decontaminated, and verifiable mathematical dataset for advancing reasoning. arXiv preprint arXiv:2504.11456.
  15. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. Preprint, arXiv:2103.03874.
  16. Jian Hu. 2025. Reinforce++: A simple and efficient approach for aligning large language models. arXiv preprint arXiv:2501.03262.
  17. Mehran Kazemi, Bahare Fatemi, Hritik Bansal, John Palowitch, Chrysovalantis Anastasiou, Sanket Vaibhav Mehta, Lalit K Jain, Virginia Aglietti, Disha Jindal, Peter Chen, Nishanth Dikkala, Gladys Tyen, Xin Liu, Uri Shalit, Silvia Chiappa, Kate Olszewska, Yi Tay, Vinh Q. Tran, Quoc V. Le, and Orhan Fir… — BIG-Bench Extra Hard
  18. Shipeng Li, Shikun Li, Zhiqin Yang, Xinghua Zhang, Gaode Chen, Xiaobo Xia, Hengyu Liu, and Zhe Peng. 2025. Learnalign: Reasoning data selection for reinforcement learning in large language models based on improved gradient alignment. Preprint, arXiv:2506.11480. — [arXiv:2506.11480](https://arxiv.org/abs/2506.11480) find
  19. Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Axel Gimeno Gil, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d’Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov… find
  20. Xiao Liang, Zhong-Zhi Li, Yeyun Gong, Yang Wang, Hengyuan Zhang, Yelong Shen, Ying Nian Wu, and Weizhu Chen. 2025. Sws: Self-aware weakness-driven problem synthesis in reinforcement learning for llm reasoning. Preprint, arXiv:2506.08989. — [arXiv:2506.08989](https://arxiv.org/abs/2506.08989) find
  21. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. arXiv preprint arXiv:2305.20050. Releases PRM800K and the MATH-500 (500-problem) test subset via https://github.com/openai/… ↗
  22. Bo Liu, Leon Guertler, Simon Yu, Zichen Liu, Penghui Qi, Daniel Balcells, Mickel Liu, Cheston Tan, Weiyan Shi, Min Lin, and 1 others. 2025a. Spiral: Self-play on zero-sum games incentivizes reasoning via multi-agent multi-turn reinforcement learning. arXiv preprint arXiv:2506.24119. — [arXiv:2506.24119](https://arxiv.org/abs/2506.24119) find
  23. Huanyu Liu, Jia Li, Hao Zhu, Kechi Zhang, Yihong Dong, and Ge Li. 2025b. Saturn: Sat-based reinforcement learning to unleash language model reasoning. arXiv preprint arXiv:2505.16368. — [arXiv:2505.16368](https://arxiv.org/abs/2505.16368) find
  24. OpenAI, :, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, and 244 others. 2024. Openai o1 sy…
  25. Shubham Parashar, Shurui Gui, Xiner Li, Hongyi Ling, Sushil Vemuri, Blake Olson, Eric Li, Yu Zhang, James Caverlee, Dileep Kalathil, and Shuiwang Ji. 2025a. Curriculum reinforcement learning from easy to hard tasks improves llm reasoning. Preprint, arXiv:2506.06632. find
  26. Shubham Parashar, Shurui Gui, Xiner Li, Hongyi Ling, Sushil Vemuri, Blake Olson, Eric Li, Yu Zhang, James Caverlee, Dileep Kalathil, and 1 others. 2025b. Curriculum reinforcement learning from easy to hard tasks improves llm reasoning. arXiv preprint arXiv:2506.06632. find
  27. Noam Razin, Zixuan Wang, Hubert Strauss, Stanley Wei, Jason D Lee, and Sanjeev Arora. 2025. What makes a reward model a good teacher? an optimization perspective. arXiv preprint arXiv:2503.15477. — [arXiv:2503.15477](https://arxiv.org/abs/2503.15477) find
  28. Noam Razin, Hattie Zhou, Omid Saremi, Vimal Thilak, Arwen Bradley, Preetum Nakkiran, Joshua Susskind, and Etai Littwin. 2023. Vanishing gradients in reinforcement finetuning of language models. arXiv preprint arXiv:2310.20703. — [arXiv:2310.20703](https://arxiv.org/abs/2310.20703) find
  29. Amrith Setlur, Saurabh Garg, Xinyang Geng, Naman Garg, Virginia Smith, and Aviral Kumar. 2024. Rl on incorrect synthetic data scales the efficiency of llm math reasoning by eight-fold. Preprint, arXiv:2406.14532. find
  30. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  31. Taiwei Shi, Yiyang Wu, Linxin Song, Tianyi Zhou, and Jieyu Zhao. 2025. Efficient reinforcement finetuning via adaptive curriculum learning. arXiv preprint arXiv:2504.05520. — [arXiv:2504.05520](https://arxiv.org/abs/2504.05520) find
  32. Zafir Stojanovski, Oliver Stanley, Joe Sharratt, Richard Jones, Abdulhakeem Adefioye, Jean Kaddour, and Andreas Köpf. 2025. Reasoning gym: Reasoning environments for reinforcement learning with verifiable rewards. arXiv preprint arXiv:2505.24760. — [arXiv:2505.24760](https://arxiv.org/abs/2505.24760) find
  33. Kimi Team. 2025a. Kimi k1.5: Scaling reinforcement learning with llms.
  34. Qwen Team. 2025b. Qwen3 technical report. Preprint, arXiv:2505.09388.
  35. Jingqi Tong, Jixin Tang, Hangcheng Li, Yurong Mou, Ming Zhang, Jun Zhao, Yanbo Wen, Fan Song, Jiahao Zhan, Yuyang Lu, and 1 others. 2025. Game-rl: Synthesizing multimodal verifiable game data to boost vlms’ general reasoning. arXiv preprint arXiv:2505.13886. — [arXiv:2505.13886](https://arxiv.org/abs/2505.13886) find
  36. Hong Wang, Zhezheng Hao, Jian Luo, Chenxing Wei, Yao Shu, Lei Liu, Qiang Lin, Hande Dong, and Jiawei Chen. 2025a. Scheduling your llm reinforcement learning with reasoning trees. Preprint, arXiv:2510.24832. — [arXiv:2510.24832](https://arxiv.org/abs/2510.24832) find
  37. Pinzheng Wang, Juntao Li, Zecheng Tang, Haijia Gui, and Min zhang. 2025b. Improving rationality in the reasoning process of language models through self-playing game. Preprint, arXiv:2506.22920. — [arXiv:2506.22920](https://arxiv.org/abs/2506.22920) find
  38. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. 2024. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark…
  39. Zijian Wu, Jinjie Ni, Xiangyan Liu, Zichen Liu, Hang Yan, and Michael Qizhe Shieh. 2025. Synthrl: Scaling visual reasoning with verifiable data synthesis. Preprint, arXiv:2506.02096.
  40. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, and 16 others. 2025. Dapo: An open-source llm reinforcement learning s…
  41. Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, and 1 others. 2025a. Glm-4.5: Agentic, reasoning, and coding (arc) foundation models. arXiv preprint arXiv:2508.06471.
  42. Zhiyuan Zeng, Hamish Ivison, Yiping Wang, Lifan Yuan, Shuyue Stella Li, Zhuorui Ye, Siting Li, Jacqueline He, Runlong Zhou, Tong Chen, and 1 others. 2025b. Rlve: Scaling up reinforcement learning for language models with adaptive verifiable environments. arXiv preprint arXiv:2511.07317. — [arXiv:2511.07317](https://arxiv.org/abs/2511.07317) find
  43. Bonan Zhang, Zhongqi Chen, Bowen Song, Qinya Li, Fan Wu, and Guihai Chen. 2025. Con-fclip: Confidence-weighted and clipped reward for reinforcement learning in llms. Preprint, arXiv:2509.17730. — [arXiv:2509.17730](https://arxiv.org/abs/2509.17730) find
  44. Yifan Zhang and Team Math-AI. 2024. find
  45. Aime 2024. https://huggingface.co/datasets/math-ai/ aime24. ↗
  46. Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Matthieu Lin, Shenzhi Wang, Qingyun Wu, Zilong Zheng, and Gao Huang. 2025. Absolute zero: Reinforced self-play reasoning with zero data. arXiv preprint arXiv:2505.03335.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xu-2026-scaler-synthetic-scalable-adaptive-learning/