Elamur Research⌕
☰
home / papers / 2026 / EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis

EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis

Paper · 2026 · Xiaoshuai Song, Haofei Chang, Guanting Dong et al.

Summary

Training LLM agents requires many stateful sandboxes in which tools can be called. Real systems are often hard to access, LLM-simulated environments hallucinate and contradict themselves, and hand-built sandboxes do not scale. EnvScaler addresses this by having LLMs write environments as executable programs instead of simulating them.

The first component, SkelBuilder, works in three steps:

  • it infers environment themes from open-source task collections;
  • it plans state entities, rules and tool operations, then implements them as runnable code;
  • it filters the result with two agents: one issues test calls and the other judges the outcomes, so only environments with a high enough pass rate are kept.

The second component, ScenGenerator, gives each environment an initial database and writes hard tasks grounded in that state. It also produces rule-based checks on the final state, and their pass rate serves as the reward.

The authors built 191 environments with about 7K scenarios and used them for SFT and RL of Qwen3 models. This produced clear gains on three tool-use benchmarks. The work matters because it provides a cheap, controllable and reproducible supply of agent training environments without needing existing toolsets.

Key points

  • The resulting resource contains 191 programmatic environments and roughly 7K task scenarios, used to train Qwen3 series models with SFT and RL.
  • Evaluation covers three benchmarks, including BFCL multi-turn. Training on both conversational and non-conversational trajectories (SFT Full) scored best overall. Cases with missing tools did not improve, because no such training samples existed.
  • Applying RL directly without SFT helps, but the gain scales with model size: Qwen3-8B improves noticeably while Qwen3-1.7B barely does. Combining SFT with RL works better.
  • Claude-4.5-Sonnet rated the environments on tool–program alignment, functional correctness and robustness. All 191 environments averaged above 8.5 out of 10 on each dimension.
  • With GPT-4.1/GPT-4.1-mini, building one environment costs about $1 and one scenario about $0.06. Most of the cost comes from the 100-round dual-agent assessment loop.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Renmin University of ChinaBeijing · China114 works · 16,118 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Xiaoshuai Song, Haofei Chang, Guanting Dong, Yutao Zhu, Ji-Rong Wen and Zhicheng Dou. 2026. EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.407

@inproceedings{song2026envscaler,
  title = {{EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis}},
  author = {Xiaoshuai Song, Haofei Chang, Guanting Dong, Yutao Zhu, Ji-Rong Wen and Zhicheng Dou},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.407},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.407}
}

References

34 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shihao Cai, Runnan Fang, Jialong Wu, Baixuan Li, Xinyu Wang, Yong Jiang, Liangcai Su, Liwen Zhang, Wenbiao Yin, Zhen Zhang, Fuli Feng, Pengjun Xie, and Xiaobin Wang. 2025. Autoforge: Automated environment synthesis for agentic reinforcement learning. Preprint, arXiv:2512.22857. — [arXiv:2512.22857](https://arxiv.org/abs/2512.22857) find
  2. Tommaso Castellani, Naimeng Ye, Daksh Mittal, Thomson Yen, and Hongseok Namkoong. 2025. Synth-tools: A framework for scaling synthetic tools for agent development. Preprint, arXiv:2511.09572. — [arXiv:2511.09572](https://arxiv.org/abs/2511.09572) find
  3. Chen Chen, Xinlong Hao, Weiwen Liu, Xu Huang, Xingshan Zeng, Shuai Yu, Dexun Li, Yuefeng Huang, Xiangcheng Liu, Wang Xinzhi, and Wu Liu. 2025. ACEBench: A comprehensive evaluation of LLM tool usage. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 12970– 12998, Suzhou,… find
  4. Guanting Dong, Yifei Chen, Xiaoxi Li, Jiajie Jin, Hongjin Qian, Yutao Zhu, Hangyu Mao, Guorui Zhou, Zhicheng Dou, and Ji-Rong Wen. 2025. Tool-star: Empowering llm-brained multi-tool reasoner via reinforcement learning. CoRR, abs/2505.16410. — [arXiv:2505.16410](https://arxiv.org/abs/2505.16410) find
  5. Runnan Fang, Shihao Cai, Baixuan Li, Jialong Wu, Guangyu Li, Wenbiao Yin, Xinyu Wang, Xiaobin Wang, Liangcai Su, Zhen Zhang, and 1 others. 2025. Towards general agentic intelligence via environment scaling. arXiv preprint arXiv:2509.13311. find
  6. Romain Froger, Pierre Andrews, Matteo Bettini, Amar Budhiraja, Ricardo Silveira Cabral, Virginie Do, Emilien Garreau, Jean-Baptiste Gaya, Hugo Laurençon, Maxime Lecanu, Kunal Malkan, Dheeraj Mekala, Pierre Menard, Gerard Moreno-Torres Bertran, Ulyana Piterbarg, Mikhail Plekhanov, Mathieu Rita, Andre… find
  7. Shangmin Guo, Omar Darwiche Domingues, Raphaël Avalos, Aaron Courville, and Florian Strub. 2025. World modelling improves language model agents. Preprint, arXiv:2506.02918. — [arXiv:2506.02918](https://arxiv.org/abs/2506.02918) find
  8. Zhicheng Guo, Sijie Cheng, Hao Wang, Shihao Liang, Yujia Qin, Peng Li, Zhiyuan Liu, Maosong Sun, and Yang Liu. 2024. StableToolBench: Towards stable large-scale benchmarking on tool learning of large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 11143… find
  9. Jian Hu, Jason Klein Liu, Haotian Xu, and Wei Shen. 2025. Reinforce++: Stabilizing critic-free policy optimization with global advantage normalization. Preprint, arXiv:2501.03262. find
  10. Yuchen Huang, Sijia Li, Zhiyuan Fan, Minghao LIU, Wei Liu, and Yi R. Fung. 2025. Scaling environments for LLM agents: Fundamentals, approaches, and future directions. In Workshop on Scaling Environments for Agents. find
  11. Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. 2023. API-bank: A comprehensive benchmark for tool-augmented LLMs. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3102–3116, Singapore. Ass…
  12. Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yongkang Wu, Ji-Rong Wen, Yutao Zhu, and Zhicheng Dou. 2025a. Webthinker: Empowering large reasoning models with deep research capability. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  13. Yuetai Li, Huseyin A Inan, Xiang Yue, Wei-Ning Chen, Lukas Wutschitz, Janardhan Kulkarni, Radha Poovendran, Robert Sim, and Saravan Rajmohan. 2025b. Simulating environments with reasoning models for agent training. Preprint, arXiv:2511.01824. — [arXiv:2511.01824](https://arxiv.org/abs/2511.01824) find
  14. Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, and 1 others. 2025a. Deepseek-v3. 2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556.
  15. Weiwen Liu, Xu Huang, Xingshan Zeng, xinlong hao, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Zhengying Liu, Yuanqing Yu, Zezhong WANG, Yuxian Wang, Wu Ning, Yutai Hou, Bin Wang, Chuhan Wu, Wang Xinzhi, Yong Liu, Yasheng Wang, and 8 others. 2025b. ToolACE: Winning the points of LLM function calling.… find
  16. Zuxin Liu, Thai Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Shirley Kokane, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng, and 1 others. 2024. Apigen: automated pipeline for generating verifiable and diverse function-calling datasets. In Proceedings of the 38th International Conference on Neural Info… find
  17. Jiarui Lu, Thomas Holleis, Yizhe Zhang, Bernhard Aumayer, Feng Nan, Haoping Bai, Shuang Ma, Shen Ma, Mengyu Li, Guoli Yin, Zirui Wang, and Ruoming Pang. 2025. ToolSandbox: A stateful, conversational, interactive evaluation benchmark for LLM tool use capabilities. In Findings of the Association for C…
  18. Junyu Luo, Weizhi Zhang, Ye Yuan, Yusheng Zhao, Junwei Yang, Yiyang Gu, Bohan Wu, Binqi Chen, Ziyue Qiao, Qingqing Long, and 1 others. 2025. Large language model agent: A survey on methodology, applications and challenges. arXiv preprint arXiv:2503.21460.
  19. Shishir G Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. 2025. The berkeley function calling leaderboard (BFCL): From tool use to agentic evaluation of large language models. In Forty-second International Conference on Machine Learning. find
  20. Wasu Top Piriyakulkij, Yichao Liang, Hao Tang, Adrian Weller, Marta Kryven, and Kevin Ellis. 2025. Poe-world: Compositional world modeling with products of programmatic experts. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  21. Akshara Prabhakar, Zuxin Liu, Ming Zhu, Jianguo Zhang, Tulika Awalgaonkar, Shiyu Wang, Zhiwei Liu, Haolin Chen, Thai Hoang, Juan Carlos Niebles, and 1 others. 2025. Apigen-mt: Agentic pipeline for multi-turn data generation via simulated agent-human interplay. arXiv preprint arXiv:2504.03601.
  22. Cheng Qian, Zuxin Liu, Akshara Prabhakar, Zhiwei Liu, Jianguo Zhang, Haolin Chen, Heng Ji, Weiran Yao, Shelby Heinecke, Silvio Savarese, and Huan Wang. 2025. Userbench: An interactive gym environment for user-centric agents. In Workshop on Scaling Environments for Agents. find
  23. Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji-Rong Wen. 2025. Tool learning with large language models: A survey. Frontiers of Computer Science, 19(8):198343.
  24. Michael Sullivan, Mareike Hartmann, and Alexander Koller. 2025. Procedural environment generation for tool-use agents. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 18555–18573, Suzhou, China. Association for Computational Linguistics. find
  25. Hao Tang, Darren Key, and Kevin Ellis. 2024. Worldcoder, a model-based llm agent: Building world models by writing code and interacting with the environment. Advances in Neural Information Processing Systems, 37:70148–70212. find
  26. Weixun Wang, Shaopan Xiong, Gengru Chen, Wei Gao, Sheng Guo, Yancheng He, Ju Huang, Jiaheng Liu, Zhendong Li, Xiaoyang Li, and 1 others. 2025. Reinforcement learning optimization for large-scale learning: An efficient and user-friendly scaling library. arXiv preprint arXiv:2506.06122. — [arXiv:2506.06122](https://arxiv.org/abs/2506.06122) find
  27. Zengzhuang Xu, Bingguang Hao, Zechuan Wang, Yuntao Wen, Xinyi Xu, Yang Liu, Long Chen, Dong Wang, Maolin Wang, Tong Zhao, Yicheng Chen, Cunyin Peng, Jinjie Gu, Leilei Gan, Xiangyu Zhao, Chenyi Zhuang, and Shi Gu. 2025. Funreason-mt technical report: Advanced data synthesis solution for real-world mu… — [arXiv:2510.24645](https://arxiv.org/abs/2510.24645) find
  28. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  29. Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik R Narasimhan. 2025. {$\\tau$}-bench: A benchmark for \\underline{T}ool-\\underline{A}gent-\\underline{U}ser interaction in real-world domains. In The Thirteenth International Conference on Learning Representations. find
  30. Junjie Ye, Changhao Jiang, Zhengyin Du, Yufei Xu, Xuesong Yao, Zhiheng Xi, Xiaoran Fan, Qi Zhang, Tao Gui, Xuanjing Huang, and 1 others. 2025. Feedback-driven tool-use improvements in large language models via automated build environments. arXiv preprint arXiv:2508.08791.
  31. Shaokun Zhang, Yi Dong, Jieyu Zhang, Jan Kautz, Bryan Catanzaro, Andrew Tao, Qingyun Wu, Zhiding Yu, and Guilin Liu. 2026. Nemotron-research-tool-n1: Exploring tool-using language models with reinforced reasoning. In The Fourteenth International Conference on Learning Representations. find
  32. Weikang Zhao, Xili Wang, Chengdi Ma, Lingbin Kong, Zhaohua Yang, Mingxiang Tuo, Xiaowei Shi, Yitao Zhai, and Xunliang Cai. 2025. Mua-rl: Multi-turn user-interacting agent reinforcement learning for agentic tool use. Preprint, arXiv:2508.18669. — [arXiv:2508.18669](https://arxiv.org/abs/2508.18669) find
  33. Yaowei Zheng, Richong Zhang, Junhao Zhang, YeYanhan YeYanhan, and Zheyan Luo. 2024. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 400–410.
  34. A counts across environments, while Figure 10(d) shows the distribution of state categories. Figure 10(e) shows synthesized scenario counts per environment after removing unqualified scenario. A.5 Example of Synthesized Environment In Example 1, we provide a program example of synthesized environmen… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/song-2026-envscaler-scaling-tool-interactive-environments/