Elamur Research⌕
☰
home / papers / 2026 / Failure makes the agent stronger: Enhancing Accuracy through Structured Reflection for Reliable Tool Interactions

Failure makes the agent stronger: Enhancing Accuracy through Structured Reflection for Reliable Tool Interactions

Paper · 2026 · Junhao Su, Yuanliang Wan, Junwei Yang et al.

Summary

Tool-using LLMs are usually trained to get a single call right. When a call fails in a multi-turn exchange, they often repeat the error instead of recovering. Prompt-based self-reflection does little to fix this. The authors make error repair an explicit, trainable action instead. After a failed call, the agent writes a short reflection that diagnoses the fault from the previous step's evidence, then issues a corrected call and a final answer.

Training is reinforcement learning that combines DAPO's decoupled clipping and dynamic sampling with GSPO's sequence-level importance sampling. The reward has several parts: format, tool name, parameters, and semantic quality of the reflection. For training and evaluation they release Tool-Reflection-Bench, built by deliberately breaking correct tool-call trajectories and attaching repair steps.

Qwen2.5-7B, Qwen3-4B and Llama-3.1-8B all improve on multi-turn tool use in BFCL v3 and on error repair in Tool-Reflection-Bench. The fine-tuned models even beat small closed-source models on repair. This suggests that learning from failure is a practical route to more reliable agents.

Key points

  • Tool-Reflection-Bench:
    • Built in the style of BUTTON, plus some XLAM data.
    • Size: about 5,000 training items and about 1,000 test items.
    • The test items are all perturbed failure cases.
    • Four failure operators: call-order swap, redundant call, missing/wrong tool, and corrupted arguments.
  • BFCL v3 multi-turn overall accuracy:
    • Qwen2.5-7B-Instruct-FC rose from 11.00 to 14.88.
    • Qwen3-4B-Instruct rose from 16.25 to 20.75.
    • The biggest gains were on Miss_Param and Long_Context; gains on Miss_Func were small.
  • Repair rates on Tool-Reflection-Bench (Repair@1/3/5):
    • Llama-3.1-8B-Instruct went from 0.7/5.1/6.8 to 4.7/20.5/26.4.
    • Qwen3-4B reached 14.9 at Repair@1.
    • The tuned models outperform GPT-4o-mini, GPT-4.1-mini and LongCat-Lite-8K-Chat.
  • Ablations:
    • The combined DAPO+GSPO objective (14.88) beats DAPO alone (13.75) and GSPO alone (13.25) on Qwen2.5.
    • For repair training, RL beats SFT, which beats prompt-only (Repair@1: 4.7 vs 3.2 vs 1.5).
  • Limitations:
    • Tested only on 4B–8B models.
    • The perturbations cover a bounded set of failure types.
    • Code and data are released on GitHub (MeiGen-AI/Tool-Reflection-Bench).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢MeituanBeijing · China26 works · 3,953 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

Cite

Cite as Junhao Su, Yuanliang Wan, Junwei Yang, Hengyu Shi, Tianyang Han, Yurui Qiu and Junfeng Luo. 2026. Failure makes the agent stronger: Enhancing Accuracy through Structured Reflection for Reliable Tool Interactions. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.618

@inproceedings{su2026failure,
  title = {{Failure makes the agent stronger: Enhancing Accuracy through Structured Reflection for Reliable Tool Interactions}},
  author = {Junhao Su, Yuanliang Wan, Junwei Yang, Hengyu Shi, Tianyang Han, Yurui Qiu and Junfeng Luo},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.618},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.618}
}

References

44 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chen Chen, Xinlong Hao, Weiwen Liu, Xu Huang, Xingshan Zeng, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Yuefeng Huang, and 1 others. 2025a. Acebench: Who wins the match point in tool learning? arXiv e-prints, pages arXiv–2501. find
  2. Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. 2025b. Sft or rl? an early investigation into training r1-like reasoning large vision-language models. arXiv preprint arXiv:2504.11468.
  3. Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. 2025. Retool: Reinforcement learning for strategic tool use in llms. arXiv preprint arXiv:2504.11536.
  4. Bingguang Hao, Maolin Wang, Zengzhuang Xu, Cunyin Peng, Yicheng Chen, Xiangyu Zhao, Jinjie Gu, and Chenyi Zhuang. 2025. Funreason: Enhancing large language models’ function calling via self-refinement multiscale loss and automated data refinement. arXiv preprint arXiv:2505.20192. — [arXiv:2505.20192](https://arxiv.org/abs/2505.20192) find
  5. Yilun Hao, Yongchao Chen, Yang Zhang, and Chuchu Fan. 2024. Large language models can plan your travels rigorously with formal verification tools. CoRR. find
  6. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2023. Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798.
  7. Shijue Huang, Wanjun Zhong, Jianqiao Lu, Qi Zhu, Jiahui Gao, Weiwen Liu, Yutai Hou, Xingshan Zeng, Yasheng Wang, Lifeng Shang, and 1 others. 2024. Planning, creation, usage: Benchmarking llms for comprehensive tool utilization in real-world complex scenarios. arXiv preprint arXiv:2401.17167. find
  8. Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, and 1 others. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186. — Qwen2.5-Coder Technical Report
  9. Yuhua Jiang, Yuwen Xiong, Yufeng Yuan, Chao Xin, Wenyuan Xu, Yu Yue, Qianchuan Zhao, and Lin Yan. 2025. Pag: Multi-turn reinforced llm self-correction with policy as generative verifier. arXiv preprint arXiv:2506.10406.
  10. Barrett Martin Lattimer, Varun Gangal, Ryan McDonald, and Yi Yang. 2024. Sparse rewards can self-train dialogue agents. arXiv preprint arXiv:2409.04617. find
  11. Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. 2023. Api-bank: A comprehensive benchmark for tool-augmented llms. arXiv preprint arXiv:2304.08244.
  12. Xuefeng Li, Haoyang Zou, and Pengfei Liu. 2025. Torl: Scaling tool-integrated rl. arXiv preprint arXiv:2503.23383.
  13. Mingyang Chen, Haoze Sun, Tianpeng Li, Fan Yang, Hao Liang, Keer Lu, Bin Cui, Wentao Zhang, Zenan Zhou, and Weipeng Chen. 2024. Facilitating multi-turn function calling for llms via compositional instruction tuning. arXiv preprint arXiv:2410.12952. — [arXiv:2410.12952](https://arxiv.org/abs/2410.12952) find
  14. Fengyuan Liu, Nouar AlDahoul, Gregory Eady, Yasir Zaki, and Talal Rahwan. 2024a. Self-reflection makes large language models safer, less biased, and ideologically neutral. arXiv preprint arXiv:2406.10400. — [arXiv:2406.10400](https://arxiv.org/abs/2406.10400) find
  15. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv e-prints, pages arXiv–2407.
  16. Weiwen Liu, Xu Huang, Xingshan Zeng, Xinlong Hao, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Zhengying Liu, Yuanqing Yu, and 1 others. 2024b. Toolace: Winning the points of llm function calling. arXiv preprint arXiv:2409.00920.
  17. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, and 1 others. 2023. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36:46534–46594. find
  18. OpenAI. 2024a. Gpt-4o system card. Preprint, arXiv:2410.21276. Accessed: 2025-09-25.
  19. OpenAI. 2024b. Hello gpt-4o. https://openai.com/ index/hello-gpt-4o/. Accessed: 2025-09-25. ↗
  20. OpenAI. 2025. Introducing gpt-4.1 in the api. https://openai.com/index/gpt-4-1/. Accessed: 2025-09-25. ↗
  21. Shishir G Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E Gonzalez. The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models. In Forty-second International Conference on Machine Learning. find
  22. Cheng Qian, Emre Can Acikgoz, Qi He, Hongru Wang, Xiusi Chen, Dilek Hakkani-Tür, Gokhan Tur, and Heng Ji. 2025. Toolrl: Reward is all tool learning needs. arXiv preprint arXiv:2504.13958.
  23. Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, and 1 others. 2023. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789.
  24. Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and J Wen. 2024a. Tool learning with large language models: A survey. corr abs/2405.17935(2024). arXiv preprint arXiv:2405.17935.
  25. Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and J Wen. 2024b. Tool learning with large language models: A survey. corr abs/2405.17935(2024). arXiv preprint arXiv:2405.17935.
  26. Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji-Rong Wen. 2025. From exploration to mastery: Enabling llms to master tools via self-driven interactions. In International Conference on Learning Representations (ICLR). find
  27. Matthew Renze and Erhan Guven. 2024. Self-reflection in llm agents: Effects on problem-solving performance. arXiv preprint arXiv:2405.06682.
  28. RI Saveliev and MV Dendiuk. 2024. Self-reflective retrieval-augmented generation (self-rag) in analytical systems. In Forestry Education and Science: Current Challenges and Development Prospects. International Science-Practical Conference, October 23-25, 2024, Lviv, Ukraine. find
  29. Meituan LongCat Team, Bei Li, Bingye Lei, Bo Wang, Bolin Rong, Chao Wang, Chao Zhang, Chen Gao, Chen Zhang, Cheng Sun, and 1 others. 2025. Longcat-flash technical report. arXiv preprint arXiv:2509.01322. — [arXiv:2509.01322](https://arxiv.org/abs/2509.01322) find
  30. Adrian Theuma and Ehsan Shareghi. 2024. Equipping language models with tool use capability for tabular data analysis in finance. arXiv preprint arXiv:2401.15328. find
  31. Juraj Vladika, Ihsan Soydemir, and Florian Matthes. 2025. Correcting hallucinations in news summaries: Exploration of self-correcting llm methods with external knowledge. arXiv preprint arXiv:2506.19607. find
  32. MAOLIN WANG, YINGYI ZHANG, CUNYIN PENG, YICHENG CHEN, WEI ZHOU, JINJIE GU, CHENYI ZHUANG, RUOCHENG GUO, BOWEN YU, WANYU WANG, and 1 others. 2025. Function calling in large language models: Industrial practices, challenges, and future directions. find
  33. Renxi Wang, Xudong Han, Lei Ji, Shu Wang, Timothy Baldwin, and Haonan Li. 2024. Toolgen: Unified tool retrieval and calling via generation. arXiv preprint arXiv:2410.03439. — [arXiv:2410.03439](https://arxiv.org/abs/2410.03439) find
  34. Zhenyu Wu, Qingkai Zeng, Zhihan Zhang, Zhaoxuan Tan, Chao Shen, and Meng Jiang. 2024. Large language models can self-correct with key condition verification. arXiv preprint arXiv:2405.14092. find
  35. Qiancheng Xu, Yongqi Li, Heming Xia, and Wenjie Li. 2024. Enhancing tool retrieval with iterative feedback from large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 9609–9619. find
  36. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  37. Ling Yang, Zhaochen Yu, Tianjun Zhang, Minkai Xu, Joseph E Gonzalez, Bin Cui, and Shuicheng Yan. 2024. Supercorrect: Supervising and correcting language models with error-driven insights. arXiv preprint arXiv:2410.09008, 9. — [arXiv:2410.09008](https://arxiv.org/abs/2410.09008) find
  38. Junjie Ye, Yilong Wu, Sixian Li, Yuming Yang, Tao Gui, Qi Zhang, Xuanjing Huang, Peng Wang, Zhongchao Shi, Jianping Fan, and 1 others. 2024. Tl-training: A task-feature-based framework for training large language models in tool use. arXiv preprint arXiv:2412.15495. find
  39. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, and 1 others. 2025. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476.
  40. Jianguo Zhang, Tian Lan, Ming Zhu, Zuxin Liu, Thai Hoang, Shirley Kokane, Weiran Yao, Juntao Tan, Akshara Prabhakar, Haolin Chen, and 1 others. 2024. xlam: A family of large action models to empower ai agent systems. arXiv preprint arXiv:2409.03215.
  41. Xutong Zhao, Tengyu Xu, Xuewei Wang, Zhengxing Chen, Di Jin, Liang Tan, Zishun Yu, Zhuokai Zhao, Yun He, Sinong Wang, and 1 others. 2025a. Boosting llm reasoning via spontaneous self-correction. arXiv preprint arXiv:2506.06923. — [arXiv:2506.06923](https://arxiv.org/abs/2506.06923) find
  42. Yuze Zhao, Jintao Huang, Jinghan Hu, Xingjun Wang, Yunlin Mao, Daoze Zhang, Zeyinzi Jiang, Zhikai Wu, Baole Ai, Ang Wang, and 1 others. 2025b. Swift: a scalable lightweight infrastructure for fine-tuning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 29733–29735. find
  43. Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, and 1 others. 2025. Group sequence policy optimization. arXiv preprint arXiv:2507.18071.
  44. Ruizhe Zhong, Xingbo Du, Shixiong Kai, Zhentao Tang, Siyuan Xu, Hui-Ling Zhen, Jianye Hao, Qiang Xu, Mingxuan Yuan, and Junchi Yan. 2023. Llm4eda: Emerging progress in large language models for electronic design automation. arXiv preprint arXiv:2401.12224. — [arXiv:2401.12224](https://arxiv.org/abs/2401.12224) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/su-2026-failure-makes-agent-stronger-enhancing/