Elamur Research⌕
☰
home / papers / 2026 / MatchTIR: Fine-Grained Supervision for Tool-Integrated Reasoning via Bipartite Matching

MatchTIR: Fine-Grained Supervision for Tool-Integrated Reasoning via Bipartite Matching

Paper · 2026 · Changle Qu, Sunhao Dai, Hengyi Cai et al.

Summary

Reinforcement learning for tool-integrated reasoning (TIR) in LLMs usually rewards whole outcomes or trajectories. Every turn of a multi-step interaction then gets the same advantage, whether a tool call helped, was redundant, or was wrong. MatchTIR addresses this by scoring each turn against reference tool calls. Predicted and ground-truth calls form a weighted bipartite graph, with edge weights based on tool names, parameter names and parameter values. Two alignment strategies turn the graph into dense per-turn rewards: a strict one-to-one matching and a softer one-to-many matching. These turn rewards are combined with the final outcome reward through a dual-level advantage. One component reflects the quality of the whole trajectory. The other reflects each turn's discounted future rewards. The policy is trained with GRPO. On three in-domain and out-of-domain benchmarks, a 4B model trained this way beats most 8B baselines, with the largest gains on long, multi-turn tasks. The trained agents also make fewer and more accurate tool calls. This matters because it gives step-level credit without costly Monte Carlo rollouts or biased learned reward models.

Key points

  • Similarity score: each predicted/gold call pair gets a value in [0, 1] built from three parts, gated by the tool-name match:
    • exact tool-name agreement;
    • Jaccard overlap of parameter-name sets;
    • counts of correct parameter values.
  • Two assignment strategies:
    • hard assignment uses the Hungarian algorithm (one-to-one);
    • soft assignment uses optimal transport (one-to-many, probabilistic).
  • Models: experiments use Qwen3-4B and Qwen3-8B. The 4B MatchTIR model outperforms the majority of 8B competitors. Code is released at github.com/quchangle1/MatchTIR.
  • Hyper-parameters:
    • a larger penalty λ for unmatched calls raises Solve-P but slightly lowers Solve-R;
    • raising the discount factor γ from 0.1 to 0.9 steadily improves all metrics.
  • Limitations: the method requires ground-truth tool-call traces, which makes open-ended tasks such as deep research hard to cover. Models larger than 8B were not tested.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢BaiduBeijing · China74 works · 15,263 citations🎓Renmin University of ChinaBeijing · China114 works · 16,118 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Changle Qu, Sunhao Dai, Hengyi Cai, Jun Xu, Shuaiqiang Wang and Dawei Yin. 2026. MatchTIR: Fine-Grained Supervision for Tool-Integrated Reasoning via Bipartite Matching. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.549

@inproceedings{qu2026matchtir,
  title = {{MatchTIR: Fine-Grained Supervision for Tool-Integrated Reasoning via Bipartite Matching}},
  author = {Changle Qu, Sunhao Dai, Hengyi Cai, Jun Xu, Shuaiqiang Wang and Dawei Yin},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.549},
  url = {https://doi.org/10.18653/v1/2026.acl-long.549}
}

References

51 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Qikai Chang, Zhenrong Zhang, Pengfei Hu, Jun Du, Jiefeng Ma, Yicheng Pan, Jianshu Zhang, Quan Liu, and Jianqing Gao. 2025. Thor: Tool-integrated hierarchical optimization via rl for mathematical reasoning. arXiv preprint arXiv:2509.13761. — [arXiv:2509.13761](https://arxiv.org/abs/2509.13761) find
  2. Cong Chen, Kaixiang Ji, Hao Zhong, Muzhi Zhu, Anzhou Li, Guo Gan, Ziyuan Huang, Cheng Zou, Jiajia Liu, Jingdong Chen, Hao Chen, and Chunhua Shen. 2025. Gui-shepherd: Reliable process reward and verification for long-sequence gui tasks. Preprint, arXiv:2509.23738. — [arXiv:2509.23738](https://arxiv.org/abs/2509.23738) find
  3. Marco Cuturi. 2013. Sinkhorn distances: lightspeed computation of optimal transport. In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, NIPS’13, page 2292–2300, Red Hook, NY, USA. Curran Associates Inc. find
  4. Debrup Das, Debopriyo Banerjee, Somak Aditya, and Ashish Kulkarni. 2024. Mathsensei: a tool-augmented large language model for mathematical reasoning. In In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies… find
  5. Guanting Dong, Hangyu Mao, Kai Ma, Licheng Bao, Yifei Chen, Zhongyuan Wang, Zhongxia Chen, Jiazhen Du, Huiyang Wang, Fuzheng Zhang, and 1 others. 2025. Agentic reinforced policy optimization. arXiv preprint arXiv:2507.19849. — [arXiv:2507.19849](https://arxiv.org/abs/2507.19849) find
  6. Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. 2025a. Retool: Reinforcement learning for strategic tool use in llms. arXiv preprint arXiv:2504.11536. find
  7. Lang Feng, Zhenghai Xue, Tingcong Liu, and Bo An. 2025b. Group-in-group policy optimization for LLM agent training. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  8. Guo Gan, Yuxuan Ding, Cong Chen, Yuwei Ren, Yin Huang, and Hong Zhou. 2026. Android coach: Improve online agentic training efficiency with single state multiple actions. Preprint, arXiv:2604.07277.
  9. Zhibin Gou, Zhihong Shao, Yeyun Gong, Yujiu Yang, Minlie Huang, Nan Duan, Weizhu Chen, and 1 others. 2024. Tora: A tool-integrated reasoning agent for mathematical problem solving. In Proceedings of the 12th International Conference on Learning Representations (ICLR).
  10. Dongfu Jiang, Yi Lu, Zhuofeng Li, Zhiheng Lyu, Ping Nie, Haozhe Wang, Alex Su, Hui Chen, Kai Zou, Chao Du, and 1 others. 2025. Verltool: Towards holistic agentic reinforcement learning with tool use. arXiv preprint arXiv:2509.01055. — [arXiv:2509.01055](https://arxiv.org/abs/2509.01055) find
  11. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan O Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-r1: Training LLMs to reason and leverage search engines with reinforcement learning. In Second Conference on Language Modeling.
  12. Harold W Kuhn. 1956. Variants of the hungarian method for assignment problems. Naval research logistics quarterly, 3(4):253–258. find
  13. Xuefeng Li, Haoyang Zou, and Pengfei Liu. 2025a. Torl: Scaling tool-integrated rl. arXiv preprint arXiv:2503.23383. find
  14. Yuchen Li, Hengyi Cai, Rui Kong, Xinran Chen, Jiamin Chen, Jun Yang, Haojie Zhang, Jiayi Li, Jiayi Wu, Yiqun Chen, and 1 others. 2025b. Towards ai search paradigm. arXiv preprint arXiv:2506.17188. — [arXiv:2506.17188](https://arxiv.org/abs/2506.17188) find
  15. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. Let’s verify step by step. In The Twelfth International Conference on Learning Representations.
  16. Heng Lin and Zhongwen Xu. 2025. Understanding tool-integrated reasoning. arXiv preprint arXiv:2508.19201. — [arXiv:2508.19201](https://arxiv.org/abs/2508.19201) find
  17. Shishir G. Patil, Huanzhi Mao, Charlie Cheng-Jie Ji, Fanjia Yan, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. 2025. The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models. In Forty-second International Conference on Machine Learning. find
  18. Cheng Qian, Emre Can Acikgoz, Qi He, Hongru WANG, Xiusi Chen, Dilek Hakkani-Tür, Gokhan Tur, and Heng Ji. 2025. ToolRL: Reward is all tool learning needs. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  19. Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, and 1 others. 2024. Toolllm: Facilitating large language models to master 16000+ real-world apis. In Proceedings of the 12th International Conference on Learning Representations (ICLR).
  20. Changle Qu, Sunhao Dai, Hengyi Cai, Yiyang Cheng, Jun Xu, Shuaiqiang Wang, and Dawei Yin. 2025a. Uplift-rag: Uplift-driven knowledge preference alignment for retrieval-augmented generation. Findings of the Association for Computational Linguistics: EMNLP. find
  21. Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji-Rong Wen. 2024. Towards completeness-oriented tool retrieval for large language models. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management. find
  22. Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji-Rong Wen. 2025b. From exploration to mastery: Enabling LLMs to master tools via self-driven interactions. In The Thirteenth International Conference on Learning Representations. find
  23. Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji-Rong Wen. 2025c. Tool learning with large language models: A survey. Frontiers of Computer Science, 19(8):198343.
  24. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  25. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2025. Hybridflow: A flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, pages 1279–1297.
  26. Hieu Tran, Zonghai Yao, and Hong Yu. 2025. Exploiting tree structure for credit assignment in rl training of llms. arXiv preprint arXiv:2509.18314. — [arXiv:2509.18314](https://arxiv.org/abs/2509.18314) find
  27. Guoqing Wang, Sunhao Dai, Guangze Ye, Zeyu Gan, Wei Yao, Yong Deng, Xiaofeng Wu, and Zhenzhe Ying. 2025a. Information gain-based policy optimization: A simple and effective approach for multi-turn llm agents. arXiv preprint arXiv:2510.14967. — [arXiv:2510.14967](https://arxiv.org/abs/2510.14967) find
  28. Hanlin Wang, Chak Tou Leong, Jiashuo Wang, Jian Wang, and Wenjie Li. 2025b. Spa-rl: Reinforcing llm agents via stepwise progress attribution. arXiv preprint arXiv:2505.20732. — [arXiv:2505.20732](https://arxiv.org/abs/2505.20732) find
  29. Hongru Wang, Cheng Qian, Wanjun Zhong, Xiusi Chen, Jiahao Qiu, Shijue Huang, Bowen Jin, Mengdi Wang, Kam-Fai Wong, and Heng Ji. 2025c. Otc: Optimal tool calls via reinforcement learning. arXiv e-prints, pages arXiv–2504. find
  30. Ke Wang, Houxing Ren, Aojun Zhou, Zimu Lu, Sichun Luo, Weikang Shi, Renrui Zhang, Linqi Song, Mingjie Zhan, and Hongsheng Li. 2024a. Mathcoder: Seamless code integration in LLMs for enhanced mathematical reasoning. In The Twelfth International Conference on Learning Representations.
  31. Shuai Wang, Weiwen Liu, Jingxuan Chen, Yuqi Zhou, Weinan Gan, Xingshan Zeng, Yuhan Che, Shuai Yu, Xinlong Hao, Kun Shao, and 1 others. 2024b. Gui agents with foundation models: A comprehensive survey. arXiv preprint arXiv:2411.04890. — [arXiv:2411.04890](https://arxiv.org/abs/2411.04890) find
  32. Yifan Wei, Xiaoyan Yu, Yixuan Weng, Tengfei Pan, Angsheng Li, and Li Du. 2025. Autotir: Autonomous tools integrated reasoning via reinforcement learning. arXiv preprint arXiv:2507.21836. — [arXiv:2507.21836](https://arxiv.org/abs/2507.21836) find
  33. Zhenghai Xue, Longtao Zheng, Qian Liu, Yingru Li, Xiaosen Zheng, Zejun Ma, and Bo An. 2025. Simpletir: End-to-end reinforcement learning for multi-turn tool-integrated reasoning. arXiv preprint arXiv:2509.02479. — [arXiv:2509.02479](https://arxiv.org/abs/2509.02479) find
  34. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  35. Shuo Yang, Caren Han, Yihao Ding, Shuhe Wang, and Eduard Hovy. 2026a. Tooltree: Efficient LLM tool planning via dual-feedback monte carlo tree search and bidirectional pruning. In The Fourteenth International Conference on Learning Representations. find
  36. Shuo Yang, Soyeon Caren Han, Xueqi Ma, Yan Li, Mohammad Reza Ghasemi Madani, and Eduard Hovy. 2026b. Evotool: Self-evolving tool-use policy optimization in llm agents via blame-aware mutation and diversity-aware selection. arXiv preprint arXiv:2603.04900.
  37. Junjie Ye, Zhengyin Du, Xuesong Yao, Weijian Lin, Yufei Xu, Zehui Chen, Zaiyuan Wang, Sining Zhu, Zhiheng Xi, Siyu Yuan, and 1 others. 2025a. Toolhop: A query-driven benchmark for evaluating large language models in multi-hop tool use. In Proceedings of the 63rd Annual Meeting of the Association for…
  38. Junjie Ye, Changhao Jiang, Zhengyin Du, Yufei Xu, Xuesong Yao, Zhiheng Xi, Xiaoran Fan, Qi Zhang, Tao Gui, Xuanjing Huang, and 1 others. 2025b. Feedback-driven tool-use improvements in large language models via automated build environments. arXiv preprint arXiv:2508.08791.
  39. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, YuYue, Weinan Dai, Tiantian Fan, Gaohong Liu, Juncai Liu, LingJun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, and 17 others. 2025a. DAPO: An open-source LLM reinforcement learning s…
  40. Yuanqing Yu, Zhefan Wang, Weizhi Ma, Shuai Wang, Chuhan Wu, Zhiqiang Guo, and Min Zhang. 2025b. Steptool: Enhancing multi-step tool usage in llms via step-grained reinforcement learning. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management, pages 3952– 3962…
  41. Hong. 2025a. Reinforcing multi-turn reasoning in LLM agents via turn-level credit assignment. In ICML 2025 Workshop on Computer Use Agents. find
  42. Yirong Zeng, Xiao Ding, Yutai Hou, Yuxian Wang, Li Du, Juyi Dai, Qiuyang Ding, Duyu Tang, Dandan Tu, Weiwen Liu, and 1 others. 2025b. Tool zero: Training tool-augmented llms via pure rl from scratch. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 9135–9147. find
  43. Shaokun Zhang, Yi Dong, Jieyu Zhang, Jan Kautz, Bryan Catanzaro, Andrew Tao, Qingyun Wu, Zhiding Yu, and Guilin Liu. 2025a. Nemotron-research-tool-n1: Tool-using language models with reinforced reasoning. arXiv preprint arXiv:2505.00024. — [arXiv:2505.00024](https://arxiv.org/abs/2505.00024) find
  44. Wenlin Zhang, Xiangyang Li, Kuicai Dong, Yichao Wang, Pengyue Jia, Xiaopeng Li, Yingyi Zhang, Derong Xu, Zhaocheng Du, Huifeng Guo, Ruiming Tang, and Xiangyu Zhao. 2025b. Process vs. outcome reward: Which is better for agentic RAG reinforcement learning. In The Thirty-ninth Annual Conference on Neur… find
  45. Yaocheng Zhang, Haohuan Huang, Zijun Song, Yuanheng Zhu, Qichao Zhang, Zijie Zhao, and Dongbin Zhao. 2025c. Criticsearch: Fine-grained credit assignment for search agents via a retrospective critic. arXiv preprint arXiv:2511.12159.
  46. Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, and 1 others. 2025a. Group sequence policy optimization. arXiv preprint arXiv:2507.18071.
  47. Xuhui Zheng, Kang An, Ziliang Wang, Yuhang Wang, and Yichao Wu. 2025b. StepSearch: Igniting LLMs search ability via step-wise proximal policy optimization. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 21816–21841, Suzhou, China. Association for Com…
  48. Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojie Cai, Lyumanshan Ye, Pengrui Lu, and Pengfei Liu. 2025c. Deepresearcher: Scaling deep research via reinforcement learning in real-world environments. arXiv preprint arXiv:2504.03160.
  49. Yuqi Zhou, Sunhao Dai, Changle Qu, Liang Pang, Jun Xu, and Ji-Rong Wen. 2026. Learning to retrieve from agent trajectories. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval. find
  50. Chuhuai Yue, Chengqi Dong, Yinan Gao, Hang He, Jiajun Chai, Guojun Yin, and Wei Lin. 2025. Promoting efficient reasoning with verifiable stepwise reward. arXiv preprint arXiv:2508.10293. — [arXiv:2508.10293](https://arxiv.org/abs/2508.10293) find
  51. Siliang Zeng, Quan Wei, William Brown, Oana Frunza, Yuriy Nevmyvaka, Yang Katie Zhao, and Mingyi find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/qu-2026-matchtir-fine-grained-supervision-tool/