Elamur Research⌕
☰
home / papers / 2025 / 大语言模型的工具使用综述

大语言模型的工具使用综述

Summary

This Chinese-language survey from Fudan University reviews how large language models (LLMs) can call external tools such as search engines, code interpreters, APIs and mobile apps. Tools help LLMs overcome their weak access to fresh information and their imprecise arithmetic. The authors divide tool use into three abilities: deciding when and how to use tools (planning), producing machine-parsable calls, and turning the returned results into a final answer.

They also contrast two modes. In intrinsic use, tools are learned during training. In in-context use, tool descriptions are supplied at inference time. The survey then traces the field chronologically, from TALM and PAL to systems that combine code-style and API-style tool use.

It catalogs datasets and evaluation methods and groups training and inference techniques by the ability they target. It also discusses applications in agents and embodied intelligence. The main contribution is a structured map of a fast-moving area, along with open problems: latency, fragmented benchmarks, hallucination and multimodal tool use.

Key points

  • Early milestones:
    • TALM (Google, May 2022) fine-tuned T5 models of 220M–3B parameters.
    • PAL (Nov 2022) let Codex write Python for math reasoning.
    • GPT-4 plugins (Mar 2023) allowed up to three tools per conversation.
    • MOSS (Fudan, Apr 2023) followed shortly after.
  • Two research paradigms:
    • Code-based tools (Python functions, neural modules) handle complex multi-tool logic well.
    • API/interface-based tools increasingly include smartphone apps, e.g. AppWorld, ToolSandbox and AndroidWorld.
  • Dataset comparison: 24 tool-use datasets are compared by:
    • tool count and data size,
    • single vs. multi-turn dialogue,
    • single vs. multi-tool calls,
    • independent, dependent or nested relations between calls.
  • Largest tool collection: ToolLLM, built on RapidAPI, has the most tools of any dataset (roughly 16k), though its data quality is limited.
  • Robustness and stability: NoisyToolBench and RoTBench target noisy or unclear inputs. StableToolBench uses a virtual API server to keep evaluations reproducible.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Natural language processing

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.16383/j.aas.c240793

@inproceedings{2025,
  title = {{大语言模型的工具使用综述}},
  author = {郑逸宁 and 余镇 and 李不凡 and 杨捷 and 殷林琪 and 印张悦 and 袁枫烨 and 魏海洋 and 陆嘉昊 and 方世成 and 陈爽 and 邱锡鹏},
  booktitle = {自动化学报 (Acta Automatica Sinica)},
  year = {2025},
  doi = {10.16383/j.aas.c240793},
  url = {https://doi.org/10.16383/j.aas.c240793}
}

References

124 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Achiam J, Adler S, Agarwal S, Ahmad L, Akkaya I, Aleman F L, et al. GPT-4 technical report. arXiv preprint arXiv: 2303.08774, 2024. find our card
  2. Liu W W, Huang X, Zeng X S, Hao X L, Yu S, Li D X, et al. ToolACE: Winning the points of LLM function calling. arXiv preprint arXiv: 2409.00920, 2024. find
  3. Abdelaziz I, Basu K, Agarwal M, Kumaravel S, Stallone M, Panda R, et al. Granite-function calling model: Introducing function calling abilities via multi-task learning of granular tasks. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing: Industry Track. Miami, USA… find
  4. Gao L Y, Madaan A, Zhou S Y, Alon U, Liu P F, Yang Y M, et al. PAL: Program-aided language models. In: Proceedings of the 40th International Conference on Machine Learning. Honolulu, USA: PMLR, 2023. 10764−10799 find
  5. Parisi A, Zhao Y, Fiedel N. TALM: Tool augmented language models. arXiv preprint arXiv: 2205.12255, 2022. find our card
  6. Schick T, Dwivedi-Yu J, Dessi R, Raileanu R, Lomeli M, Hambro E, et al. Toolformer: Language models can teach themselves to use tools. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2023. 68539−68551 find our card
  7. Sun T X, Zhang X T, He Z F, Li P, Cheng Q Y, Liu X Y, et al. MOSS: An open conversational large language model. Machine Intelligence Research, 2024, 21(5): 888−905 find
  8. Grattafiori A, Dubey A, Jauhri A, Pandey A, Kadian A, Al-Dahle A, et al. The llama 3 herd of models. arXiv preprint arXiv: 2407.21783, 2024. find
  9. Qwen Team. QwQ-32B: Embracing the power of reinforcement learning [Online], available: https://qwenlm.github.io/blog/qwq-32b/, May 14, 2025 ↗
  10. Huang Y, Shi J W, Li Y, Fan C R, Wu S Y, Zhang Q H, et al. MetaTool benchmark for large language models: Deciding whether to use tools and which to use. In: Proceedings of the 12th International Conference on Learning Representations. Vienna, Austria: OpenReview.net, 2024. find
  11. Shen Y L, Song K T, Tan X, Li D S, Lu W M, Zhuang Y T. HuggingGPT: Solving AI tasks with ChatGPT and its friends in hugging face. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2023. Article No. 1657 find our card
  12. Gou Z B, Shao Z H, Gong Y Y, Shen Y L, Yang Y J, Duan N, et al. CRITIC: Large language models can self-correct with tool-interactive critiquing. In: Proceedings of the 12th International Conference on Learning Representations. Vienna, Austria: OpenReview.net, 2024. find our card
  13. Chen Z H, Du W H, Zhang W W, Liu K K, Liu J N, Zheng M, et al. T-eval: Evaluating the tool utilization capability of large language models step by step. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. Bangkok, Thailand: Association for Computational Lingu… find
  14. Qin Y J, Liang S H, Ye Y N, Zhu K L, Yan L, Lu Y X, et al. ToolLLM: Facilitating large language models to master 16000+ real-world APIs. In: Proceedings of the 12th International Conference on Learning Representations. Vienna, Austria: OpenReview.net, 2024. find our card
  15. Yang R, Song L, Li Y W, Zhao S J, Ge Y X, Li X, et al. GPT4Tools: Teaching large language model to use tools via self-instruction. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2023. Article No. 3149 find
  16. Wang B S, Fang H, Eisner J, van Durme B, Su Y. LLMs in the imaginarium: Tool learning through simulated trial and error. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. Bangkok, Thailand: ACL, 2024. 10583−10604 find
  17. Xu Q T, Hong F L, Li B, Hu C R, Chen Z Y, Zhang J. On the tool manipulation capability of open-source large language models. arXiv preprint arXiv: 2305.16504, 2023. find
  18. Qiao S F, Gui H H, Lv C F, Jia Q H, Chen H J, Zhang N Y. Making language models better tool learners with execution feedback. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). Mexico… find
  19. Li M H, Zhao Y X, Yu B W, Song F F, Li H Y, Yu H Y, et al. API-bank: A comprehensive benchmark for tool-augmented LLMs. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Singapore, Singapore: ACL, 2023. 3102−3116 find our card
  20. Tang Q Y, Deng Z L, Lin H Y, Han X P, Liang Q, Cao B X, et al. ToolAlpaca: Generalized tool learning for language models with 3000 simulated cases. arXiv preprint arXiv: 2306.05301, 2023. find our card
  21. Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, et al. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 2020, 21(1): Article No. 140 find
  22. Chen M, Tworek J, Jun H W, Yuan Q M, de Oliveira Pinto H P, Kaplan J, et al. Evaluating large language models trained on code. arXiv preprint arXiv: 2107.03374, 2021. find our card
  23. Surís D, Menon S, Vondrick C. ViperGPT: Visual inference via python execution for reasoning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (IC-CV). Paris, France: IEEE, 2023. 11854−11864 find our card
  24. Cai T L, Wang X Z, Ma T Y, Chen X Y, Zhou D. Large language models as tool makers. In: Proceedings of the 12th International Conference on Learning Representations. Vienna, Austria: OpenReview.net, 2024. find
  25. Qian C, Han C, Fung Y, Qin Y J, Liu Z Y, Ji H. CREATOR: Tool creation for disentangling abstract and concrete reasoning of large language models. In: Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2023. find
  26. Singapore, Singapore: ACL, 2023. 6922−6939 find
  27. Gou Z B, Shao Z H, Gong Y Y, Shen Y L, Yang Y J, Huang M L, et al. ToRA: A tool-integrated reasoning agent for mathematical problem solving. In: Proceedings of the 12th International Conference on Learning Representations. Vienna, Austria: OpenReview.net, 2024. find our card
  28. Trivedi H, Khot T, Hartmann M, Manku R, Dong V, Li E, et al. AppWorld: A controllable world of apps and people for benchmarking interactive coding agents. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Bangkok, Thailand: ACL, 2024… find
  29. Shen H Y, Li Y, Meng D S, Cai D Q, Qi S, Zhang L, et al. ShortcutsBench: A large-scale real-world benchmark for API-based agents. In: Proceedings of the 13th International Conference on Learning Representations. Singapore, Singapore: Open-Review.net, 2025. find
  30. Lu J R, Holleis T, Zhang Y Z, Aumayer B, Nan F, Bai H P, et al. ToolSandbox: A stateful, conversational, interactive evaluation benchmark for LLM tool use capabilities. In: Proceedings of the Findings of the Association for Computational Linguistics: NAACL 2025. find
  31. Albuquerque, USA: ACL, 2025. 1160−1183 find
  32. Rawles C, Clinckemaillie S, Chang Y F, Waltz J, Lau G, Fair M, et al. AndroidWorld: A dynamic benchmarking environment for autonomous agents. In: Proceedings of the 13th International Conference on Learning Representations. Singapore, Singapore: OpenReview.net, 2025. find
  33. Patil S G, Zhang T J, Wang X, Gonzalez J E. Gorilla: Large language model connected with massive APIs. In: Proceedings of the 38th International Conference on Neural Information Processing Systems. Vancouver, Canada: NeurIPS, 2024. find our card
  34. Hao S B, Liu T Y, Wang Z, Hu Z T. ToolkenGPT: Augmenting frozen language models with massive tools via tool embeddings. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2023. find our card
  35. Gao S, Shi Z L, Zhu M H, Fang B W, Xin X, Ren P J, et al. Confucius: Iterative tool learning from introspection feedback by easy-to-difficult curriculum. In: Proceedings of the 38th AAAI Conference on Artificial Intelligence. Vancouver, Canada: AAAI Press, 2024. 18030−18038 find
  36. Yuan L F, Chen Y Y, Wang X Y, Fung Y, Peng H, Ji H. CRAFT: Customizing LLMs by creating and retrieving from specialized toolsets. In: Proceedings of the 12th International Conference on Learning Representations. Vienna, Austria: OpenReview.net, 2024. find
  37. Zhuang Y C, Chen X, Yu T, Mitra S, Bursztyn V, Rossi R A, et al. ToolChain*: Efficient action space navigation in large language models with A* search. In: Proceedings of the 12th International Conference on Learning Representations. Vienna, Austria: OpenReview.net, 2024. find
  38. Farn N, Shin R. ToolTalk: Evaluating tool-usage in a conversational setting. arXiv preprint arXiv: 2311.10775, 2023. — [arXiv:2311.10775](https://arxiv.org/abs/2311.10775) find
  39. Gao Z, Du Y T, Zhang X T, Ma X J, Han W, Zhu S C, et al. CLOVA: A closed-loop visual assistant with tool usage and update. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2024. 13258−13268 find
  40. Ye J J, Li G Y, Gao S Y, Huang C S, Wu Y L, Li S X, et al. ToolEyes: Fine-grained evaluation for tool learning capabilities of large language models in real-world scenarios. In: Proceedings of the 31st International Conference on Computational Linguistics. Abu Dhabi, UAE: ACL, 2025. 156−187 find
  41. Wang C Y, Luo W X, Dong S X, Xuan X H, Li Z X, Ma L, et al. MLLM-Tool: A multimodal large language model for tool agent learning. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Tucson, USA: IEEE, 2025. 6678−6687 find
  42. Wang Z R, Neubig G, Fried D. TroVE: Inducing verifiable and efficient toolboxes for solving programmatic tasks. In: Proceedings of the 41st International Conference on Machine Learning. Vienna, Austria: OpenReview.net, 2024. find
  43. Yuan S Y, Song K T, Chen J J, Tan X, Shen Y L, Ren K, et al. EASYTOOL: Enhancing LLM-based agents with concise tool instruction. In: Proceedings of the ICLR Workshop on Large Language Model (LLM) Agents. Vienna, Austria: OpenReview.net, 2024. find
  44. Du Y, Wei F Y, Zhang H Y. AnyTool: Self-reflective, hierarchical agents for large-scale API calls. In: Proceedings of the 41st International Conference on Machine Learning. Vienna, Austria: OpenReview.net, 2024. find
  45. Ma Y B, Gou Z B, Hao J H, Xu R C, Wang S H, Pan L M, et al. SciAgent: Tool-augmented language models for scientific reasoning. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Miami, USA: ACL, 2024. 15701−15736 find
  46. Zheng Y H, Li P, Liu W, Liu Y, Luan J, Wang B. ToolRerank: Adaptive and hierarchy-aware reranking for tool retrieval. In: Proceedings of the Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). find
  47. Torino, Italia: ACL, 2024. 16263−16273 find
  48. Wu M S, Zhu T, Han H, Tan C Y, Zhang X, Chen W L. Seal-tools: Self-instruct tool learning dataset for agent tuning and detailed benchmark. In: Proceedings of the 13th National CCF Conference on Natural Language Processing and Chinese Computing. Hangzhou, China: Springer, 2024. 372−384 find
  49. Chen S J, Wang Y B, Wu Y F, Chen Q G, Xu Z, Luo W H, et al. Advancing tool-augmented large language models: Integrating insights from errors in inference trees. In: Proceedings of the 38th International Conference on Neural Information Processing Systems. Vancouver, Canada: NeurIPS, 2024. find
  50. Huang S J, Zhong W J, Lu J Q, Zhu Q, Gao J H, Liu W W, et al. Planning, creation, usage: Benchmarking LLMs for comprehensive tool utilization in real-world complex scenarios. In: Proceedings of the Findings of the Association for Computational Linguistics: ACL 2024. find
  51. Bangkok, Thailand: ACL, 2024. 4363−4400 find
  52. Wang J Z, Ma Z R, Li Y N, Zhang S Y, Chen C L, Chen K, et al. GTA: A benchmark for general tool agents. In: Proceedings of the 38th International Conference on Neural Information Processing Systems. Vancouver, Canada: NeurIPS, 2024. find
  53. Yu Y Q, Wang Z F, Ma W Z, Guo Z C, Zhan J T, Wang S, et al. StepTool: A step-grained reinforcement learning framework for tool learning in LLMs. arXiv preprint arXiv: 2410.07745, 2024. find
  54. Wang P, Wu Y N, Wang N, Liu J H, Song X S, Peng Z Y, et al. MTU-Bench: A multi-granularity tool-use benchmark for large language models. In: Proceedings of the 13th International Conference on Learning Representations. Singapore, Singapore: OpenReview.net, 2025. find
  55. Wang R X, Han X D, Ji L, Wang S, Baldwin T, Li H N. Tool-Gen: Unified tool retrieval and calling via generation. In: Proceedings of the 13th International Conference on Learning Representations. Singapore, Singapore: OpenReview.net, 2025. find
  56. RapidAPI. RapidAPI: A platform for discovering and connecting to APIs [Online], available: https://rapidapi.com/, May 15, 2024 ↗
  57. Georgiev P, Lei V I, Burnell R, Bai L B, Gulati A, Tanzer G, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv: 2403.05530, 2024. find our card
  58. Yang A, Yang B S, Hui B Y, Zheng B, Yu B W, Zhou C, et al. Qwen2 technical report. arXiv preprint arXiv: 2407.10671, 2024. find
  59. Zhu Q H, Guo D Y, Shao Z H, Yang D J, Wang P Y, Xu R X, et al. DeepSeek-coder-V2: Breaking the barrier of closed-source models in code intelligence. arXiv preprint arXiv: 2406.11931, 2024. find our card
  60. Song Y F, Xiong W M, Zhu D W, Wu W H, Qian H, Song M B, et al. RestGPT: Connecting large language models with real-world RESTful APIs. arXiv preprint arXiv: 2306.06624, 2023. find
  61. Zhuang Y C, Yu Y, Wang K, Sun H T, Zhang C. ToolQA: A dataset for LLM question answering with external tools. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2023. Article No. 2180 find our card
  62. Shen Y L, Song K T, Tan X, Zhang W Q, Ren K, Yuan S Y, et al. TaskBench: Benchmarking large language models for task automation. In: Proceedings of the 38th International Conference on Neural Information Processing Systems. Vancouver, Canada: NeurIPS, 2024. find
  63. Guo Z S, Huang Y F, Xiong D Y. CToolEval: A Chinese benchmark for LLM-powered agent evaluation in real-world API interactions. In: Proceedings of the Findings of the Association for Computational Linguistics: ACL 2024. Bangkok, Thailand: Association for Computational Linguistics, 2024. 15711−15724 find
  64. Basu K, Abdelaziz I, Chaudhury S, Dan S, Crouse M, Munawar A, et al. API-BLEND: A comprehensive corpora for training and benchmarking API LLMs. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Bangkok, Thailand: ACL, 2024. 12859−128… find
  65. Wang H R, Wang R, Xue B Y, Xia H M, Cao J T, Liu Z M, et al. AppBench: Planning of multiple APIs from various APPs for complex user instruction. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Miami, USA: ACL, 2024. 15322−15336 find
  66. Wang W X, Shi J L, Wang C Z, Lee C, Yuan Y L, Huang J T, et al. Learning to ask: When LLMs meet unclear instruction. arXiv preprint arXiv: 2409.00557, 2024. — [arXiv:2409.00557](https://arxiv.org/abs/2409.00557) find
  67. Ye J J, Li S X, Li G Y, Huang C S, Gao S Y, Wu Y L, et al. ToolSword: Unveiling safety issues of large language models in tool learning across three stages. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Bangkok, Thailand: ACL, 20… find
  68. Ye J J, Wu Y L, Gao S Y, Huang C S, Li S X, Li G Y, et al. RoTBench: A multi-level benchmark for evaluating the robustness of large language models in tool learning. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Miami, USA: ACL, 2024. 313−333 find
  69. Guo Z C, Cheng S J, Wang H, Liang S H, Qin Y J, Li P, et al. StableToolBench: Towards stable large-scale benchmarking on tool learning of large language models. In: Proceedings of the Findings of the Association for Computational Linguistics: ACL 2024. find
  70. Bangkok, Thailand: ACL, 2024. 11143−11156 find
  71. Papineni K, Roukos S, Ward T, Zhu W J. BLEU: A method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. Philadelphia, USA: Association for Computational Linguistics, 2002. 311−318 find our card
  72. Lin C Y. ROUGE: A package for automatic evaluation of summaries. In: Proceedings of the Text Summarization Branches Out. Barcelona, Spain: Association for Computational Linguistics, 2004. 74−81 find
  73. Bergroth L, Hakonen H, Raita T. A survey of longest common subsequence algorithms. In: Proceedings of the 7th International Symposium on String Processing and Information Retrieval. SPIRE 2000. find
  74. A Curuna, Spain: IEEE, 2000. 39−48 find
  75. Liu Y M, Peng X Y, Zhang Y W, Cao J N, Zhang X H, Cheng S, et al. Tool-planner: Dynamic solution tree planning for large language model with tool clustering. arXiv preprint arXiv: 2406.03807, 2024. — [arXiv:2406.03807](https://arxiv.org/abs/2406.03807) find
  76. Qiao S F, Fang R N, Qiu Z S, Wang X B, Zhang N Y, Jiang Y, et al. Benchmarking agentic workflow generation. arXiv preprint arXiv: 2410.07869, 2024. — [arXiv:2410.07869](https://arxiv.org/abs/2410.07869) find
  77. OpenMOSS. UnifiedToolHub [Online], available: https://git-hub.com/OpenMOSS/UnifiedToolHub, May 14, 2025 ↗
  78. Zhou S Y, Xu F F, Zhu H, Zhou X H, Lo R, Sridhar A, et al. WebArena: A realistic web environment for building autonomous agents. In: Proceedings of the 12th International Conference on Learning Representations. Vienna, Austria: OpenReview.net, 2024. find our card
  79. Kim G, Baldi P, McAleer S. Language models can solve computer tasks. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2023. Article No. 1723 find our card
  80. Liu Y L, Yuan Y L, Wang C W, Han J H, Ma Y Q, Zhang L, et al. From summary to action: Enhancing large language models for complex tasks with open world APIs. arXiv preprint arXiv: 2402.18157, 2024. — [arXiv:2402.18157](https://arxiv.org/abs/2402.18157) find
  81. Liu X, Qin B, Liang D Z, Dong G, Lai H Y, Zhang H C, et al. AutoGLM: Autonomous foundation agents for GUIs. arXiv preprint arXiv: 2411.00820, 2024. — [arXiv:2411.00820](https://arxiv.org/abs/2411.00820) find
  82. Qi Z H, Liu X, Iong I L, Lai H Y, Sun X Q, Sun J D, et al. WebRL: Training LLM web agents via self-evolving online curriculum reinforcement learning. In: Proceedings of the 13th International Conference on Learning Representations. Singapore, Singapore: OpenReview.net, 2025. find
  83. Wu Q Z, Liu W, Luan J, Wang B. ToolPlanner: A tool augmented LLM for multi granularity instructions with path planning and feedback. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Miami, USA: Association for Computational Linguistics, 2024. 18315−18339 find
  84. Chen K, Cusumano-Towner M, Huval B, Petrenko A, Hamburger J, Koltun V, et al. Reinforcement learning for long-horizon interactive LLM agents. arXiv preprint arXiv: 2502.01600, 2025. — [arXiv:2502.01600](https://arxiv.org/abs/2502.01600) find
  85. Kong Y L, Ruan J Q, Chen Y H, Zhang B, Bao T P, Shiwei S, et al. TPTU-v2: Boosting task planning and tool usage of large language model-based agents in real-world industry systems. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing: Industry Track. Miami, US: Assoc… find
  86. Liu X K, Peng Z Y, Yi X Y, Xie X, Xiang L R, Liu Y C, et al. ToolNet: Connecting large language models with massive tools via tool graph. arXiv preprint arXiv: 2403.00839, 2024. — [arXiv:2403.00839](https://arxiv.org/abs/2403.00839) find
  87. Huang W L, Abbeel P, Pathak D, Mordatch I. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In: Proceedings of the 39th International Conference on Machine Learning. Baltimore, USA: PMLR, 2022. 9118−9147 find our card
  88. Xu H S, Zhu S, Wang Z H, Zheng H, Ma D, Cao R S, et al. Reducing tool hallucination via reliability alignment. arXiv preprint arXiv: 2412.04141, 2024. — [arXiv:2412.04141](https://arxiv.org/abs/2412.04141) find
  89. Xu G W, Jin P, Li H, Song Y B, Sun L C, Yuan L. LLaVA-CoT: Let vision language models reason step-by-step. arXiv preprint arXiv: 2411.10440, 2024. find
  90. Koh J Y, McAleer S, Fried D, Salakhutdinov R. Tree search for language model agents. arXiv preprint arXiv: 2407.01476, 2024. find
  91. Chen P, Bu P, Song J, Gao Y, Zheng B. Can VLMs play action role-playing games? Take black myth Wukong as a study case. In: Proceedings of the NeurIPS Workshop on Open-World Agents. Vancouver, Canada: NeurIPS, 2024. find
  92. Nakano R, Hilton J, Balaji S, Wu J, Ouyang L, Kim C, et al. WebGPT: Browser-assisted question-answering with human feedback. arXiv preprint arXiv: 2112.09332, 2022. find our card
  93. Yao S Y, Chen H, Yang J, Narasimhan K. WebShop: Towards scalable real-world web interaction with grounded language agents. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2022. Article No. 1508 find our card
  94. Qiao S F, Fang R N, Zhang N Y, Zhu Y Q, Chen X, Deng S M, et al. Agent planning with world knowledge model. In: Proceedings of the 38th International Conference on Neural Information Processing Systems. Vancouver, Canada: NeurIPS, 2024. find
  95. Cao H Y, Zhang Y Q, Feng S, Yang X C, Wang D L, Zhang Y F. TOOL-ED: Enhancing empathetic response generation with the tool calling capability of LLM. In: Proceedings of the 31st International Conference on Computational Linguistics. Abu Dhabi, UAE: Association for Computational Linguistics, 2025. 53… find
  96. Liao Z Y, Mo L B, Xu C J, Kang M T, Zhang J W, Xiao C W, et al. EIA: Environmental injection attack on generalist web agents for privacy leakage. arXiv preprint arXiv: 2409.11295, 2025. — [arXiv:2409.11295](https://arxiv.org/abs/2409.11295) find
  97. Chen Z R, Xiang Z, Xiao C W, Song D, Li B. AgentPoison: Red-teaming LLM agents via poisoning memory or knowledge bases. In: Proceedings of the 38th International Conference on Neural Information Processing Systems. Vancouver, Canada: NeurIPS, 2024. find
  98. Xiang Z, Zheng L Z, Li Y J, Hong J Y, Li Q B, Xie H, et al. GuardAgent: Safeguard LLM agents by a guard agent via knowledge-enabled reasoning. arXiv preprint arXiv: 2406.09187, 2024. — [arXiv:2406.09187](https://arxiv.org/abs/2406.09187) find
  99. Andrychowicz M, Baker B, Chociej M, Józefowicz R, McGrew B, Pachocki J, et al. Learning dexterous in-hand manipulation. The International Journal of Robotics Research, 2020, 39(1): 3− 20 find
  100. Kavraki L E, Svestka P, Latombe J C, Overmars M H. Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE Transactions on Robotics and Automation, 1996, 12(4): 566−580 find
  101. Shen Z Y, Wilson J P, Harvey R, Gupta S. MRRT: Multiple rapidly-exploring random trees for fast online replanning in dynamic environments. arXiv preprint arXiv: 2104.11059, 2021. — [arXiv:2104.11059](https://arxiv.org/abs/2104.11059) find
  102. Liang J, Huang W L, Xia F, Xu P, Hausman K, Ichter B, et al. Code as policies: Language model programs for embodied control. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). London, UK: IEEE, 2023. 9493−9500 find
  103. Ichter B, Brohan A, Chebotar Y, Finn C, Hausman K, Herzog A, et al. Do as I can, not as I say: Grounding language in robotic affordances. In: Proceedings of the 6th Conference on Robot Learning. Auckland, New Zealand: PMLR, 2022. 287−318 find our card
  104. Yu Q J, Huang S Y, Yuan X B, Jiang Z K, Hao C, Li X, et al. UniAff: A unified representation of affordances for tool usage and articulation with vision-language models. arXiv preprint arXiv: 2409.20551, 2024. — [arXiv:2409.20551](https://arxiv.org/abs/2409.20551) find
  105. Huang W L, Wang C, Zhang R H, Li Y Z, Wu J J, Fei-Fei L. VoxPoser: Composable 3D value maps for robotic manipulation with language models. In: Proceedings of the 7th Conference on Robot Learning. Atlanta, USA: PMLR, 2023. 540−562 find
  106. Huang W L, Wang C, Li Y Z, Zhang R H, Fei-Fei L. Rekep: Spatio-temporal reasoning of relational keypoint constraints for robotic manipulation. In: Proceedings of the 8th Conference on Robot Learning. Munich, Germany: PMLR, 2025. 4573−4602 find
  107. Cai M X, Wang D L, Feng S, Zhang Y F. PECER: Empathetic response generation via dynamic personality extraction and contextual emotional reasoning. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Seoul, South Korea: IEEE, 2024. 10631− 10635 find
  108. Jin Q, Yang Y F, Chen Q Y, Lu Z Y. GeneGPT: Augmenting large language models with domain tools for improved access to biomedical information. Bioinformatics, 2024, 40(2): btae075 find
  109. Xiao S T, Liu Z, Zhang P T, Muennighoff N, Lian D, Nie J Y. C-pack: Packed resources for general Chinese embeddings. In: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. Washington, USA: Association for Computing Machinery, 2024. 641−64… find our card
  110. Li Z C, Wang J H, Jiang Z S, Mao H Y, Chen Z X, Du J Z, et al. DMQR-RAG: Diverse multi-query rewriting for RAG. arXiv preprint arXiv: 2411.13154, 2024. — [arXiv:2411.13154](https://arxiv.org/abs/2411.13154) find
  111. Guo D Y, Yang D J, Zhang H W, Song J X, Zhang R Y, Xu R X, et al. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv: 2501.12948, 2025. find our card
  112. Zeng Z Y, Cheng Q Y, Yin Z Y, Wang B, Li S M, Zhou Y H, et al. Scaling of search and learning: A roadmap to reproduce o1 from reinforcement learning perspective. arXiv preprint arXiv: 2412.14135, 2024. find
  113. 郑逸宁 复旦大学计算与智能创新学 院博士研究生. 2019 年获得复旦大学 学士学位. 主要研究方向为大语言模 型中的代码生成与工具使用. E-mail: ynzheng19@fudan.edu.cn (ZHENG Yi-Ning Ph.D. candidate at the College of Computer Science and Artificial Intelligence, Fudan University. He received his bachelor degree from Fudan University in 2019. His research interest c… find
  114. 余 镇 复旦大学计算与智能创新学 院硕士研究生. 2022 年获得江西师范 大学学士学位. 主要研究方向为工具 学习和个性化智能体. E-mail: yuz24@m.fudan.edu.cn (YU Zhen Master student at the College of Computer Science and Artificial Intelligence, Fudan University. He received his bachelor degree from Jiangxi Normal University in 2022. His research interest cove… find
  115. 李不凡 复旦大学计算与智能创新学 院硕士研究生. 2024 年获得东北大学 学士学位. 主要研究方向为大语言模 型的后训练. E-mail: 24210240197@ m.fudan.edu.cn (LI Bu-Fan Master student at the College of Computer Science and Artificial Intelligence, Fudan University. He received his bachelor degree from Northeastern University in 2024. His main research inter… find
  116. 杨 捷 复旦大学计算与智能创新学 院硕士研究生. 2021 年获得武汉大学 学士学位. 主要研究方向为工具调用 与智能体. E-mail: yangj24@m.fudan.edu.cn (YANG Jie Master student at the College of Computer Science and Artificial Intelligence, Fudan University. He received his bachelor degree from Wuhan University in 2021. find
  117. His research interest covers tool use and agent.) 殷林琪 复旦大学计算与智能创新学 院博士研究生. 2025 年获得复旦大学 学士学位. 主要研究方向为工具学习. E-mail: 21307140112@m.fudan.edu. cn (YIN Lin-Qi Ph.D. candidate at the College of Computer Science and Artificial Intelligence, Fudan University. She received her bachelor degree from Fudan Uni… find
  118. 印张悦 复旦大学计算与智能创新学 院博士研究生. 2021 年获得华东师范 大学学士学位. 主要研究方向为大语 言模型和机器推理. E-mail: yinzy21@m.fudan.edu.cn (YIN Zhang-Yue Ph.D. candidate at the College of Computer Science and Artificial Intelligence. He received his bachelor degree from East China Normal University in 2021. His research interest covers larg… find
  119. 袁枫烨 复旦大学计算与智能创新学 院硕士研究生. 2024 年获得同济大学 学士学位. 主要研究方向为大语言模 型的工具调用与智能体. E-mail: fyyuan24@m.fudan.edu.cn (YUAN Feng-Ye Master student at the College of Computer Science and Artificial Intelligence, Fudan University. He received his bachelor degree from Tongji University in 2024. His research interest cov… find
  120. 魏海洋 复旦大学计算与智能创新学 院硕士研究生. 2022 年获得安徽大学 学士学位. 主要研究方向为工具学习 和复杂意图检测. E-mail: 23210240325@m.fudan.edu. cn (WEI Hai-Yang Master student at the College of Computer Science and Artificial Intelligence, Fudan University. He received his bachelor degree from Anhui University in 2022. His research interest cov… find
  121. 陆嘉昊 复旦大学计算与智能创新学 院博士研究生. 2025 年获得复旦大学 学士学位. 主要研究方向为工具学习. E-mail: 21307130022@m.fudan.edu. cn (LU Jia-Hao Ph.D. candidate at the College of Computer Science and Artificial Intelligence, Fudan University. He received his bachelor degree from Fudan University in 2025. His main research interest is tool… find
  122. 方世成 复旦大学计算与智能创新学 院博士研究生. 2025 年获得复旦大学 学士学位. 主要研究方向为工具学习. E-mail: 21307140067@m.fudan.edu. cn (FANG Shi-Cheng Ph.D. candidate at the College of Computer Science and Artificial Intelligence, Fudan University. He received his bachelor degree from Fudan University in 2025. His main research interest is… find
  123. 陈 爽 复旦大学计算与智能创新学 院博士后. 2023 年获得东北大学博士 学位. 主要研究方向为自然语言处理 和大语言模型中的情感计算. E-mail: chenshuang_fd@fudan.edu.cn (CHEN Shuang Postdoctor at the College of Computer Science and Artificial Intelligence, Fudan University. She received her Ph.D. degree from Northeastern University in 2023. Her research interest… find
  124. 邱锡鹏 复旦大学计算与智能创新学 院教授. 2006 年获得复旦大学博士学 位. 主要研究方向为自然语言处理和 大语言模型. 本文通信作者. E-mail: xpqiu@fudan.edu.cn (QIU Xi-Peng Professor at the College of Computer Science and Artificial Intelligence, Fudan University. He received his Ph.D. degree from Fudan University in 2006. His research interest covers natural… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zheng-2025-tool-use-large/