Elamur Research⌕
☰
home / papers / 2026 / ToolGate: Contract-Grounded and Verified Tool Execution for LLMs

ToolGate: Contract-Grounded and Verified Tool Execution for LLMs

Paper · 2026 · Yanming Liu, Xinyue Peng, Jiannan Cao et al.

Summary

LLM agents that call external APIs usually decide in free-form text when a tool may be used and whether its output can be trusted. This leaves no formal safeguard against malformed calls or hallucinated results spreading through a multi-step reasoning chain. ToolGate addresses this with a forward-execution framework built around an explicit, typed symbolic state, a key–value store of verified facts. Each tool is wrapped in a Hoare-style contract. The precondition checks that the required inputs already exist in the state before the tool can run. The postcondition validates the returned output before anything is written back.

The contracts are not generated by the LLM. They are extracted from interface specifications: required parameters define preconditions, and response schemas define postconditions. Tool retrieval relies on embedding-based semantic search. Experiments on ToolBench and MCP-Universe show that the approach improves reliability and verifiability while task performance stays competitive. The framework also degrades gracefully when contracts are incomplete and adds little latency. This matters for building debuggable, auditable agents.

Key points

  • The state is a set of typed tuples (key, value, type). Tools are formalized as {P_t} t {Q_t}. A call is allowed only if the state satisfies P_t, and a result is committed only if it passes Q_t.
  • Contracts are extracted deterministically from OpenAPI/JSON Schema. For MCP-Universe, they come from the MCP inputSchema and typed response interfaces.
  • About 25% of ToolBench tools lack a structured response schema. For these tools, the postcondition defaults to Q = True.
  • Acknowledged limitations:
    • Only text and structured-data tools are covered, and the evaluation environment is static.
    • ToolBench response schemas were produced with LLM assistance, so they are not ground-truth specifications.
    • Keeping contracts in sync as APIs change is left for future work.
  • Code is available at github.com/OceannTwT/ToolGate. The paper is published in Findings of ACL 2026, pp. 9653–9684.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Yanming Liu, Xinyue Peng, Jiannan Cao, Xinyi Wang, Songhang Deng, Jintao Chen, Jianwei Yin and Xuhong Zhang. 2026. ToolGate: Contract-Grounded and Verified Tool Execution for LLMs. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.470

@inproceedings{liu2026toolgate,
  title = {{ToolGate: Contract-Grounded and Verified Tool Execution for LLMs}},
  author = {Yanming Liu, Xinyue Peng, Jiannan Cao, Xinyi Wang, Songhang Deng, Jintao Chen, Jianwei Yin and Xuhong Zhang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.470},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.470}
}

References

49 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Danil S Grigorev, Alexey K Kovalev, and Aleksandr I Panov. 2025. Verifyllm: Llm-based pre-execution task plan verification for robots. arXiv preprint arXiv:2507.05118. — [arXiv:2507.05118](https://arxiv.org/abs/2507.05118) find
  2. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter, Chr… — FOLIO: Natural Language Reasoning with First-Order Logic
  3. Charles Antony Richard Hoare. 1969. An axiomatic basis for computer programming. Communications of the ACM, 12(10):576–580. find
  4. Xiao Hu. 2025a. AdPercept: Visual saliency and attention modeling in ad 3D design. In 2025 5th International Conference on Artificial Intelligence, Big Data and Algorithms (CAIBDA), pages 1258–1261. IEEE. find
  5. Xiao Hu. 2025b. GenPlayAds: Procedural playable 3D ad creation via generative model. In 2025 International Conference on Artificial Intelligence, Human-Computer Interaction and Natural Language Processing (ICAHN), pages 128–131. IEEE. find
  6. Yue Huang, Chujie Gao, Siyuan Wu, Haoran Wang, Xiangqi Wang, Yujun Zhou, Yanbo Wang, Jiayi Ye, Jiawen Shi, Qihui Zhang, et al. 2025. On the trustworthiness of generative foundation models: Guideline, assessment, and perspective. arXiv preprint arXiv:2502.14296. find
  7. Christine P. Lee, David Porfirio, Xinyu Jessica Wang, Kevin Chenkai Zhao, and Bilge Mutlu. 2025. Veriplan: Integrating formal verification and llms into end-user planning. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, CHI ’25, New York, NY, USA. Association for Com… find
  8. Chengpeng Li, Zhengyang Tang, Ziniu Li, Mingfeng Xue, Keqin Bao, Tian Ding, Ruoyu Sun, Benyou Wang, Xiang Wang, Junyang Lin, et al. 2025. Teaching language models to reason with tools. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  9. Xiaohan Lin, Qingxing Cao, Yinya Huang, Haiming Wang, Jianqiao Lu, Zhengying Liu, Linqi Song, and Xiaodan Liang. 2024. Fvel: Interactive formal verification environment with large language models via theorem proving. Advances in Neural Information Processing Systems, 37:54932–54946. find
  10. Ziyang Luo, Zhiqi Shen, Wenzhuo Yang, Zirui Zhao, Prathyusha Jwalapuram, Amrita Saha, Doyen Sahoo, Silvio Savarese, Caiming Xiong, and Junnan Li. 2025. MCP-universe: Benchmarking large language models with real-world model context protocol servers. In Workshop on Scaling Environments for Agents. find
  11. Mark Huasong Meng, Guangdong Bai, Sin Gee Teo, Zhe Hou, Yan Xiao, Yun Lin, and Jin Song Dong. 2022. Adversarial robustness of deep neural networks: A survey from a formal verification perspective. IEEE Transactions on Dependable and Secure Computing. find
  12. OpenAI. 2025. Introducing gpt-5. https://openai.com/index/introducing-gpt-5/. Accessed: 2025-08-14. ↗
  13. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke E. Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Francis Christiano, Jan Leike, and Ryan J. Low… — Training language models to follow instructions with human feedback
  14. Bhargavi Paranjape, Scott Lundberg, Sameer Singh, Hannaneh Hajishirzi, Luke Zettlemoyer, and Marco Tulio Ribeiro. 2023. Art: Automatic multi-step reasoning and tool-use for large language models. arXiv preprint arXiv:2303.09014. find
  15. Shishir G Patil, Tianjun Zhang, Xin Wang, and Joseph E Gonzalez. 2024. Gorilla: Large language model connected with massive apis. Advances in Neural Information Processing Systems, 37:126544–126565.
  16. Cheng Qian, Emre Can Acikgoz, Qi He, Hongru WANG, Xiusi Chen, Dilek Hakkani-Tür, Gokhan Tur, and Heng Ji. 2025. ToolRL: Reward is all tool learning needs. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  17. Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, et al. 2025a. Deepseekv3. 2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556.
  18. Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Xuanhe Zhou, Yufei Huang, Chaojun Xiao, et al. 2024. Tool learning with foundation models. ACM Computing Surveys, 57(4):1–40.
  19. Guangyi Liu, Pengxiang Zhao, Yaozhen Liang, Liang Liu, Yaxuan Guo, Han Xiao, Weifeng Lin, Yuxiang Chai, Yue Han, Shuai Ren, et al. 2025b. Llm-powered gui agents in phone automation: Surveying progress and prospects. arXiv preprint arXiv:2504.19838. — [arXiv:2504.19838](https://arxiv.org/abs/2504.19838) find
  20. Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. 2023. Toolllm: Facilitating large language models to master 16000+ real-world apis. In The Twelfth International Conference on Learning Representations.
  21. Yanming Liu, Xinyue Peng, Jiannan Cao, Shi Bo, Yuwei Zhang, Xuhong Zhang, Sheng Cheng, Xun Wang, Jianwei Yin, and Tianyu Du. 2025c. Tool-planner: Task planning with clusters across multiple tools. In The Thirteenth International Conference on Learning Representations. find
  22. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36:68539–68551.
  23. Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Schärli, and Denny Zhou. 2023. Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning, pages 31210–31227. PMLR.
  24. Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, et al. 2025. Openthinkimg: Learning to think with images via visual tool reinforcement learning. arXiv preprint arXiv:2505.08617. — [arXiv:2505.08617](https://arxiv.org/abs/2505.08617) find
  25. Anand Swaroop, Abhishek Singh, Girish Chandra, Shiv Prakash, Sohan Kumar Yadav, Tiansheng Yang, and Rajkumar Singh Rathore. 2024. A comprehensive overview of formal methods and deep learning for verification and optimization. In 2024 International Conference on Decision Aid Sciences and Applications… find
  26. Tongwei Tu. 2025a. Log2Learn: Intelligent log analysis for real-time network optimization. In 2025 4th International Conference on Artificial Intelligence, Internet and Digital Economy (ICAID), pages 162– 166. IEEE. find
  27. Tongwei Tu. 2025b. SmartFITLab: Intelligent execution and validation platform for 5G field interoperability testing. In 2025 7th International Conference on Next Generation Data-driven Networks (NGDN), pages 108–111. IEEE. find
  28. Po-Wei Wang, Priya Donti, Bryan Wilder, and Zico Kolter. 2019. Satnet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver. In International Conference on Machine Learning, pages 6545–6554. PMLR. find
  29. Minhui Xie and Shujian Chen. 2025. InVis: Interactive neural visualization system for human-centered data interpretation. In 2025 IEEE 7th International Conference on Communications, Information System and Computer Engineering (CISCE), pages 636–640. IEEE. find
  30. Minhui Xie and Boyan Liu. 2025. EvalNet: Sentiment analysis and multimodal data fusion for recruitment interview processing. In 2025 7th International Conference on Artificial Intelligence Technologies and Applications (ICAITA), pages 444–448. IEEE. find
  31. Haoran Xu. 2025. CivicMorph: Generative modeling for public space form development. In 2025 5th International Conference on Artificial Intelligence, Big Data and Algorithms (CAIBDA), pages 497–501. IEEE. find
  32. Qiancheng Xu, Yongqi Li, Heming Xia, and Wenjie Li. 2024. Enhancing tool retrieval with iterative feedback from large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 9609–9619. find
  33. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025a. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  34. Hsiu-Wei Yang, Abhinav Agrawal, Pavlos Fragkogian-nis, and Shubham Nitin Mulay. 2024a. Can ai models appreciate document aesthetics? an exploration of legibility and layout quality in relation to prediction confidence. ArXiv, abs/2403.18183. — [arXiv:2403.18183](https://arxiv.org/abs/2403.18183) find
  35. Jie Yang, Yiqiu Tang, Yongjie Li, Lihua Zhang, and Haoran Zhang. 2025b. Cross-asset risk management: Integrating LLMs for real-time monitoring of equity, fixed income, and currency markets. In 2025 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE. find
  36. Jie Yang, Yiqiu Tang, Yongjie Li, Lihua Zhang, and Haoran Zhang. 2025c. Dynamic hedging strategies in derivatives markets with LLM-driven sentiment and news analytics. In 2025 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE. find
  37. Yu’an Yang, Siheng Xiong, Ali Payani, Ehsan Shareghi, and Faramarz Fekri. 2024b. Harnessing the power of large language models for natural language to first-order logic translation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pa… find
  38. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. In The eleventh international conference on learning representations.
  39. Junjie Ye, Guanyu Li, Songyang Gao, Caishuang Huang, Yilong Wu, Sixian Li, Xiaoran Fan, Shihan Dou, Tao Ji, Qi Zhang, et al. 2025. Tooleyes: Fine-grained evaluation for tool learning capabilities of large language models in real-world scenarios. In Proceedings of the 31st international conference on…
  40. Yuanqing Yu, Zhefan Wang, Weizhi Ma, Shuai Wang, Chuhan Wu, Zhiqiang Guo, and Min Zhang. 2025. Steptool: Enhancing multi-step tool usage in llms via step-grained reinforcement learning. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management, CIKM ’25, page 39…
  41. Chaoyun Zhang, Shilin He, Liqun Li, Si Qin, Yu Kang, Qingwei Lin, Saravan Rajmohan, and Dongmei Zhang. 2025a. API agents vs. GUI agents: Divergence and convergence. In ICML 2025 Workshop on Computer Use Agents. find
  42. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, et al. 2025b. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176.
  43. Yuhan Zhang. 2025a. CrossPlatformStack: Enabling high availability and safe deployment for products across meta services. In 2025 4th International Symposium on Robotics, Artificial Intelligence and Information Engineering (RAIIE), pages 360–363. IEEE. find
  44. Yuhan Zhang. 2025b. InfraMLForge: Developer tooling for rapid LLM development and scalable deployment. In 2025 8th International Conference on Computer Information Science and Application Technology (CISAT), pages 273–277. IEEE. find
  45. Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu-Xiong Wang. 2024. Language agent tree search unifies reasoning, acting, and planning in language models. In International Conference on Machine Learning, pages 62138–62160. PMLR. find
  46. Bingxin Zhu. 2025a. RAID: Reliability automation through intelligent detection in large-scale ad systems. In Proceedings of the 2025 2nd International Conference on Digital Society and Artificial Intelligence, pages 964–967. find
  47. Bingxin Zhu. 2025b. TraceLM: Temporal root-cause analysis with contextual embedding language models. In 2025 6th International Conference on Electronic Communication and Artificial Intelligence (ICECAI), pages 855–858. IEEE. find
  48. Xiaochen Zhu, Caiqi Zhang, Tom Stafford, Nigel Collier, and Andreas Vlachos. 2025. Conformity in large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3854–3872.
  49. Yuchen Zhuang, Xiang Chen, Tong Yu, Saayan Mitra, Victor Bursztyn, Ryan A Rossi, Somdeb Sarkhel, and Chao Zhang. 2024. Toolchain*: Efficient action space navigation in large language models with a* search. In The Twelfth International Conference on Learning Representations. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2026-toolgate-contract-grounded-verified-tool/