Elamur Research⌕
☰
home / papers / 2026 / MedMCP-Calc: Benchmarking LLMs for Realistic Medical Calculator Scenarios via MCP Integration

MedMCP-Calc: Benchmarking LLMs for Realistic Medical Calculator Scenarios via MCP Integration

Paper · 2026 · Yakun Zhu, Yutong Huang, Shengqian Qin et al.

Summary

Real clinical use of medical calculators rarely involves one isolated formula. A clinician pulls scattered values from the EHR, decides which scores fit the evolving situation, and chains several computations together. Existing benchmarks such as MedCalc-Bench instead give clean patient notes and name the calculator explicitly. MedMCP-Calc addresses this gap with 118 scenario tasks in four clinical domains. Each task is posed as a vague, goal-driven clinician request. Models solve the tasks through Model Context Protocol (MCP) servers: a PostgreSQL database with EHR data built from MIMIC-IV plus synthetic records, Google Search for reference lookup, and a Python executor for computation. Scoring covers the whole process: calculator selection, evidence acquisition and numerical accuracy. The authors tested 23 models and found that even the strongest perform poorly. The main problems were missing required calculators, weak iterative SQL querying and a reluctance to call computational tools. The authors also fine-tuned CalcMate using scenario-planning data and heavy tool-use traces, and it became the best open-source model on the benchmark. The work shows how far current agents are from dependable clinical workflow support.

Key points

  • The benchmark has 118 tasks across 4 clinical domains and three MCP environments (PostgreSQL, Google Search, Python Executor). Code is released at github.com/SPIRAL-MED/MedMCP-Calc.
  • The top model, Claude Opus 4.5, reached only 33.92% calculation accuracy. Results also differed sharply by domain, with complex scoring systems and cognitive scales being especially hard.
  • Tool use was conservative. Models made about 3–5 SQL calls per task, but most averaged under one Python call. Search and fetch were close to unused; Qwen3-235B-Instruct-2507 made zero calls to either.
  • CalcMate was trained on 1,000 GPT-5-generated scenario tasks with global plans, plus ReAct-style traces covering calculator lookup, database exploration and code-based computation. Its EA metric rose from 9.34 to 71.06, and it outperformed both models used to generate its training data.
  • Stated limitations include possible region-specific bias in the EHR data. CalcMate's tool-heavy approach also lengthens context, which adds latency and compute cost.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Shanghai Jiao Tong UniversityShanghai · China149 works · 16,161 citations🎓Shanghai Innovation InstituteShanghai · China24 works · 262 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksMedicine · 241 worksHealth Informatics · 184 works

Cite

Cite as Yakun Zhu, Yutong Huang, Shengqian Qin, Zhongzhen Huang, Shaoting Zhang and Xiaofan Zhang. 2026. MedMCP-Calc: Benchmarking LLMs for Realistic Medical Calculator Scenarios via MCP Integration. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.221

@inproceedings{zhu2026medmcp,
  title = {{MedMCP-Calc: Benchmarking LLMs for Realistic Medical Calculator Scenarios via MCP Integration}},
  author = {Yakun Zhu, Yutong Huang, Shengqian Qin, Zhongzhen Huang, Shaoting Zhang and Xiaofan Zhang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.221},
  url = {https://doi.org/10.18653/v1/2026.acl-long.221}
}

References

76 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anthropic. 2024. Introducing computer use, a new claude 3.5 sonnet, and claude 3.5 haiku. Accessed: 2025-06-30. find
  2. Anthropic. 2024. Introducing the model context protocol. find
  3. Anthropic. 2024. Introducing the model context protocol. Accessed: 2025-06-30. find
  4. Anthropic. 2025a. Claude sonnet 4.5. find
  5. Anthropic. 2025b. Introducing claude 4. Accessed: 2025-07-28. find
  6. Anthropic. 2025c. Introducing claude opus 4.5. find
  7. Kedi Chen, Zhikai Lei, Xu Guo, Xuecheng Wu, Siyuan Zeng, Jianghao Yin, Yinqi Zhang, Qin Chen, Jie Zhou, Liang He, and 1 others. 2025a. Code-driven number sequence calculation: Enhancing the inductive reasoning abilities of large language models. arXiv preprint arXiv:2510.14620. — [arXiv:2510.14620](https://arxiv.org/abs/2510.14620) find
  8. Kedi Chen, Dezhao Ruan, Yuhao Dan, Yaoting Wang, Siyu Yan, Xuecheng Wu, Yinqi Zhang, Qin Chen, Jie Zhou, Liang He, and 1 others. 2025b. A survey of inductive reasoning for large language models. arXiv preprint arXiv:2510.10182.
  9. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.
  10. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arX…
  11. Jean-Philippe Corbeil, Asma Ben Abacha, George Michalopoulos, Phillip Swazinna, Miguel Del-Agua, Jérôme Tremblay, Akila Jeeson Daniel, Cari Bader, Kevin Cho, Pooja Krishnan, and 1 others. 2025. Empowering healthcare practitioners with language models: Structuring speech transcripts in two real-world… find
  12. Google Deepmind. 2025. Gemini 3 pro. find
  13. DeepSeek-AI. 2024. Deepseek-v3 technical report. Preprint, arXiv:2412.19437. find
  14. Shuai Dong, Siyuan Wang, Xingyu Liu, and Zhongyu Wei. 2025a. Interleaved latent visual reasoning with selective perceptual modeling. ArXiv, abs/2512.05665.
  15. Shuai Dong, Jie Zhang, Guoying Zhao, Shiguang Shan, and Xilin Chen. 2025b. Semantic mismatch and perceptual degradation: A new perspective on image editing immunity. ArXiv, abs/2512.14320. — [arXiv:2512.14320](https://arxiv.org/abs/2512.14320) find
  16. Zixuan Huang, Yikun Ban, Lean Fu, Xiaojie Li, Zhongxiang Dai, Jianxin Li, and Deqing Wang. 2025. Adaptive batch-wise sample scheduling for direct preference optimization. arXiv preprint arXiv:2506.17252. — [arXiv:2506.17252](https://arxiv.org/abs/2506.17252) find
  17. Chengfeng Dou, Chong Liu, Fan Yang, Fei Li, Jiyuan Jia, Mingyang Chen, Qiang Ju, Shuai Wang, Shunya Dang, Tianpeng Li, and 1 others. 2025. Baichuanm2: Scaling medical capability with large verifier system. arXiv preprint arXiv:2509.02208.
  18. Zixuan Huang, Xin Xia, Yuxi Ren, Jianbin Zheng, Xuanda Wang, Zhixia Zhang, Hongyan Xie, Songshi Liang, Zehao Chen, Xuefeng Xiao, and 1 others. 2026a. Does your reasoning model implicitly know when to stop thinking? arXiv preprint arXiv:2602.08354. — [arXiv:2602.08354](https://arxiv.org/abs/2602.08354) find
  19. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv e-prints, pages arXiv–2407.
  20. Shengda Fan, Xin Cong, Yuepeng Fu, Zhong Zhang, Shuyan Zhang, Yuanwei Liu, Yesai Wu, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2024. Workflowllm: Enhancing workflow orchestration capability of large language models. arXiv preprint arXiv:2411.05451. — [arXiv:2411.05451](https://arxiv.org/abs/2411.05451) find
  21. Xuanqi Gao, Siyi Xie, Juan Zhai, Shiqing Ma, and Chao Shen. 2025. Mcp-radar: A multi-dimensional benchmark for evaluating tool use capabilities in large language models. arXiv preprint arXiv:2505.16700. — [arXiv:2505.16700](https://arxiv.org/abs/2505.16700) find
  22. Alex J Goodell, Simon N Chu, Dara Rouholiman, and Larry F Chu. 2023. Augmentation of chatgpt with clinician-informed tools improves performance on medical calculation tasks. MedRxiv, pages 2023–12. find
  23. Zikang Guo, Benfeng Xu, Chiwei Zhu, Wentao Hong, Xiaorui Wang, and Zhendong Mao. 2025. Mcp-agentbench: Evaluating real-world language agent performance with mcp-mediated tools. arXiv preprint arXiv:2509.09734. — [arXiv:2509.09734](https://arxiv.org/abs/2509.09734) find
  24. Mohammed Mehedi Hasan, Hao Li, Emad Fallahzadeh, Gopi Krishnan Rajbahadur, Bram Adams, and Ahmed E Hassan. 2025. Model context protocol (mcp) at first glance: Studying the security and maintainability of mcp servers. arXiv preprint arXiv:2506.13538. — [arXiv:2506.13538](https://arxiv.org/abs/2506.13538) find
  25. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874.
  26. Zixuan Huang, Xin Xia, Yuxi Ren, Jianbin Zheng, Xuefeng Xiao, Hongyan Xie, Li Huaqiu, Songshi Liang, Zhongxiang Dai, Fuzhen Zhuang, and 1 others. 2026b. Real-time aligned reward model beyond semantics. arXiv preprint arXiv:2601.22664. — [arXiv:2601.22664](https://arxiv.org/abs/2601.22664) find
  27. Hongrui Jia, Jitong Liao, Xi Zhang, Haiyang Xu, Tianbao Xie, Chaoya Jiang, Ming Yan, Si Liu, Wei Ye, and Fei Huang. 2025. Osworld-mcp: Benchmarking mcp tool invocation in computer-use agents. arXiv preprint arXiv:2510.24563. — [arXiv:2510.24563](https://arxiv.org/abs/2510.24563) find
  28. Qiao Jin, Zhizheng Wang, Yifan Yang, Qingqing Zhu, Donald Wright, Thomas Huang, Nikhil Khandekar, Nicholas Wan, Xuguang Ai, W John Wilbur, and 1 others. 2025. Agentmd: Empowering language agents for risk prediction with large-scale clinical tool learning. Nature Communications, 16(1):9377. find
  29. Nikhil Khandekar, Qiao Jin, Guangzhi Xiong, Soren Dunn, Serina Applebaum, Zain Anwar, Maame Sarfo-Gyamfi, Conrad Safranek, Abid Anwar, Andrew Zhang, and 1 others. 2024. Medcalc-bench: Evaluating large language models for medical calculations. Advances in Neural Information Processing Systems, 37:847… find
  30. Fei Lei, Yibo Yang, Wenxiu Sun, and Dahua Lin. 2025. Mcpverse: An expansive, real-world benchmark for agentic tool use. arXiv preprint arXiv:2508.16260. — [arXiv:2508.16260](https://arxiv.org/abs/2508.16260) find
  31. Chenglin Li, Feng Han, Feng Tao, Ruilin Li, Qianglong Chen, Jingqi Tong, Yin Zhang, and Jiaqi Wang. 2025. Videopro: Adaptive program reasoning for long video understanding.
  32. Rick Hightower. 2025. Mcp the usb-c for ai. find
  33. Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. 2023. Api-bank: A comprehensive benchmark for tool-augmented llms. arXiv preprint arXiv:2304.08244.
  34. Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, and 1 others. 2023. Metagpt: Meta programming for a multi-agent collaborative framework. In The Twelfth International Conference on Learning Representations.
  35. Zhiwei Liu, Jielin Qiu, Shiyu Wang, Jianguo Zhang, Zuxin Liu, Roshan Ram, Haolin Chen, Weiran Yao, Shelby Heinecke, Silvio Savarese, and 1 others. 2025. Mcpeval: Automatic mcp-based deep evaluation for ai agent models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Pr… find
  36. Xinyi Hou, Yanjie Zhao, Shenao Wang, and Haoyu Wang. 2025. Model context protocol (mcp): Landscape, security threats, and future research directions. arXiv preprint arXiv:2503.23278. — [arXiv:2503.23278](https://arxiv.org/abs/2503.23278) find
  37. Zhizhao Liu, Zhihua Wen, Zhiliang Tian, Zhen Huang, Miaorong Zhu, Zimian Wei, Yifu Gao, Liang Ding, and Dongsheng Li. 2026. Rethinking the hidden risk of reranking: Achieving risk-aware reranking with information gain for rag with llms. In Proceedings of the ACM Web Conference 2026, pages 1887–1898. find
  38. Ziyang Luo, Zhiqi Shen, Wenzhuo Yang, Zirui Zhao, Prathyusha Jwalapuram, Amrita Saha, Doyen Sahoo, Silvio Savarese, Caiming Xiong, and Junnan Li. 2025. Mcp-universe: Benchmarking large language models with real-world model context protocol servers. arXiv preprint arXiv:2508.14704. — [arXiv:2508.14704](https://arxiv.org/abs/2508.14704) find
  39. Kanato Masayoshi, Masahiro Hashimoto, Ryoichi Yokoyama, Naoki Toda, Yoshifumi Uwamino, Shogo Fukuda, Ho Namkoong, and Masahiro Jinzaki. 2025. Ehr-mcp: Real-world evaluation of clinical information retrieval by large language models via model context protocol. arXiv preprint arXiv:2509.15957. — [arXiv:2509.15957](https://arxiv.org/abs/2509.15957) find
  40. Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. 2023. Gaia: a benchmark for general ai assistants. In The Twelfth International Conference on Learning Representations.
  41. Guozhao Mo, Wenliang Zhong, Jiawei Chen, Xuanang Chen, Yaojie Lu, Hongyu Lin, Ben He, Xianpei Han, and Le Sun. 2025. Livemcpbench: Can agents navigate an ocean of mcp tools? arXiv preprint arXiv:2508.01780. — [arXiv:2508.01780](https://arxiv.org/abs/2508.01780) find
  42. OpenAI. 2025a. gpt-oss-120b gpt-oss-20b model card. Preprint, arXiv:2508.10925.
  43. OpenAI. 2025b. Introducing gpt-5. find
  44. OpenAI. 2025. Introducing operator: An agent that can use a computer. The underlying model is the Computer-Using Agent (CUA). find
  45. Shishir G Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E Gonzalez. The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models. In Forty-second International Conference on Machine Learning. find
  46. Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, Chelsea Finn, Divyansh Garg, and Rafael Rafailov. 2024. Agent q: Advanced reasoning and learning for autonomous ai agents. arXiv preprint arXiv:2408.07199. find
  47. Shuofei Qiao, Runnan Fang, Zhisong Qiu, Xiaobin Wang, Ningyu Zhang, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. 2024. Benchmarking agentic workflow generation. arXiv preprint arXiv:2410.07869. — [arXiv:2410.07869](https://arxiv.org/abs/2410.07869) find
  48. Shengqian Qin, Yakun Zhu, Linjie Mu, Shaoting Zhang, and Xiaofan Zhang. 2025. Meta-tool: Unleash open-world function calling capabilities of general-purpose large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page… find
  49. Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, and 1 others. 2023. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789.
  50. Qwen Team. 2025. Qwen3 technical report. Technical Report; arXiv preprint. find
  51. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36:68539–68551.
  52. Andrew Sellergren, Sahar Kazemzadeh, Tiam Jaroensri, Atilla Kiraly, Madeleine Traverse, Timo Kohlberger, Shawn Xu, Fayaz Jamil, Cían Hughes, Charles Lau, and 1 others. 2025. Medgemma technical report. arXiv preprint arXiv:2507.05201. find
  53. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  54. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634–8652.
  55. GLM Team, Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, Kedong Wang, Lucen Zhong, Mingdao Liu, Rui Lu, Shulin Cao, Xiaohan Zhang, Xuancheng Huang, Yao Wei, and 152 others. 2025a. Glm-4.5: Agentic, reasoning, and coding (arc) fou…
  56. Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, and 1 others. 2025b. Kimi k2: Open agentic intelligence. arXiv preprint arXiv:2507.20534.
  57. Qwen Team. 2024. Qwen2.5: A party of foundation models. find
  58. Qwen Team. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  59. Dianyi Wang, Wei Song, Yikun Wang, Siyuan Wang, Kaicheng yu, Zhongyu Wei, and Jiaqi Wang. 2025a. Autoregressive semantic visual reconstruction helps vlms understand better. ArXiv, abs/2506.09040.
  60. Junyang Wang, Haiyang Xu, Jiabo Ye, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, and Jitao Sang. 2024. Mobile-agent: Autonomous multi-modal mobile device agent with visual perception. arXiv preprint arXiv:2401.16158. — [arXiv:2401.16158](https://arxiv.org/abs/2401.16158) find
  61. Zhenting Wang, Qi Chang, Hemani Patel, Shashank Biju, Cheng-En Wu, Quan Liu, Aolin Ding, Alireza Rezazadeh, Ankit Shah, Yujia Bao, and 1 others. 2025b. Mcp-bench: Benchmarking tool-using llm agents with complex real-world tasks via mcp servers. arXiv preprint arXiv:2508.20453. — [arXiv:2508.20453](https://arxiv.org/abs/2508.20453) find
  62. Shaohang Wei, Wei Li, Feifan Song, Wen Luo, Tianyi Zhuang, Haochen Tan, Zhijiang Guo, and Houfeng Wang. 2025. TIME: A multi-level benchmark for temporal reasoning of llms in real-world scenarios. arXiv preprint arXiv:2505.12891. find
  63. Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, and 1 others. 2024. Autogen: Enabling next-gen llm applications via multi-agent conversations. In First Conference on Language Modeling. find
  64. Zijian Wu, Xiangyan Liu, Xinyuan Zhang, Lingjun Chen, Fanqing Meng, Lingxiao Du, Yiran Zhao, Fanshi Zhang, Yaoqi Ye, Jiawei Wang, and 1 others. 2025. Mcpmark: A benchmark for stress-testing realistic and comprehensive mcp use. arXiv preprint arXiv:2509.24002. — [arXiv:2509.24002](https://arxiv.org/abs/2509.24002) find
  65. Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. 2024a. Travelplanner: A benchmark for real-world planning with language agents. arXiv preprint arXiv:2402.01622. find
  66. Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh J Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, and 1 others. 2024b. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments. Advances in Neural Information Processing Systems, 3…
  67. Jie Zhang, Shuai Dong, Shiguang Shan, and Xilin Chen. 2025a. Dual attention guided defense against malicious edits. ArXiv, abs/2512.14333. — [arXiv:2512.14333](https://arxiv.org/abs/2512.14333) find
  68. Yunyan Zhang, Zhihong Zhu, and Xian Wu. 2025b. Cmedcalc-bench: A fine-grained benchmark for chinese medical calculations in llm. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 25661–25670. find
  69. Zhixia Zhang, Zixuan Huang, Xin Xia, Deqing Wang, Fuzhen Zhuang, Shuai Ma, Ning Ding, Yaodong Yang, Jianxin Li, and Yikun Ban. 2026. Heterogeneous agent collaborative reinforcement learning. arXiv preprint arXiv:2603.02604. — [arXiv:2603.02604](https://arxiv.org/abs/2603.02604) find
  70. Boyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun, and Yu Su. 2024a. Gpt-4v (ision) is a generalist web agent, if grounded. arXiv preprint arXiv:2401.01614.
  71. Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Livia Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, and 1 others. 2024b. Sglang: Efficient execution of structured language model programs. Advances in neural information processing systems, 37:62557– 6…
  72. Yakun Zhu, Shaohang Wei, Xu Wang, Kui Xue, Shaoting Zhang, and Xiaofan Zhang. 2025. Menti: Bridging medical calculator and llm agent with nested tool calling. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language… find
  73. John Yang, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. Swe-agent: Agent computer interfaces enable software engineering language models. arXiv preprint arXiv.2405.15793. find
  74. Yingxuan Yang, Huacan Chai, Yuanyi Song, Siyuan Qi, Muning Wen, Ning Li, Junwei Liao, Haoyi Hu, Jianghao Lin, Gaowei Chang, and 1 others. 2025. A survey of ai agent protocols. arXiv preprint arXiv:2504.16736. find
  75. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. In The eleventh international conference on learning representations.
  76. Hongmei Zhang, Songshi Liang, Luke A Matkovic, Shadab Momin, Kai Wang, Xiaofeng Yang, and Michael F Insana. 2023. Deep q-learning to globally optimize a kd parameter search for medical imaging. Quantitative Imaging in Medicine and Surgery, 13(8):4879. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhu-2026-medmcp-calc-benchmarking-llms-realistic/