Elamur Research⌕
☰
home / papers / 2026 / TAGS: A Test-Time Generalist–Specialist Framework with Retrieval-Augmented Reasoning and Verification

TAGS: A Test-Time Generalist–Specialist Framework with Retrieval-Augmented Reasoning and Verification

Paper · 2026 · Jianghao Wu, Feilong Tang, Yulong Li et al.

Summary

The paper targets a weakness of LLMs in medical question answering. Prompting tricks such as Chain-of-Thought tend to give shallow or unstable reasoning, and fine-tuned medical models struggle when the data distribution shifts. The authors propose TAGS, which runs purely at inference time and changes no model weights. A generalist agent and a domain specialist agent each reason over the question in two rounds, producing four candidate rationale–answer pairs. Two helper components support them:

  • Hierarchical retrieval: exemplars are first fetched from a frozen medical-QA corpus by question similarity, then re-selected by similarity to the agents' draft rationales.
  • Reasoning consistency evaluator: an LLM-based verifier scores each candidate, and the highest-scoring answer is chosen.

Across nine MedQA benchmarks, TAGS beats prompting, retrieval and multi-agent baselines on GPT-4o, DeepSeek-R1 and a small 7B model. It sometimes outperforms fine-tuned medical LLMs. This matters because it offers a training-free way to make domain QA more robust when fine-tuning is impractical. Evaluation covers only English multiple-choice tasks.

Key points

  • Headline gains: accuracy rises by 13.8% for GPT-4o and 16.8% for DeepSeek-R1. A vanilla 7B model improves from 14.1% to 23.9%.
  • Retrieval setup: it uses M3-Embedding (1024-dim vectors) with cosine similarity. The default is K = 2 exemplars, and larger K adds noise.
  • Specialist count: adding one specialist to the generalist lifts MMLU from 34.2% to 45.2% and MedXpert-U from 16.0% to 22.0%. Further specialists bring little extra benefit.
  • Robustness checks: results stay competitive when the top-10 nearest exemplars are excluded (RAG-w/o-top10). They also hold with the small m1k corpus of 1,000 CoT exemplars.
  • Cost: with GPT-4o on MedQA, TAGS takes about 72 s per question, compared with 27.7 s for CoT-SC and 109.6 s for Multi-Persona. Code is released at github.com/JianghaoWu/TAGS.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Monash UniversityMelbourne · Australia60 works · 54,866 citations🎓Mohamed bin Zayed University of Artificial IntelligenceAbu Dhabi · United Arab Emirates124 works · 5,676 citations🎓University of SouthamptonSouthampton · United Kingdom10 works · 2,495 citations🎓Xi’an Jiaotong-Liverpool UniversitySuzhou · China10 works · 319 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksMedicine · 241 worksHealth Informatics · 184 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksInformation retrieval & search · 473 works

Cite

Cite as Jianghao Wu, Feilong Tang, Yulong Li, Ming Hu, Haochen Xue, Shoaib Jameel, Zongyuan Ge, Yutong Xie and Imran Razzak. 2026. TAGS: A Test-Time Generalist–Specialist Framework with Retrieval-Augmented Reasoning and Verification. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.757

@inproceedings{wu2026tags,
  title = {{TAGS: A Test-Time Generalist–Specialist Framework with Retrieval-Augmented Reasoning and Verification}},
  author = {Jianghao Wu, Feilong Tang, Yulong Li, Ming Hu, Haochen Xue, Shoaib Jameel, Zongyuan Ge, Yutong Xie and Imran Razzak},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.757},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.757}
}

References

46 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mert Cemri, Melissa Z Pan, Shuyi Yang, Lakshya A Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, et al. 2025. Why do multi-agent llm systems fail? arXiv preprint arXiv:2503.13657. find
  2. Payal Chandak, Kexin Huang, and Marinka Zitnik. 2023. Building a knowledge graph to enable precision medicine. Scientific Data, 10(1):67.
  3. Hanjie Chen, Zhouxiang Fang, Yash Singla, and Mark Dredze. 2024a. Benchmarking large language models on answering and explaining challenging medical questions. arXiv preprint arXiv:2402.18060.
  4. Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024b. M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 2318–2335.
  5. Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, Jianye Hou, and Benyou Wang. 2024c. Huatuogpt-o1, towards medical complex reasoning with llms. arXiv preprint arXiv:2412.18925. find
  6. Qingyu Chen, Jingcheng Du, Yan Hu, Vipina Kuttichi Keloth, Xueqing Peng, Kalpana Raja, Rui Zhang, Zhiyong Lu, and Hua Xu. 2023. Large language models in biomedical natural language processing: benchmarks, baselines, and recommendations. arXiv e-prints, pages arXiv–2305. find
  7. Xi Chen, Huahui Yi, Mingke You, WeiZhi Liu, Li Wang, Hairui Li, Xue Zhang, Yingman Guo, Lei Fan, Gang Chen, et al. 2025. Enhancing diagnostic capability with multi-agents conversational large language models. NPJ digital medicine, 8(1):159. find
  8. Xuhang Chen, Shenghong Luo, Chi-Man Pun, and Shuqiang Wang. 2024d. MedPrompt: Cross-modal prompting for multi-task medical image translation. In Chinese Conference on Pattern Recognition and Computer Vision (PRCV), pages 61–75. Springer. find
  9. Zhihao Fan, Lai Wei, Jialong Tang, Wei Chen, Wang Siyuan, Zhongyu Wei, and Fei Huang. 2025. Ai hospital: Benchmarking large language models in a multi-agent medical interaction simulator. In Proceedings of the 31st International Conference on Computational Linguistics, pages 10183–10213.
  10. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  11. Qiangqiang He, Shuwei Qian, Jie Zhang, and Chongjun Wang. 2025. Inference retrieval-augmented multimodal chain-of-thoughts reasoning for language models. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. find
  12. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2024. Self-Refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36. find
  13. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300.
  14. Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. 2022. MedMCQA: A large-scale multi-subject multi-choice dataset for medical domain question answering. In Conference on health, inference, and learning, pages 248–260. PMLR.
  15. Xiaoke Huang, Juncheng Wu, Hui Liu, Xianfeng Tang, and Yuyin Zhou. 2025. m1: Unleash the potential of test-time scaling for medical reasoning with large language models. arXiv preprint arXiv:2504.00869. — [arXiv:2504.00869](https://arxiv.org/abs/2504.00869) find
  16. Malaikannan Sankarasubbu Ankit Pal and Malaikannan Sankarasubbu. 2024. Openbiollms: Advancing open-source large language models for healthcare and life sciences. Hugging Face repository. find
  17. Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2023. Atlas: Few-shot learning with retrieval augmented language models. Journal of Machine Learning Research, 24(251):1–43.
  18. Pengcheng Qiu, Chaoyi Wu, Xiaoman Zhang, Weixiong Lin, Haicheng Wang, Ya Zhang, Yanfeng Wang, and Weidi Xie. 2024. Towards building multilingual language model for medicine. Nature Communications, 15(1):8384.
  19. Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2021. What disease does this patient have? A large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):6421.
  20. Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2023. Replug: Retrieval-augmented black-box language models. arXiv preprint arXiv:2301.12652.
  21. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W Cohen, and Xinghua Lu. 2019. PubMedQA: A dataset for biomedical research question answering. arXiv preprint arXiv:1909.06146.
  22. Wenqi Shi, Ran Xu, Yuchen Zhuang, Yue Yu, Haotian Sun, Hang Wu, Carl Yang, and May D Wang. 2024. Medadapter: Efficient test-time adaptation of large language models towards medical reasoning. arXiv preprint arXiv:2405.03000. find
  23. Qiao Jin, Zheng Yuan, Guangzhi Xiong, Qianlan Yu, Huaiyuan Ying, Chuanqi Tan, Mosha Chen, Songfang Huang, Xiaozhong Liu, and Sheng Yu. 2022. Biomedical question answering: a survey of approaches and challenges. ACM Computing Surveys (CSUR), 55(2):1–36. find
  24. Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R Pfohl, Heather Cole-Lewis, et al. 2025. Toward expert-level medical question answering with large language models. Nature Medicine, pages 1–8. find
  25. Yubin Kim, Chanwoo Park, Hyewon Jeong, Yik Siu Chan, Xuhai Xu, Daniel McDuff, Hyeonhoon Lee, Marzyeh Ghassemi, Cynthia Breazeal, and Hae Won Park. 2024a. MDAgents: An adaptive collaboration of llms in medical decision making. arXiv preprint arXiv:2404.15155. — [arXiv:2404.15155](https://arxiv.org/abs/2404.15155) find
  26. Xiangru Tang, Daniel Shao, Jiwoong Sohn, Jiapeng Chen, Jiayi Zhang, Jinyu Xiang, Fang Wu, Yilun Zhao, Chenglin Wu, Wenqi Shi, et al. 2025. Medagentsbench: Benchmarking thinking models and agent frameworks for complex medical reasoning. arXiv preprint arXiv:2503.07459. — [arXiv:2503.07459](https://arxiv.org/abs/2503.07459) find
  27. Yunsoo Kim, Jinge Wu, Yusuf Abdulle, and Honghan Wu. 2024b. MedExQA: Medical question answering benchmark with multiple explanations. arXiv preprint arXiv:2406.06331. find
  28. Xiangru Tang, Anni Zou, Zhuosheng Zhang, Ziming Li, Yilun Zhao, Xingyao Zhang, Arman Cohan, and Mark Gerstein. 2023. MedAgents: Large language models as collaborators for zero-shot medical reasoning. arXiv preprint arXiv:2311.10537.
  29. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33:9459–9474.
  30. Jiaxiang Liu, Yuan Wang, Jiawei Du, Joey Tianyi Zhou, and Zuozhu Liu. 2024. Medcot: Medical chain of thought via hierarchical expert. arXiv preprint arXiv:2412.13736. find
  31. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations.
  32. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. 2024. MMLU-Pro: A more robust and challenging multi-task language understanding benchmark. arXiv preprint arXiv:2406.01574.
  33. Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. 2023. Unleashing the emergent cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration. arXiv preprint arXiv:2307.05300.
  34. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837.
  35. Jianghao Wu, Yasmeen George, Jin Ye, Yicheng Wu, Daniel F Schmidt, and Jianfei Cai. 2025a. Spine: Token-selective test-time reinforcement learning with entropy-band regularization. arXiv preprint arXiv:2511.17938. — [arXiv:2511.17938](https://arxiv.org/abs/2511.17938) find
  36. Jianghao Wu, Xinya Liu, Guotai Wang, and Shaoting Zhang. 2026. Sictta: Single image continual test time adaptation for medical image segmentation. Medical Image Analysis, 108:103859. find
  37. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024a. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115. find
  38. Zhaorui Yang, Tianyu Pang, Haozhe Feng, Han Wang, Wei Chen, Minfeng Zhu, and Qian Liu. 2024b. Self-distillation bridges distribution gap in language model fine-tuning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, pages 1028–1043.
  39. Hongjian Zhou, Fenglin Liu, Boyang Gu, Xinyu Zou, Jinfa Huang, Jinge Wu, Yiru Li, Sam S Chen, Peilin Zhou, Junling Liu, et al. 2023. A survey of large language models in medicine: Progress, application, and challenge. arXiv preprint arXiv:2311.05112.
  40. Yuxin Zuo, Shang Qu, Yifei Li, Zhangren Chen, Xuekai Zhu, Ermo Hua, Kaiyan Zhang, Ning Ding, and Bowen Zhou. 2025. MedXpertQA: Benchmarking expert-level medical reasoning and understanding. arXiv preprint arXiv:2501.18362. find
  41. Jianghao Wu, Xiangde Luo, Yubo Zhou, Lianming Wu, Guotai Wang, and Shaoting Zhang. 2025b. A3-tta: Adaptive anchor alignment test-time adaptation for image segmentation. IEEE Transactions on Image Processing, 34:8511–8522. find
  42. Juncheng Wu, Wenlong Deng, Xingxuan Li, Sheng Liu, Taomian Mi, Yifan Peng, Ziyang Xu, Yi Liu, Hyunjin Cho, Chang-In Choi, et al. 2025c. Medreason: Eliciting factual medical reasoning steps in llms via knowledge graphs. arXiv preprint arXiv:2504.00993. — [arXiv:2504.00993](https://arxiv.org/abs/2504.00993) find
  43. Xiao Wu, Ting-Zhu Huang, Liang-Jian Deng, Yanyuan Qiao, Imran Razzak, and Yutong Xie. 2025d. A knowledge-driven adaptive collaboration of llms for enhancing medical decision-making. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 33483–33500. find
  44. Xiao Wu, Ting-Zhu Huang, Liang-Jian Deng, Xiaobing Yu, Yu Zhong, Shangqi Deng, Ufaq Khan, Jianghao Wu, Xiaofeng Liu, Imran Razzak, et al. 2025e. Selfai: Building a self-training ai system with llm agents. arXiv preprint arXiv:2512.00403. — [arXiv:2512.00403](https://arxiv.org/abs/2512.00403) find
  45. Mengdi Xu, Yikang Shen, Shun Zhang, Yuchen Lu, Ding Zhao, Joshua Tenenbaum, and Chuang Gan. 2022. Prompting decision transformer for few-shot policy generalization. In international conference on machine learning, pages 24631–24645. PMLR. find
  46. Shaochen Xu, Yifan Zhou, Zhengliang Liu, Zihao Wu, Tianyang Zhong, Huaqin Zhao, Yiwei Li, Hanqi Jiang, Yi Pan, Junhao Chen, et al. 2024. Towards next-generation medical agent: How o1 is reshaping decision-making in medical scenarios. arXiv preprint arXiv:2411.14461. — [arXiv:2411.14461](https://arxiv.org/abs/2411.14461) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2026-tags-test-time-generalist-specialist/