Elamur Research⌕
☰
home / papers / 2026 / Do Clinical Question Answering Systems Really Need Specialised Medical Fine Tuning?

Do Clinical Question Answering Systems Really Need Specialised Medical Fine Tuning?

Paper · 2026 · Sushant Kumar Ray, Gautam Siddharth Kashyap, Sahil Tripathi et al.

Summary

The paper questions a common assumption in industrial clinical question answering (CQA): that a model must be fine-tuned on medical data to perform well. The authors call this belief the SPECIALISATION FALLACY. They argue that specialised models such as BioBERT, BioGPT and PubMedBERT cover narrow domains, are expensive to retrain, and break down outside their training scope.

As an alternative, they propose MEDASSESS-X, which aligns a model during inference instead of through supervised fine-tuning (SFT). The method builds steering vectors from the difference in hidden activations between correct and incorrect clinical answers. These vectors are added to the model's hidden states, and the weights are never changed. There are separate vectors for triage, literature and patient-guidance questions, and a small classifier decides which one to apply.

The authors test the method on general-purpose, encoder–decoder and medical LLMs, using an expert-validated TRUE/FALSE medical QA set. Every model family improves, and the computational cost stays small. This suggests that an inference-time layer could replace costly domain retraining in deployed systems.

Key points

  • Dataset: the CQA dataset of Azeez et al. (2025), with 1,077 expert-reviewed TRUE/FALSE questions on consumer health, clinical knowledge and anatomy. It is split 80/20 into 861 training and 216 test items.
  • Backbones tested:
    • Decoder-only: Gemma-3-27B, Llama-3-8B-Instruct, Mistral-7B-Instruct-v0.3, DeepSeek-7B.
    • Encoder–decoder: T5-Large, Flan-T5-XL.
    • Medical: BioBERT, PubMedBERT, BioGPT.
  • Reported gains: Accuracy up to +6%, Factual Consistency up to +7%, and Safety Error Rate cut by up to about half. For patient-facing questions, SER falls from 0.24 to 0.11.
  • Overhead: latency rises by roughly 7–9%, memory by at most 6%, and FLOPs by under 8%, measured on an NVIDIA A100 80GB.
  • Ablation and tuning:
    • Swapping the contrastive vectors for random directions gives almost no benefit over the base model.
    • A single global vector leaves SER noticeably higher than task-specific vectors.
    • Steering strength α = 1.0 works best.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Stanford UniversityStanford · United States434 works · 314,419 citations🎓Cornell UniversityIthaca · United States120 works · 59,940 citations🎓Macquarie UniversitySydney · Australia42 works · 5,776 citations🎓University of DelhiNew Delhi · India5 works · 15 citations🏢Expedia GroupBellevue · United States2 works · 66 citations

Also on the byline: Jamia Hamdard.

 Fields, topics and concepts

📚 Shelves 8The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksBiochemistry, Genetics and Molecular Biology · 234 worksMolecular Biology · 231 worksMedicine · 241 worksHealth Informatics · 184 works

🧭 Topics 1Research subjects and languages the work is about.Question answering · 441 works

Cite

Cite as Sushant Kumar Ray, Gautam Siddharth Kashyap, Sahil Tripathi, Nipun Joshi, Vijay Govindarajan, Rafiq Ali, Jiechao Gao and Usman Naseem. 2026. Do Clinical Question Answering Systems Really Need Specialised Medical Fine Tuning?. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-industry.64

@inproceedings{ray2026do,
  title = {{Do Clinical Question Answering Systems Really Need Specialised Medical Fine Tuning?}},
  author = {Sushant Kumar Ray, Gautam Siddharth Kashyap, Sahil Tripathi, Nipun Joshi, Vijay Govindarajan, Rafiq Ali, Jiechao Gao and Usman Naseem},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-industry.64},
  url = {https://doi.org/10.18653/v1/2026.eacl-industry.64}
}

References

28 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Basel Alshaikhdeeb and Kamsuriah Ahmad. 2016. Biomedical named entity recognition: a review. International Journal on Advanced Science, Engineering and Information Technology, 6(6):889–895. find
  2. DM Anisuzzaman, Jeffrey G Malins, Paul A Friedman, and Zachi I Attia. 2025. Fine-tuning large language models for specialized use cases. Mayo Clinic Proceedings: Digital Health, 3(1):100184. find
  3. Mohammad Anas Azeez, Rafiq Ali, Ebad Shabbir, Zohaib Hasan Siddiqui, Gautam Siddharth Kashyap, Jiechao Gao, and Usman Naseem. 2025. Truth, trust, and trouble: Medical AI on the edge. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 1017… find
  4. Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021. Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1–23.
  5. Yao He, Xuanbing Zhu, Donghan Li, and Hongyu Wang. 2025. Enhancing large language models for specialized domains: A two-stage framework with parameter-sensitive lora fine-tuning and chain-of-thought rag. Electronics, 14(10):1961. find
  6. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019. Pubmedqa: A dataset for biomedical research question answering. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language proce…
  7. Gautam Siddharth Kashyap, Mark Dras, and Usman Naseem. 2025. We think, therefore we align llms to helpful, harmless and honest before they go wrong. arXiv preprint arXiv:2509.22510. find
  8. Chenqian Le, Ziheng Gong, Chihang Wang, Haowei Ni, Panfeng Li, and Xupeng Chen. 2025. Instruction tuning and cot prompting for contextual medical qa with llms. arXiv preprint arXiv:2506.12182. find
  9. Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234–1240.
  10. Shuyue Stella Li, Jimin Mun, Faeze Brahman, Jonathan S Ilgen, Yulia Tsvetkov, and Maarten Sap. 2025. Aligning llms to ask good questions a case study in clinical reasoning. arXiv preprint arXiv:2502.14860. — [arXiv:2502.14860](https://arxiv.org/abs/2502.14860) find
  11. Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie-Yan Liu. 2022. find
  12. Biogpt: generative pre-trained transformer for biomedical text generation and mining. Briefings in bioinformatics, 23(6):bbac409. find
  13. Subhankar Maity and Manob Jyoti Saikia. 2025. Large language models in healthcare and medical applications: A review. Bioengineering, 12(6):631. find
  14. Utsav Maskey, ZHU Chencheng, and Usman Naseem. 2025. Benchmarking large language models for cryptanalysis and side-channel vulnerabilities. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 19849–19865. find
  15. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in gpt. Advances in neural information processing systems, 35:17359–17372. find
  16. Afrozah Nadeem, Mark Dras, and Usman Naseem. 2025. Context-aware fairness evaluation and mitigation in llms. arXiv preprint arXiv:2510.18914. — [arXiv:2510.18914](https://arxiv.org/abs/2510.18914) find
  17. Usman Naseem, Gautam Siddharth Kashyap, Kaixuan Ren, Yiran Zhang, Utsav Maskey, Juan Ren, and Afrozah Nadeem. 2025. Alignment of large language models with human preferences and values. In Proceedings of the 23rd Annual Workshop of the Australasian Language Technology Association, pages 245–245. find
  18. Dandan Wang and Shiqing Zhang. 2024. Large language models in medical and healthcare fields: applications, advances, and challenges. Artificial intelligence review, 57(11):299. find
  19. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171.
  20. Paul Youssef, Zhixue Zhao, Christin Seifert, and Jörg Schlötterer. 2025. Has this fact been edited? detecting knowledge edits in language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies… find
  21. Yiran Zhang, Jincheng Hu, Mark Dras, and Usman Naseem. 2025a. Cogmem: A cognitive memory architecture for sustained multi-turn reasoning in large language models. arXiv preprint arXiv:2512.14118. — [arXiv:2512.14118](https://arxiv.org/abs/2512.14118) find
  22. Yiran Zhang, Mingyang Lin, Mark Dras, and Usman Naseem. 2025b. Beyond the black box: Demystifying multi-turn llm reasoning with vista. arXiv preprint arXiv:2511.10182. — [arXiv:2511.10182](https://arxiv.org/abs/2511.10182) find
  23. Zabir Al Nazi and Wei Peng. 2024. Large language models in healthcare and medical domain: A review. In Informatics, volume 11, page 57. MDPI. find
  24. Benjamin Nye, Junyi Jessy Li, Roma Patel, Yinfei Yang, Iain Marshall, Ani Nenkova, and Byron C Wallace. 2018. A corpus with multi-level annotations of patients, interventions and outcomes to support language processing for medical literature. In Proceedings of the 56th annual meeting of the associat…
  25. Juan Ren, Mark Dras, and Usman Naseem. 2025. Shield: Classifier-guided prompting for robust and safer lvlms. In Proceedings of the 23rd Annual Workshop of the Australasian Language Technology Association, pages 76–89. find
  26. Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. 2024. A systematic survey of prompt engineering in large language models: Techniques and applications. arXiv preprint arXiv:2402.07927.
  27. Sina Shool, Sara Adimi, Reza Saboori Amleshi, Ehsan Bitaraf, Reza Golpira, and Mahmood Tara. 2025. A systematic review of large language model (llm) evaluations in clinical medicine. BMC Medical Informatics and Decision Making, 25(1):117. find
  28. Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, and 1 others. 2023. Large language models encode clinical knowledge. Nature, 620(7972):172–180.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ray-2026-do-clinical-question-answering-systems/