Mind Your Steps in Biomedical Named Entity Recognition: First Extract, Tag Afterwards
Paper · 2026 · Darya Shlyk, Stefano Montanelli, Marco Mesiti et al.
Summary
Biomedical named entity recognition is difficult for few-shot prompted LLMs. The domain has vast vocabularies, terms that belong to several classes, and ambiguous local abbreviations, so these models still trail supervised, fine-tuned systems. The authors propose FETA (First Extract, Tag Afterwards), a two-call prompt chain for off-the-shelf LLMs that requires no domain-specific training.
In the first call, the model lists candidate mentions. It is steered mainly by expert-written annotation guidelines rather than by demonstrations. In the second call, the model marks every candidate inside the original passage with HTML-style type tags. Candidates assigned a type outside the target set are discarded. This step lets the model re-check its own predictions using only the surrounding context. It needs no knowledge bases or entity linking and handles all candidates in a single pass.
Evaluated on several established biomedical benchmarks with commercial and open-source LLMs, FETA reports state-of-the-art results that surpass fine-tuned BioNER models. The work shows that careful prompting can make general LLMs useful where labelled biomedical data is scarce.
Key points
-
Ablation, average F1 gains over the base setup:
- Type disambiguation: +3.5 (exact match) and +3.8 (relaxed match).
- Annotation guidelines: +1.8 and +2.3.
- Both combined: +5.3 and +5.9.
- Guidelines mainly raise recall, while the tagging step mainly raises precision.
-
A prompt with guidelines and a single example beats a prompt with five examples and no guidelines.
-
Comparison with other verification methods (exact-match F1):
Method BC5CDR GENIA FETA 79.2 58.0 VerifiNER 61.9 55.5 GPT-NER 59.5 48.3 -
Verification costs one LLM call per passage (O(1)). VerifiNER and GPT-NER check entities one at a time, so their cost grows linearly (O(n)) with the number of predicted entities.
-
The moderately sized open model Gemma 2 performs on par with commercial models of similar size and can rival the larger LLaMA 3.3.
Institutions
United States45 works · 50,617 citations🎓University of MilanMilan · 
Italy6 works · 38,727 citationsFields, topics and concepts
📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksBiochemistry, Genetics and Molecular Biology · 234 worksMolecular Biology · 231 works
🧭 Topics 1Research subjects and languages the work is about.Named entity recognition · 200 works
Cite
Cite as Darya Shlyk, Stefano Montanelli, Marco Mesiti and Lawrence Hunter. 2026. Mind Your Steps in Biomedical Named Entity Recognition: First Extract, Tag Afterwards. Proceedings of the 1st Workshop on Linguistic Analysis for Health (HeaLing 2026).
@inproceedings{shlyk2026mind,
title = {{Mind Your Steps in Biomedical Named Entity Recognition: First Extract, Tag Afterwards}},
author = {Darya Shlyk, Stefano Montanelli, Marco Mesiti and Lawrence Hunter},
booktitle = {Proceedings of the 1st Workshop on Linguistic Analysis for Health (HeaLing 2026)},
year = {2026},
doi = {10.18653/v1/2026.healing-1.11},
url = {https://doi.org/10.18653/v1/2026.healing-1.11}
}References
- Monica Agrawal, Stefan Hegselmann, Hunter Lang, Yoon Kim, and David A. Sontag. 2022. Large language models are few-shot clinical information extractors. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-…
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/papers/shlyk-2026-mind-your-steps-biomedical-named/