Mind Your Steps in Biomedical Named Entity Recognition: First Extract, Tag Afterwards

Darya Shlyk, Stefano Montanelli, Marco Mesiti, Lawrence Hunter


Abstract
Few-shot prompting with Large Language Models (LLMs) has emerged as a promising paradigm for advancing information extraction, particularly in data-scarce domains like biomedicine, where high annotation costs constrain the availability of training data.However, challenges persist in biomedical Named Entity Recognition (NER), where LLMs fail to achieve necessary accuracy and lag behind supervised fine-tuned models. In this study, we introduce FETA (First Extract, Tag Afterwards), a two-stage approach for entity recognition that combines instruction-guided prompting and a novel self-verification strategy to improve accuracy and reliability of LLM predictions in domain-specific NER tasks. FETA achieves state-of-the-art results on multiple established biomedical datasets.Our experiments demonstrate that carefully designed prompts, using self-verification and instruction guidance, can steer general-purpose LLMs to outperform fine-tuned models in knowledge-intensive NER tasks, unlocking their potential for more reliable and accurate information extraction in resource-constrained settings.
Anthology ID:
2026.healing-1.11
Volume:
Proceedings of the 1st Workshop on Linguistic Analysis for Health (HeaLing 2026)
Month:
March
Year:
2026
Address:
Rabat, Morocco
Editors:
Vera Danilova, Murathan Kurfalı, Ylva Söderfeldt, Julia Reed, Andrew Burchell
Venues:
HeaLing | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
127–141
Language:
URL:
https://preview.aclanthology.org/ingest-eacl/2026.healing-1.11/
DOI:
Bibkey:
Cite (ACL):
Darya Shlyk, Stefano Montanelli, Marco Mesiti, and Lawrence Hunter. 2026. Mind Your Steps in Biomedical Named Entity Recognition: First Extract, Tag Afterwards. In Proceedings of the 1st Workshop on Linguistic Analysis for Health (HeaLing 2026), pages 127–141, Rabat, Morocco. Association for Computational Linguistics.
Cite (Informal):
Mind Your Steps in Biomedical Named Entity Recognition: First Extract, Tag Afterwards (Shlyk et al., HeaLing 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-eacl/2026.healing-1.11.pdf