Mind Your Steps in Biomedical Named Entity Recognition: First Extract, Tag Afterwards
Darya Shlyk, Stefano Montanelli, Marco Mesiti, Lawrence Hunter
Abstract
Few-shot prompting with Large Language Models (LLMs) has emerged as a promising paradigm for advancing information extraction, particularly in data-scarce domains like biomedicine, where high annotation costs constrain the availability of training data.However, challenges persist in biomedical Named Entity Recognition (NER), where LLMs fail to achieve necessary accuracy and lag behind supervised fine-tuned models. In this study, we introduce FETA (First Extract, Tag Afterwards), a two-stage approach for entity recognition that combines instruction-guided prompting and a novel self-verification strategy to improve accuracy and reliability of LLM predictions in domain-specific NER tasks. FETA achieves state-of-the-art results on multiple established biomedical datasets.Our experiments demonstrate that carefully designed prompts, using self-verification and instruction guidance, can steer general-purpose LLMs to outperform fine-tuned models in knowledge-intensive NER tasks, unlocking their potential for more reliable and accurate information extraction in resource-constrained settings.- Anthology ID:
- 2026.healing-1.11
- Volume:
- Proceedings of the 1st Workshop on Linguistic Analysis for Health (HeaLing 2026)
- Month:
- March
- Year:
- 2026
- Address:
- Rabat, Morocco
- Editors:
- Vera Danilova, Murathan Kurfalı, Ylva Söderfeldt, Julia Reed, Andrew Burchell
- Venues:
- HeaLing | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 127–141
- Language:
- URL:
- https://preview.aclanthology.org/ingest-eacl/2026.healing-1.11/
- DOI:
- Cite (ACL):
- Darya Shlyk, Stefano Montanelli, Marco Mesiti, and Lawrence Hunter. 2026. Mind Your Steps in Biomedical Named Entity Recognition: First Extract, Tag Afterwards. In Proceedings of the 1st Workshop on Linguistic Analysis for Health (HeaLing 2026), pages 127–141, Rabat, Morocco. Association for Computational Linguistics.
- Cite (Informal):
- Mind Your Steps in Biomedical Named Entity Recognition: First Extract, Tag Afterwards (Shlyk et al., HeaLing 2026)
- PDF:
- https://preview.aclanthology.org/ingest-eacl/2026.healing-1.11.pdf