Importance of Prompt Optimisation for Error Detection in Medical Notes Using Language Models

Craig Myles, Patrick Schrempf, David Harris-Birtill


Abstract
Errors in medical text can cause delays or even result in incorrect treatment for patients. Recently, language models have shown promise in their ability to automatically detect errors in medical text, an ability that has the opportunity to significantly benefit healthcare systems. In this paper, we explore the importance of prompt optimisation for small and large language models when applied to the task of error detection. We perform rigorous experiments and analysis across frontier language models and open-source language models. We show that automatic prompt optimisation with Genetic-Pareto (GEPA) improves error detection over the baseline accuracy performance from 0.669 to 0.785 with GPT-5 and 0.578 to 0.690 with Qwen3-32B, approaching the performance of medical doctors and achieving state-of-the-art performance on the MEDEC benchmark dataset. Code available on GitHub: https://github.com/CraigMyles/clinical-note-error-detection
Anthology ID:
2026.healing-1.20
Volume:
Proceedings of the 1st Workshop on Linguistic Analysis for Health (HeaLing 2026)
Month:
March
Year:
2026
Address:
Rabat, Morocco
Editors:
Vera Danilova, Murathan Kurfalı, Ylva Söderfeldt, Julia Reed, Andrew Burchell
Venues:
HeaLing | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
236–248
Language:
URL:
https://preview.aclanthology.org/ingest-eacl/2026.healing-1.20/
DOI:
Bibkey:
Cite (ACL):
Craig Myles, Patrick Schrempf, and David Harris-Birtill. 2026. Importance of Prompt Optimisation for Error Detection in Medical Notes Using Language Models. In Proceedings of the 1st Workshop on Linguistic Analysis for Health (HeaLing 2026), pages 236–248, Rabat, Morocco. Association for Computational Linguistics.
Cite (Informal):
Importance of Prompt Optimisation for Error Detection in Medical Notes Using Language Models (Myles et al., HeaLing 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-eacl/2026.healing-1.20.pdf