WangLab at MEDIQA-CORR 2024: Optimized LLM-based Programs for Medical Error Detection and Correction
Augustin Toma, Ronald Xie, Steven Palayew, Patrick Lawler, Bo Wang
Abstract
Medical errors in clinical text pose significant risks to patient safety. The MEDIQA-CORR 2024 shared task focuses on detecting and correcting these errors across three subtasks: identifying the presence of an error, extracting the erroneous sentence, and generating a corrected sentence. In this paper, we present our approach that achieved top performance in all three subtasks. For the MS dataset, which contains subtle errors, we developed a retrieval-based system leveraging external medical question-answering datasets. For the UW dataset, reflecting more realistic clinical notes, we created a pipeline of modules to detect, localize, and correct errors. Both approaches utilized the DSPy framework for optimizing prompts and few-shot examples in large language model (LLM) based programs. Our results demonstrate the effectiveness of LLM based programs for medical error correction. However, our approach has limitations in addressing the full diversity of potential errors in medical documentation. We discuss the implications of our work and highlight future research directions to advance the robustness and applicability of medical error detection and correction systems.- Anthology ID:
- 2024.clinicalnlp-1.59
- Volume:
- Proceedings of the 6th Clinical Natural Language Processing Workshop
- Month:
- June
- Year:
- 2024
- Address:
- Mexico City, Mexico
- Editors:
- Tristan Naumann, Asma Ben Abacha, Steven Bethard, Kirk Roberts, Danielle Bitterman
- Venues:
- ClinicalNLP | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 616–623
- Language:
- URL:
- https://aclanthology.org/2024.clinicalnlp-1.59
- DOI:
- Cite (ACL):
- Augustin Toma, Ronald Xie, Steven Palayew, Patrick Lawler, and Bo Wang. 2024. WangLab at MEDIQA-CORR 2024: Optimized LLM-based Programs for Medical Error Detection and Correction. In Proceedings of the 6th Clinical Natural Language Processing Workshop, pages 616–623, Mexico City, Mexico. Association for Computational Linguistics.
- Cite (Informal):
- WangLab at MEDIQA-CORR 2024: Optimized LLM-based Programs for Medical Error Detection and Correction (Toma et al., ClinicalNLP-WS 2024)
- PDF:
- https://preview.aclanthology.org/jeptaln-2024-ingestion/2024.clinicalnlp-1.59.pdf