Abstract
Annotated clinical text corpora are essential for machine learning studies that model and predict care processes and disease progression. However, few studies describe the necessary experimental design of the annotation guideline and annotation phases. This makes replication, reuse, and adoption challenging.Using clinical questions about sepsis, we designed a semantic annotation guideline to capture sepsis signs from clinical text. The clinical questions aid guideline design, application, and evaluation. Our method incrementally evaluates each change in the guideline by testing the resulting annotated corpus using clinical questions. Additionally, our method uses inter-annotator agreement to judge the annotator compliance and quality of the guideline. We show that the method, combined with controlled design increments, is simple and allows the development and measurable improvement of a purpose-built semantic annotation guideline. We believe that our approach is useful for incremental design of semantic annotation guidelines in general.- Anthology ID:
- 2023.clinicalnlp-1.29
- Volume:
- Proceedings of the 5th Clinical Natural Language Processing Workshop
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Venue:
- ClinicalNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 236–246
- Language:
- URL:
- https://aclanthology.org/2023.clinicalnlp-1.29
- DOI:
- Cite (ACL):
- Melissa Yan, Lise Gustad, Lise Høvik, and Øystein Nytrø. 2023. Method for Designing Semantic Annotation of Sepsis Signs in Clinical Text. In Proceedings of the 5th Clinical Natural Language Processing Workshop, pages 236–246, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Method for Designing Semantic Annotation of Sepsis Signs in Clinical Text (Yan et al., ClinicalNLP 2023)
- PDF:
- https://preview.aclanthology.org/paclic-22-ingestion/2023.clinicalnlp-1.29.pdf