Annotate the Way You Think: An Incremental Note Generation Framework for the Summarization of Medical Conversations
Longxiang Zhang, Caleb D. Hart, Susanne Burger, Thomas Schaaf
Abstract
The scarcity of public datasets for the summarization of medical conversations has been a limiting factor for advancing NLP research in the healthcare domain, and the structure of the existing data is largely limited to the simple format of conversation-summary pairs. We therefore propose a novel Incremental Note Generation (ING) annotation framework capable of greatly enriching summarization datasets in the healthcare domain and beyond. Our framework is designed to capture the human summarization process via an annotation task by instructing the annotators to first incrementally create a draft note as they accumulate information through a conversation transcript (Generation) and then polish the draft note into a reference note (Rewriting). The annotation results include both the reference note and a comprehensive editing history of the draft note in tabular format. Our pilot study on the task of SOAP note generation showed reasonable consistency between four expert annotators, established a solid baseline for quantitative targets of inter-rater agreement, and demonstrated the ING framework as an improvement over the traditional annotation process for future modeling of summarization.- Anthology ID:
- 2024.lrec-main.105
- Volume:
- Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
- Month:
- May
- Year:
- 2024
- Address:
- Torino, Italia
- Editors:
- Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
- Venues:
- LREC | COLING
- SIG:
- Publisher:
- ELRA and ICCL
- Note:
- Pages:
- 1173–1186
- Language:
- URL:
- https://aclanthology.org/2024.lrec-main.105
- DOI:
- Cite (ACL):
- Longxiang Zhang, Caleb D. Hart, Susanne Burger, and Thomas Schaaf. 2024. Annotate the Way You Think: An Incremental Note Generation Framework for the Summarization of Medical Conversations. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 1173–1186, Torino, Italia. ELRA and ICCL.
- Cite (Informal):
- Annotate the Way You Think: An Incremental Note Generation Framework for the Summarization of Medical Conversations (Zhang et al., LREC-COLING 2024)
- PDF:
- https://preview.aclanthology.org/naacl-24-ws-corrections/2024.lrec-main.105.pdf