EchoGen: Generating Conclusions from Echocardiogram Notes
Liyan Tang, Shravan Kooragayalu, Yanshan Wang, Ying Ding, Greg Durrett, Justin F. Rousseau, Yifan Peng
Abstract
Generating a summary from findings has been recently explored (Zhang et al., 2018, 2020) in note types such as radiology reports that typically have short length. In this work, we focus on echocardiogram notes that is longer and more complex compared to previous note types. We formally define the task of echocardiography conclusion generation (EchoGen) as generating a conclusion given the findings section, with emphasis on key cardiac findings. To promote the development of EchoGen methods, we present a new benchmark, which consists of two datasets collected from two hospitals. We further compare both standard and start-of-the-art methods on this new benchmark, with an emphasis on factual consistency. To accomplish this, we develop a tool to automatically extract concept-attribute tuples from the text. We then propose an evaluation metric, FactComp, to compare concept-attribute tuples between the human reference and generated conclusions. Both automatic and human evaluations show that there is still a significant gap between human-written and machine-generated conclusions on echo reports in terms of factuality and overall quality.- Anthology ID:
- 2022.bionlp-1.35
- Volume:
- Proceedings of the 21st Workshop on Biomedical Language Processing
- Month:
- May
- Year:
- 2022
- Address:
- Dublin, Ireland
- Editors:
- Dina Demner-Fushman, Kevin Bretonnel Cohen, Sophia Ananiadou, Junichi Tsujii
- Venue:
- BioNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 359–368
- Language:
- URL:
- https://aclanthology.org/2022.bionlp-1.35
- DOI:
- 10.18653/v1/2022.bionlp-1.35
- Cite (ACL):
- Liyan Tang, Shravan Kooragayalu, Yanshan Wang, Ying Ding, Greg Durrett, Justin F. Rousseau, and Yifan Peng. 2022. EchoGen: Generating Conclusions from Echocardiogram Notes. In Proceedings of the 21st Workshop on Biomedical Language Processing, pages 359–368, Dublin, Ireland. Association for Computational Linguistics.
- Cite (Informal):
- EchoGen: Generating Conclusions from Echocardiogram Notes (Tang et al., BioNLP 2022)
- PDF:
- https://preview.aclanthology.org/improve-issue-templates/2022.bionlp-1.35.pdf
- Data
- MIMIC-III