A Survey of Evaluation Methods of Generated Medical Textual Reports

Yongxin Zhou, Fabien Ringeval, François Portet

Abstract
Medical Report Generation (MRG) is a sub-task of Natural Language Generation (NLG) and aims to present information from various sources in textual form and synthesize salient information, with the goal of reducing the time spent by domain experts in writing medical reports and providing support information for decision-making. Given the specificity of the medical domain, the evaluation of automatically generated medical reports is of paramount importance to the validity of these systems. Therefore, in this paper, we focus on the evaluation of automatically generated medical reports from the perspective of automatic and human evaluation. We present evaluation methods for general NLG evaluation and how they have been applied to domain-specific medical tasks. The study shows that MRG evaluation methods are very diverse, and that further work is needed to build shared evaluation methods. The state of the art also emphasizes that such an evaluation must be task specific and include human assessments, requesting the participation of experts in the field.
Anthology ID:
2023.clinicalnlp-1.48
Volume:
Proceedings of the 5th Clinical Natural Language Processing Workshop
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Tristan Naumann, Asma Ben Abacha, Steven Bethard, Kirk Roberts, Anna Rumshisky
Venue:
ClinicalNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
447–459
Language:
URL:
https://aclanthology.org/2023.clinicalnlp-1.48
DOI:
10.18653/v1/2023.clinicalnlp-1.48
Bibkey:
Cite (ACL):
Yongxin Zhou, Fabien Ringeval, and François Portet. 2023. A Survey of Evaluation Methods of Generated Medical Textual Reports. In Proceedings of the 5th Clinical Natural Language Processing Workshop, pages 447–459, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
A Survey of Evaluation Methods of Generated Medical Textual Reports (Zhou et al., ClinicalNLP 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-4/2023.clinicalnlp-1.48.pdf
PDF Search