Converting Annotated Clinical Cases into Structured Case Report Forms

Pietro Ferrazzi, Alberto Lavelli, Bernardo Magnini


Abstract
Case Report Forms (CRFs) are largely used in medical research as they ensure accuracy, reliability, and validity of results in clinical studies. However, publicly available, well-annotated CRF datasets are scarce, limiting the development of CRF slot filling systems able to fill in a CRF from clinical notes. To mitigate the scarcity of CRF datasets, we propose to take advantage of available datasets annotated for information extraction tasks and to convert them into structured CRFs. We present a semi-automatic conversion methodology, which has been applied to the E3C dataset in two languages (English and Italian), resulting in a new, high-quality dataset for CRF slot filling. Through several experiments on the created dataset, we report that slot filling achieves 59.7% for Italian and 67.3% for English on a closed Large Language Models (zero-shot) and worse performances on three families of open-source models, showing that filling CRFs is challenging even for recent state-of-the-art LLMs.
Anthology ID:
2025.bionlp-1.26
Volume:
ACL 2025
Month:
August
Year:
2025
Address:
Viena, Austria
Editors:
Dina Demner-Fushman, Sophia Ananiadou, Makoto Miwa, Junichi Tsujii
Venues:
BioNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
307–318
Language:
URL:
https://preview.aclanthology.org/acl25-workshop-ingestion/2025.bionlp-1.26/
DOI:
Bibkey:
Cite (ACL):
Pietro Ferrazzi, Alberto Lavelli, and Bernardo Magnini. 2025. Converting Annotated Clinical Cases into Structured Case Report Forms. In ACL 2025, pages 307–318, Viena, Austria. Association for Computational Linguistics.
Cite (Informal):
Converting Annotated Clinical Cases into Structured Case Report Forms (Ferrazzi et al., BioNLP 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/acl25-workshop-ingestion/2025.bionlp-1.26.pdf
Supplementarymaterial:
 2025.bionlp-1.26.SupplementaryMaterial.zip
Supplementarymaterial:
 2025.bionlp-1.26.SupplementaryMaterial.txt