SubmissionNumber#=%=#54
FinalPaperTitle#=%=#Converting Annotated Clinical Cases into Structured Case Report Forms
ShortPaperTitle#=%=#
NumberOfPages#=%=#12
CopyrightSigned#=%=#Pietro Ferrazzi
JobTitle#==#
Organization#==#Fondazione Bruno Kessler, Via delle sommarive 18, Povo, Trento, Italy,
Abstract#==#Case Report Forms (CRFs) are largely used in medical research as they ensure accuracy, reliability, and validity of results in clinical studies. However, publicly available, well-annotated CRF datasets are scarce, limiting the development of CRF slot filling systems able to fill in a CRF from clinical notes. To mitigate the scarcity of CRF datasets, we propose to take advantage of available datasets annotated for information extraction tasks and to convert them into structured CRFs. We present a semi-automatic conversion methodology, which has been applied to the E3C dataset in two languages (English and Italian), resulting in a new, high-quality dataset for CRF slot filling. Through several experiments on the created dataset, we report that slot filling achieves 59.7% for Italian and 67.3% for English on a closed Large Language Models (zero-shot) and worse performances on three families of open-source models, showing that filling CRFs is challenging even for recent state-of-the-art LLMs.
Author{1}{Firstname}#=%=#Pietro
Author{1}{Lastname}#=%=#Ferrazzi
Author{1}{Username}#=%=#ferrazzipietro
Author{1}{Email}#=%=#ferrazzipietro@gmail.com
Author{1}{Affiliation}#=%=#University of Padova
Author{2}{Firstname}#=%=#Alberto
Author{2}{Lastname}#=%=#Lavelli
Author{2}{Username}#=%=#lavelli
Author{2}{Email}#=%=#lavelli@fbk.eu
Author{2}{Affiliation}#=%=#FBK
Author{3}{Firstname}#=%=#Bernardo
Author{3}{Lastname}#=%=#Magnini
Author{3}{Username}#=%=#magnini
Author{3}{Email}#=%=#magnini@fbk.eu
Author{3}{Affiliation}#=%=#FBK

==========
èéáğö