SemEval-2017 Task 12: Clinical TempEval

Steven Bethard, Guergana Savova, Martha Palmer, James Pustejovsky


Abstract
Clinical TempEval 2017 aimed to answer the question: how well do systems trained on annotated timelines for one medical condition (colon cancer) perform in predicting timelines on another medical condition (brain cancer)? Nine sub-tasks were included, covering problems in time expression identification, event expression identification and temporal relation identification. Participant systems were evaluated on clinical and pathology notes from Mayo Clinic cancer patients, annotated with an extension of TimeML for the clinical domain. 11 teams participated in the tasks, with the best systems achieving F1 scores above 0.55 for time expressions, above 0.70 for event expressions, and above 0.40 for temporal relations. Most tasks observed about a 20 point drop over Clinical TempEval 2016, where systems were trained and evaluated on the same domain (colon cancer).
Anthology ID:
S17-2093
Volume:
Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)
Month:
August
Year:
2017
Address:
Vancouver, Canada
Venue:
SemEval
SIGs:
SIGLEX | SIGSEM
Publisher:
Association for Computational Linguistics
Note:
Pages:
565–572
Language:
URL:
https://aclanthology.org/S17-2093
DOI:
10.18653/v1/S17-2093
Bibkey:
Cite (ACL):
Steven Bethard, Guergana Savova, Martha Palmer, and James Pustejovsky. 2017. SemEval-2017 Task 12: Clinical TempEval. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 565–572, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):
SemEval-2017 Task 12: Clinical TempEval (Bethard et al., SemEval 2017)
Copy Citation:
PDF:
https://preview.aclanthology.org/update-css-js/S17-2093.pdf