SemEval-2017 Task 12: Clinical TempEval
Steven Bethard, Guergana Savova, Martha Palmer, James Pustejovsky
Abstract
Clinical TempEval 2017 aimed to answer the question: how well do systems trained on annotated timelines for one medical condition (colon cancer) perform in predicting timelines on another medical condition (brain cancer)? Nine sub-tasks were included, covering problems in time expression identification, event expression identification and temporal relation identification. Participant systems were evaluated on clinical and pathology notes from Mayo Clinic cancer patients, annotated with an extension of TimeML for the clinical domain. 11 teams participated in the tasks, with the best systems achieving F1 scores above 0.55 for time expressions, above 0.70 for event expressions, and above 0.40 for temporal relations. Most tasks observed about a 20 point drop over Clinical TempEval 2016, where systems were trained and evaluated on the same domain (colon cancer).- Anthology ID:
- S17-2093
- Volume:
- Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)
- Month:
- August
- Year:
- 2017
- Address:
- Vancouver, Canada
- Editors:
- Steven Bethard, Marine Carpuat, Marianna Apidianaki, Saif M. Mohammad, Daniel Cer, David Jurgens
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 565–572
- Language:
- URL:
- https://aclanthology.org/S17-2093
- DOI:
- 10.18653/v1/S17-2093
- Cite (ACL):
- Steven Bethard, Guergana Savova, Martha Palmer, and James Pustejovsky. 2017. SemEval-2017 Task 12: Clinical TempEval. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 565–572, Vancouver, Canada. Association for Computational Linguistics.
- Cite (Informal):
- SemEval-2017 Task 12: Clinical TempEval (Bethard et al., SemEval 2017)
- PDF:
- https://preview.aclanthology.org/naacl24-info/S17-2093.pdf