Ensemble-based Fine-Tuning Strategy for Temporal Relation Extraction from the Clinical Narrative
Lijing Wang, Timothy Miller, Steven Bethard, Guergana Savova
Abstract
In this paper, we investigate ensemble methods for fine-tuning transformer-based pretrained models for clinical natural language processing tasks, specifically temporal relation extraction from the clinical narrative. Our experimental results on the THYME data show that ensembling as a fine-tuning strategy can further boost model performance over single learners optimized for hyperparameters. Dynamic snapshot ensembling is particularly beneficial as it fine-tunes a wide array of parameters and results in a 2.8% absolute improvement in F1 over the base single learner.- Anthology ID:
- 2022.clinicalnlp-1.11
- Volume:
- Proceedings of the 4th Clinical Natural Language Processing Workshop
- Month:
- July
- Year:
- 2022
- Address:
- Seattle, WA
- Editors:
- Tristan Naumann, Steven Bethard, Kirk Roberts, Anna Rumshisky
- Venue:
- ClinicalNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 103–108
- Language:
- URL:
- https://aclanthology.org/2022.clinicalnlp-1.11
- DOI:
- 10.18653/v1/2022.clinicalnlp-1.11
- Cite (ACL):
- Lijing Wang, Timothy Miller, Steven Bethard, and Guergana Savova. 2022. Ensemble-based Fine-Tuning Strategy for Temporal Relation Extraction from the Clinical Narrative. In Proceedings of the 4th Clinical Natural Language Processing Workshop, pages 103–108, Seattle, WA. Association for Computational Linguistics.
- Cite (Informal):
- Ensemble-based Fine-Tuning Strategy for Temporal Relation Extraction from the Clinical Narrative (Wang et al., ClinicalNLP 2022)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/2022.clinicalnlp-1.11.pdf