Abstract
This paper presents our system descriptions for SemEval 2023-Task 7: Multi-evidence Natural Language Inference for Clinical Trial Data sub-tasks one and two. Provided with a collection of Clinical Trial Reports (CTRs) and corresponding expert-annotated claim statements, sub-task one involves determining an inferential relationship between the statement and CTR premise: contradiction or entailment. Sub-task two involves retrieving evidence from the CTR which is necessary to determine the entailment in sub-task one. For sub-task two we employ a recent transformer-based language model pretrained on biomedical literature, which we domain-adapt on a set of clinical trial reports. For sub-task one, we take an ensemble approach in which we leverage the evidence retrieval model from sub-task two to extract relevant sections, which are then passed to a second model of equivalent architecture to determine entailment. Our system achieves a ranking of seventh on sub-task one with an F1-score of 0.705 and sixth on sub-task two with an F1-score of 0.806. In addition, we find that the high rate of success of language models on this dataset may be partially attributable to the existence of annotation artifacts.- Anthology ID:
- 2023.semeval-1.220
- Volume:
- Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1598–1602
- Language:
- URL:
- https://aclanthology.org/2023.semeval-1.220
- DOI:
- 10.18653/v1/2023.semeval-1.220
- Cite (ACL):
- Ahamed Alameldin and Ashton Williamson. 2023. Clemson NLP at SemEval-2023 Task 7: Applying GatorTron to Multi-Evidence Clinical NLI. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1598–1602, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Clemson NLP at SemEval-2023 Task 7: Applying GatorTron to Multi-Evidence Clinical NLI (Alameldin & Williamson, SemEval 2023)
- PDF:
- https://preview.aclanthology.org/remove-xml-comments/2023.semeval-1.220.pdf