Abstract
We present our entry to the Multi-evidence Natural Language Inference for Clinical Trial Datatask at SemEval 2023. We submitted entries forboth the evidence retrieval and textual entailment sub-tasks. For the evidence retrieval task,we fine-tuned the PubMedBERT transformermodel to extract relevant evidence from clinicaltrial data given a hypothesis concerning either asingle clinical trial or pair of clinical trials. Ourbest performing model achieved an F1 scoreof 0.804. For the textual entailment task, inwhich systems had to predict whether a hypothesis about either a single clinical trial or pair ofclinical trials is true or false, we fine-tuned theBioLinkBERT transformer model. We passedour evidence retrieval model’s output into ourtextual entailment model and submitted its output for the evaluation. Our best performingmodel achieved an F1 score of 0.695.- Anthology ID:
- 2023.semeval-1.179
- Volume:
- Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Atul Kr. Ojha, A. Seza Doğruöz, Giovanni Da San Martino, Harish Tayyar Madabushi, Ritesh Kumar, Elisa Sartori
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1287–1292
- Language:
- URL:
- https://aclanthology.org/2023.semeval-1.179
- DOI:
- 10.18653/v1/2023.semeval-1.179
- Cite (ACL):
- Robert Bevan, Oisín Turbitt, and Mouhamad Aboshokor. 2023. MDC at SemEval-2023 Task 7: Fine-tuning Transformers for Textual Entailment Prediction and Evidence Retrieval in Clinical Trials. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1287–1292, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- MDC at SemEval-2023 Task 7: Fine-tuning Transformers for Textual Entailment Prediction and Evidence Retrieval in Clinical Trials (Bevan et al., SemEval 2023)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/2023.semeval-1.179.pdf