FMI-SU at SemEval-2023 Task 7: Two-level Entailment Classification of Clinical Trials Enhanced by Contextual Data Augmentation

Sylvia Vassileva, Georgi Grazhdanski, Svetla Boytcheva, Ivan Koychev


Abstract
The paper presents an approach for solving SemEval 2023 Task 7 - identifying the inference relation in a clinical trials dataset. The system has two levels for retrieving relevant clinical trial evidence for a statement and then classifying the inference relation based on the relevant sentences. In the first level, the system classifies the evidence-statement pairs as relevant or not using a BERT-based classifier and contextual data augmentation (subtask 2). Using the relevant parts of the clinical trial from the first level, the system uses an additional BERT-based classifier to determine whether the relation is entailment or contradiction (subtask 1). In both levels, the contextual data augmentation is showing a significant improvement in the F1 score on the test set of 3.7% for subtask 2 and 7.6% for subtask 1, achieving final F1 scores of 82.7% for subtask 2 and 64.4% for subtask 1.
Anthology ID:
2023.semeval-1.200
Volume:
Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Atul Kr. Ojha, A. Seza Doğruöz, Giovanni Da San Martino, Harish Tayyar Madabushi, Ritesh Kumar, Elisa Sartori
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
1454–1462
Language:
URL:
https://aclanthology.org/2023.semeval-1.200
DOI:
10.18653/v1/2023.semeval-1.200
Bibkey:
Cite (ACL):
Sylvia Vassileva, Georgi Grazhdanski, Svetla Boytcheva, and Ivan Koychev. 2023. FMI-SU at SemEval-2023 Task 7: Two-level Entailment Classification of Clinical Trials Enhanced by Contextual Data Augmentation. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1454–1462, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
FMI-SU at SemEval-2023 Task 7: Two-level Entailment Classification of Clinical Trials Enhanced by Contextual Data Augmentation (Vassileva et al., SemEval 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-1/2023.semeval-1.200.pdf
Video:
 https://preview.aclanthology.org/nschneid-patch-1/2023.semeval-1.200.mp4