HEVS-TUW at SemEval-2023 Task 8: Ensemble of Language Models and Rule-based Classifiers for Claims Identification and PICO Extraction
Anjani Dhrangadhariya, Wojciech Kusa, Henning Müller, Allan Hanbury
Abstract
This paper describes the HEVS-TUW team submission to the SemEval-2023 Task 8: Causal Claims. We participated in two subtasks: (1) causal claims detection and (2) PIO identification. For subtask 1, we experimented with an ensemble of weakly supervised question detection and fine-tuned Transformer-based models. For subtask 2 of PIO frame extraction, we used a combination of deep representation learning and a rule-based approach. Our best model for subtask 1 ranks fourth with an F1-score of 65.77%. It shows moderate benefit from ensembling models pre-trained on independent categories. The results for subtask 2 warrant further investigation for improvement.- Anthology ID:
- 2023.semeval-1.246
- Volume:
- Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Atul Kr. Ojha, A. Seza Doğruöz, Giovanni Da San Martino, Harish Tayyar Madabushi, Ritesh Kumar, Elisa Sartori
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1776–1782
- Language:
- URL:
- https://aclanthology.org/2023.semeval-1.246
- DOI:
- 10.18653/v1/2023.semeval-1.246
- Cite (ACL):
- Anjani Dhrangadhariya, Wojciech Kusa, Henning Müller, and Allan Hanbury. 2023. HEVS-TUW at SemEval-2023 Task 8: Ensemble of Language Models and Rule-based Classifiers for Claims Identification and PICO Extraction. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1776–1782, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- HEVS-TUW at SemEval-2023 Task 8: Ensemble of Language Models and Rule-based Classifiers for Claims Identification and PICO Extraction (Dhrangadhariya et al., SemEval 2023)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-3/2023.semeval-1.246.pdf