Abstract
This paper describes the DuluthNLP system that participated in Task 7 of SemEval-2022 on Identifying Plausible Clarifications of Implicit and Underspecified Phrases in Instructional Texts. Given an instructional text with an omitted token, the task requires models to classify or rank the plausibility of potential fillers. To solve the task, we fine–tuned the models BERT, RoBERTa, and ELECTRA on training data where potential fillers are rated for plausibility. This is a challenging problem, as shown by BERT-based models achieving accuracy less than 45%. However, our ELECTRA model with tuned class weights on CrossEntropyLoss achieves an accuracy of 53.3% on the official evaluation test data, which ranks 6 out of the 8 total submissions for Subtask A.- Anthology ID:
- 2022.semeval-1.149
- Volume:
- Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)
- Month:
- July
- Year:
- 2022
- Address:
- Seattle, United States
- Venue:
- SemEval
- SIGs:
- SIGLEX | SIGSEM
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1062–1066
- Language:
- URL:
- https://aclanthology.org/2022.semeval-1.149
- DOI:
- 10.18653/v1/2022.semeval-1.149
- Cite (ACL):
- Samuel Akrah and Ted Pedersen. 2022. DuluthNLP at SemEval-2022 Task 7: Classifying Plausible Alternatives with Pre–trained ELECTRA. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1062–1066, Seattle, United States. Association for Computational Linguistics.
- Cite (Informal):
- DuluthNLP at SemEval-2022 Task 7: Classifying Plausible Alternatives with Pre–trained ELECTRA (Akrah & Pedersen, SemEval 2022)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2022.semeval-1.149.pdf