MULTISEM at SemEval-2020 Task 3: Fine-tuning BERT for Lexical Meaning

Aina Garí Soler, Marianna Apidianaki


Abstract
We present the MULTISEM systems submitted to SemEval 2020 Task 3: Graded Word Similarity in Context (GWSC). We experiment with injecting semantic knowledge into pre-trained BERT models through fine-tuning on lexical semantic tasks related to GWSC. We use existing semantically annotated datasets, and propose to approximate similarity through automatically generated lexical substitutes in context. We participate in both GWSC subtasks and address two languages, English and Finnish. Our best English models occupy the third and fourth positions in the ranking for the two subtasks. Performance is lower for the Finnish models which are mid-ranked in the respective subtasks, highlighting the important role of data availability for fine-tuning.
Anthology ID:
2020.semeval-1.18
Volume:
Proceedings of the Fourteenth Workshop on Semantic Evaluation
Month:
December
Year:
2020
Address:
Barcelona (online)
Venues:
COLING | SemEval
SIGs:
SIGLEX | SIGSEM
Publisher:
International Committee for Computational Linguistics
Note:
Pages:
158–165
Language:
URL:
https://aclanthology.org/2020.semeval-1.18
DOI:
10.18653/v1/2020.semeval-1.18
Bibkey:
Cite (ACL):
Aina Garí Soler and Marianna Apidianaki. 2020. MULTISEM at SemEval-2020 Task 3: Fine-tuning BERT for Lexical Meaning. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 158–165, Barcelona (online). International Committee for Computational Linguistics.
Cite (Informal):
MULTISEM at SemEval-2020 Task 3: Fine-tuning BERT for Lexical Meaning (Garí Soler & Apidianaki, SemEval 2020)
Copy Citation:
PDF:
https://preview.aclanthology.org/update-css-js/2020.semeval-1.18.pdf
Code
 ainagari/semeval2020-task3-multisem
Data
GLUEOpusparcusWiC