Abstract
This paper describes the system of the Cambridge team submitted to the SemEval-2021 shared task on Multilingual and Cross-lingual Word-in-Context Disambiguation. Building on top of a pre-trained masked language model, our system is first pre-trained on out-of-domain data, and then fine-tuned on in-domain data. We demonstrate the effectiveness of the proposed two-step training strategy and the benefits of data augmentation from both existing examples and new resources. We further investigate different representations and show that the addition of distance-based features is helpful in the word-in-context disambiguation task. Our system yields highly competitive results in the cross-lingual track without training on any cross-lingual data; and achieves state-of-the-art results in the multilingual track, ranking first in two languages (Arabic and Russian) and second in French out of 171 submitted systems.- Anthology ID:
- 2021.semeval-1.96
- Volume:
- Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021)
- Month:
- August
- Year:
- 2021
- Address:
- Online
- Editors:
- Alexis Palmer, Nathan Schneider, Natalie Schluter, Guy Emerson, Aurelie Herbelot, Xiaodan Zhu
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 730–737
- Language:
- URL:
- https://aclanthology.org/2021.semeval-1.96
- DOI:
- 10.18653/v1/2021.semeval-1.96
- Cite (ACL):
- Zheng Yuan and David Strohmaier. 2021. Cambridge at SemEval-2021 Task 2: Neural WiC-Model with Data Augmentation and Exploration of Representation. In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 730–737, Online. Association for Computational Linguistics.
- Cite (Informal):
- Cambridge at SemEval-2021 Task 2: Neural WiC-Model with Data Augmentation and Exploration of Representation (Yuan & Strohmaier, SemEval 2021)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2021.semeval-1.96.pdf
- Data
- WiC, XL-WiC