On the Cross-lingual Transferability of Contextualized Sense Embeddings
Kiamehr Rezaee, Daniel Loureiro, Jose Camacho-Collados, Mohammad Taher Pilehvar
Abstract
In this paper we analyze the extent to which contextualized sense embeddings, i.e., sense embeddings that are computed based on contextualized word embeddings, are transferable across languages. To this end, we compiled a unified cross-lingual benchmark for Word Sense Disambiguation. We then propose two simple strategies to transfer sense-specific knowledge across languages and test them on the benchmark. Experimental results show that this contextualized knowledge can be effectively transferred to similar languages through pre-trained multilingual language models, to the extent that they can out-perform monolingual representations learnednfrom existing language-specific data.- Anthology ID:
- 2021.mrl-1.10
- Volume:
- Proceedings of the 1st Workshop on Multilingual Representation Learning
- Month:
- November
- Year:
- 2021
- Address:
- Punta Cana, Dominican Republic
- Editors:
- Duygu Ataman, Alexandra Birch, Alexis Conneau, Orhan Firat, Sebastian Ruder, Gozde Gul Sahin
- Venue:
- MRL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 107–115
- Language:
- URL:
- https://aclanthology.org/2021.mrl-1.10
- DOI:
- 10.18653/v1/2021.mrl-1.10
- Cite (ACL):
- Kiamehr Rezaee, Daniel Loureiro, Jose Camacho-Collados, and Mohammad Taher Pilehvar. 2021. On the Cross-lingual Transferability of Contextualized Sense Embeddings. In Proceedings of the 1st Workshop on Multilingual Representation Learning, pages 107–115, Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- On the Cross-lingual Transferability of Contextualized Sense Embeddings (Rezaee et al., MRL 2021)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/2021.mrl-1.10.pdf
- Data
- Word Sense Disambiguation: a Unified Evaluation Framework and Empirical Comparison