Abstract
Domain adaptation or transfer learning using pre-trained language models such as BERT has proven to be an effective approach for many natural language processing tasks. In this work, we propose to formulate word sense disambiguation as a relevance ranking task, and fine-tune BERT on sequence-pair ranking task to select the most probable sense definition given a context sentence and a list of candidate sense definitions. We also introduce a data augmentation technique for WSD using existing example sentences from WordNet. Using the proposed training objective and data augmentation technique, our models are able to achieve state-of-the-art results on the English all-words benchmark datasets.- Anthology ID:
- 2020.findings-emnlp.4
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2020
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Editors:
- Trevor Cohn, Yulan He, Yang Liu
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 41–46
- Language:
- URL:
- https://aclanthology.org/2020.findings-emnlp.4
- DOI:
- 10.18653/v1/2020.findings-emnlp.4
- Cite (ACL):
- Boon Peng Yap, Andrew Koh, and Eng Siong Chng. 2020. Adapting BERT for Word Sense Disambiguation with Gloss Selection Objective and Example Sentences. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 41–46, Online. Association for Computational Linguistics.
- Cite (Informal):
- Adapting BERT for Word Sense Disambiguation with Gloss Selection Objective and Example Sentences (Yap et al., Findings 2020)
- PDF:
- https://preview.aclanthology.org/naacl24-info/2020.findings-emnlp.4.pdf
- Code
- BPYap/BERT-WSD
- Data
- Word Sense Disambiguation: a Unified Evaluation Framework and Empirical Comparison