A Transformational Biencoder with In-Domain Negative Sampling for Zero-Shot Entity Linking
Kai Sun, Richong Zhang, Samuel Mensah, Yongyi Mao, Xudong Liu
Abstract
Recent interest in entity linking has focused in the zero-shot scenario, where at test time the entity mention to be labelled is never seen during training, or may belong to a different domain from the source domain. Current work leverage pre-trained BERT with the implicit assumption that it bridges the gap between the source and target domain distributions. However, fine-tuned BERT has a considerable underperformance at zero-shot when applied in a different domain. We solve this problem by proposing a Transformational Biencoder that incorporates a transformation into BERT to perform a zero-shot transfer from the source domain during training. As like previous work, we rely on negative entities to encourage our model to discriminate the golden entities during training. To generate these negative entities, we propose a simple but effective strategy that takes the domain of the golden entity into perspective. Our experimental results on the benchmark dataset Zeshel show effectiveness of our approach and achieve new state-of-the-art.- Anthology ID:
- 2022.findings-acl.114
- Volume:
- Findings of the Association for Computational Linguistics: ACL 2022
- Month:
- May
- Year:
- 2022
- Address:
- Dublin, Ireland
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1449–1458
- Language:
- URL:
- https://aclanthology.org/2022.findings-acl.114
- DOI:
- 10.18653/v1/2022.findings-acl.114
- Cite (ACL):
- Kai Sun, Richong Zhang, Samuel Mensah, Yongyi Mao, and Xudong Liu. 2022. A Transformational Biencoder with In-Domain Negative Sampling for Zero-Shot Entity Linking. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1449–1458, Dublin, Ireland. Association for Computational Linguistics.
- Cite (Informal):
- A Transformational Biencoder with In-Domain Negative Sampling for Zero-Shot Entity Linking (Sun et al., Findings 2022)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2022.findings-acl.114.pdf
- Data
- ZESHEL