Real World Conversational Entity Linking Requires More Than Zero-Shots
Mohanna Hoveyda, Arjen Vries, Faegheh Hasibi, Maarten de Rijke
Abstract
Entity linking (EL) in conversations faces notable challenges in practical applications, primarily due to scarcity of entity-annotated conversational datasets and sparse knowledge bases (KB) containing domain-specific, long-tail entities. We designed targeted evaluation scenarios to measure the efficacy of EL models under resource constraints. Our evaluation employs two KBs: Fandom, exemplifying real-world EL complexities, and the widely used Wikipedia. First, we assess EL models’ ability to generalize to a new unfamiliar KB using Fandom and a novel zero-shot conversational entity linking dataset that we curated based on Reddit discussions on Fandom entities. We then evaluate the adaptability of EL models to conversational settings without prior training. Our results indicate that current zero-shot EL models falter when introduced to new, domain-specific KBs without prior training, significantly dropping in performance.Our findings reveal that previous evaluation approaches fall short of capturing real-world complexities for zero-shot EL, highlighting the necessity for new approaches to design and assess conversational EL models to adapt to limited resources. The evaluation frame-work and dataset proposed are tailored to facilitate this research.- Anthology ID:
- 2024.findings-acl.829
- Volume:
- Findings of the Association for Computational Linguistics: ACL 2024
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Lun-Wei Ku, Andre Martins, Vivek Srikumar
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 13938–13946
- Language:
- URL:
- https://aclanthology.org/2024.findings-acl.829
- DOI:
- 10.18653/v1/2024.findings-acl.829
- Cite (ACL):
- Mohanna Hoveyda, Arjen Vries, Faegheh Hasibi, and Maarten de Rijke. 2024. Real World Conversational Entity Linking Requires More Than Zero-Shots. In Findings of the Association for Computational Linguistics: ACL 2024, pages 13938–13946, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- Real World Conversational Entity Linking Requires More Than Zero-Shots (Hoveyda et al., Findings 2024)
- PDF:
- https://preview.aclanthology.org/dois-2013-emnlp/2024.findings-acl.829.pdf