Abstract
In recent years, few-shot models have been applied successfully to a variety of NLP tasks. Han et al. (2018) introduced a few-shot learning framework for relation classification, and since then, several models have surpassed human performance on this task, leading to the impression that few-shot relation classification is solved. In this paper we take a deeper look at the efficacy of strong few-shot classification models in the more common relation extraction setting, and show that typical few-shot evaluation metrics obscure a wide variability in performance across relations. In particular, we find that state of the art few-shot relation classification models overly rely on entity type information, and propose modifications to the training routine to encourage models to better discriminate between relations involving similar entity types.- Anthology ID:
- 2021.emnlp-main.433
- Volume:
- Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
- Month:
- November
- Year:
- 2021
- Address:
- Online and Punta Cana, Dominican Republic
- Editors:
- Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 5338–5345
- Language:
- URL:
- https://aclanthology.org/2021.emnlp-main.433
- DOI:
- 10.18653/v1/2021.emnlp-main.433
- Cite (ACL):
- Sam Brody, Sichao Wu, and Adrian Benton. 2021. Towards Realistic Few-Shot Relation Extraction. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5338–5345, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- Towards Realistic Few-Shot Relation Extraction (Brody et al., EMNLP 2021)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/2021.emnlp-main.433.pdf
- Code
- bloomberg/emnlp21_fewrel
- Data
- FewRel, FewRel 2.0