Few-Shot Anaphora Resolution in Scientific Protocols via Mixtures of In-Context Experts

Nghia T. Le, Fan Bai, Alan Ritter


Abstract
Anaphora resolution is an important task for information extraction across a range of languages, text genres, and domains, motivating the need for methods that do not require large annotated datasets. In-context learning has emerged as a promising approach, yet there are a number of challenges in applying in-context learning to resolve anaphora. For example, encoding a single in-context demonstration that consists of: an anaphor, a paragraph-length context, and a list of corresponding antecedents, requires conditioning a language model on a long sequence of tokens, limiting the number of demonstrations per prompt.In this paper, we present Mice (Mixtures of In-Context Experts), which we demonstrate is effective for few-shot anaphora resolution in scientific protocols. Given only a handful of training examples, Mice combines the predictions of hundreds of in-context experts, yielding a 30% increase in F1 score over a competitive prompt retrieval baseline. Furthermore, we show Mice can be used to train compact student models without sacrificing performance. As far as we are aware, this is the first work to present experimental results demonstrating the effectiveness of in-context learning on the task of few-shot anaphora resolution in scientific protocols.
Anthology ID:
2022.findings-emnlp.197
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2022
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2693–2706
Language:
URL:
https://aclanthology.org/2022.findings-emnlp.197
DOI:
Bibkey:
Cite (ACL):
Nghia T. Le, Fan Bai, and Alan Ritter. 2022. Few-Shot Anaphora Resolution in Scientific Protocols via Mixtures of In-Context Experts. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 2693–2706, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Cite (Informal):
Few-Shot Anaphora Resolution in Scientific Protocols via Mixtures of In-Context Experts (Le et al., Findings 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2022.findings-emnlp.197.pdf