Abstract
Recent approaches have explored language- guided classifiers capable of classifying examples from novel tasks when provided with task-specific natural language explanations, instructions or prompts (Sanh et al., 2022; R. Menon et al., 2022). While these classifiers can generalize in zero-shot settings, their task performance often varies substantially between different language explanations in unpredictable ways (Lu et al., 2022; Gonen et al., 2022). Also, current approaches fail to leverage unlabeled examples that may be available in many scenarios. Here, we introduce TALC, a framework that uses data programming to adapt a language-guided classifier for a new task during inference when provided with explanations from multiple teachers and unlabeled test examples. Our results show that TALC consistently outperforms a competitive baseline from prior work by an impressive 9.3% (relative improvement). Further, we demonstrate the robustness of TALC to variations in the quality and quantity of provided explanations, highlighting its potential in scenarios where learning from multiple teachers or a crowd is involved. Our code is available at: https://github.com/WeiKangda/TALC.git.- Anthology ID:
- 2023.findings-emnlp.471
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2023
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 7068–7088
- Language:
- URL:
- https://aclanthology.org/2023.findings-emnlp.471
- DOI:
- 10.18653/v1/2023.findings-emnlp.471
- Cite (ACL):
- Kangda Wei, Sayan Ghosh, Rakesh Menon, and Shashank Srivastava. 2023. Leveraging Multiple Teachers for Test-Time Adaptation of Language-Guided Classifiers. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7068–7088, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Leveraging Multiple Teachers for Test-Time Adaptation of Language-Guided Classifiers (Wei et al., Findings 2023)
- PDF:
- https://preview.aclanthology.org/dois-2013-emnlp/2023.findings-emnlp.471.pdf