Abstract
We study the problem of building text classifiers with little or no training data, commonly known as zero and few-shot text classification. In recent years, an approach based on neural textual entailment models has been found to give strong results on a diverse range of tasks. In this work, we show that with proper pre-training, Siamese Networks that embed texts and labels offer a competitive alternative. These models allow for a large reduction in inference cost: constant in the number of labels rather than linear. Furthermore, we introduce label tuning, a simple and computationally efficient approach that allows to adapt the models in a few-shot setup by only changing the label embeddings. While giving lower performance than model fine-tuning, this approach has the architectural advantage that a single encoder can be shared by many different tasks.- Anthology ID:
- 2022.acl-long.584
- Volume:
- Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Month:
- May
- Year:
- 2022
- Address:
- Dublin, Ireland
- Editors:
- Smaranda Muresan, Preslav Nakov, Aline Villavicencio
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 8532–8545
- Language:
- URL:
- https://aclanthology.org/2022.acl-long.584
- DOI:
- 10.18653/v1/2022.acl-long.584
- Cite (ACL):
- Thomas Müller, Guillermo Pérez-Torró, and Marc Franco-Salvador. 2022. Few-Shot Learning with Siamese Networks and Label Tuning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8532–8545, Dublin, Ireland. Association for Computational Linguistics.
- Cite (Informal):
- Few-Shot Learning with Siamese Networks and Label Tuning (Müller et al., ACL 2022)
- PDF:
- https://preview.aclanthology.org/improve-issue-templates/2022.acl-long.584.pdf
- Code
- symanto-research/few-shot-learning-label-tuning
- Data
- AG News, CoLA, HeadQA, IMDb Movie Reviews, ISEAR, MultiNLI, SB10k, SNLI