Abstract
Pre-trained Language Models (PLMs) can be accurately fine-tuned for downstream text processing tasks. Recently, researchers have introduced several parameter-efficient fine-tuning methods that optimize input prompts or adjust a small number of model parameters (e.g LoRA). In this study, we explore the impact of altering the input text of the original task in conjunction with parameter-efficient fine-tuning methods. To most effectively rewrite the input text, we train a few-shot paraphrase model with a Maximum-Marginal Likelihood objective. Using six few-shot text classification datasets, we show that enriching data with paraphrases at train and test time enhances the performance beyond what can be achieved with parameter-efficient fine-tuning alone. The code used for our experiments can be found at https://github.com/SaeedNajafi/RIFF.- Anthology ID:
- 2024.findings-acl.85
- Volume:
- Findings of the Association for Computational Linguistics ACL 2024
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand and virtual meeting
- Editors:
- Lun-Wei Ku, Andre Martins, Vivek Srikumar
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1447–1466
- Language:
- URL:
- https://aclanthology.org/2024.findings-acl.85
- DOI:
- 10.18653/v1/2024.findings-acl.85
- Cite (ACL):
- Saeed Najafi and Alona Fyshe. 2024. RIFF: Learning to Rephrase Inputs for Few-shot Fine-tuning of Language Models. In Findings of the Association for Computational Linguistics ACL 2024, pages 1447–1466, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
- Cite (Informal):
- RIFF: Learning to Rephrase Inputs for Few-shot Fine-tuning of Language Models (Najafi & Fyshe, Findings 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-5/2024.findings-acl.85.pdf