How Many Data Samples is an Additional Instruction Worth?
Ravsehaj Singh Puri, Swaroop Mishra, Mihir Parmar, Chitta Baral
Abstract
Recently introduced instruction-paradigm empowers non-expert users to leverage NLP resources by defining a new task in natural language. Instruction-tuned models have significantly outperformed multitask learning models (without instruction); however they are far from state-of-the-art task-specific models. Conventional approaches to improve model performance via creating datasets with large number of task instances or architectural changes in the model may not be feasible for non-expert users. However, they can write alternate instructions to represent an instruction task. Is Instruction-augmentation helpful? We augment a subset of tasks in the expanded version of NATURAL INSTRUCTIONS with additional instructions and find that it significantly improves model performance (up to 35%), especially in the low-data regime. Our results indicate that an additional instruction can be equivalent to ~200 data samples on average across tasks.- Anthology ID:
- 2023.findings-eacl.77
- Volume:
- Findings of the Association for Computational Linguistics: EACL 2023
- Month:
- May
- Year:
- 2023
- Address:
- Dubrovnik, Croatia
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1012–1027
- Language:
- URL:
- https://aclanthology.org/2023.findings-eacl.77
- DOI:
- Cite (ACL):
- Ravsehaj Singh Puri, Swaroop Mishra, Mihir Parmar, and Chitta Baral. 2023. How Many Data Samples is an Additional Instruction Worth?. In Findings of the Association for Computational Linguistics: EACL 2023, pages 1012–1027, Dubrovnik, Croatia. Association for Computational Linguistics.
- Cite (Informal):
- How Many Data Samples is an Additional Instruction Worth? (Puri et al., Findings 2023)
- PDF:
- https://preview.aclanthology.org/author-url/2023.findings-eacl.77.pdf