Learning to Generate Examples for Semantic Processing Tasks
Danilo Croce, Simone Filice, Giuseppe Castellucci, Roberto Basili
Abstract
Even if recent Transformer-based architectures, such as BERT, achieved impressive results in semantic processing tasks, their fine-tuning stage still requires large scale training resources. Usually, Data Augmentation (DA) techniques can help to deal with low resource settings. In Text Classification tasks, the objective of DA is the generation of well-formed sentences that i) represent the desired task category and ii) are novel with respect to existing sentences. In this paper, we propose a neural approach to automatically learn to generate new examples using a pre-trained sequence-to-sequence model. We first learn a task-oriented similarity function that we use to pair similar examples. Then, we use these example pairs to train a model to generate examples. Experiments in low resource settings show that augmenting the training material with the proposed strategy systematically improves the results on text classification and natural language inference tasks by up to 10% accuracy, outperforming existing DA approaches.- Anthology ID:
- 2022.naacl-main.340
- Volume:
- Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Month:
- July
- Year:
- 2022
- Address:
- Seattle, United States
- Editors:
- Marine Carpuat, Marie-Catherine de Marneffe, Ivan Vladimir Meza Ruiz
- Venue:
- NAACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 4587–4601
- Language:
- URL:
- https://aclanthology.org/2022.naacl-main.340
- DOI:
- 10.18653/v1/2022.naacl-main.340
- Cite (ACL):
- Danilo Croce, Simone Filice, Giuseppe Castellucci, and Roberto Basili. 2022. Learning to Generate Examples for Semantic Processing Tasks. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4587–4601, Seattle, United States. Association for Computational Linguistics.
- Cite (Informal):
- Learning to Generate Examples for Semantic Processing Tasks (Croce et al., NAACL 2022)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2022.naacl-main.340.pdf
- Data
- SST