Learning to Generate Examples for Semantic Processing Tasks

Danilo Croce, Simone Filice, Giuseppe Castellucci, Roberto Basili


Abstract
Even if recent Transformer-based architectures, such as BERT, achieved impressive results in semantic processing tasks, their fine-tuning stage still requires large scale training resources. Usually, Data Augmentation (DA) techniques can help to deal with low resource settings. In Text Classification tasks, the objective of DA is the generation of well-formed sentences that i) represent the desired task category and ii) are novel with respect to existing sentences. In this paper, we propose a neural approach to automatically learn to generate new examples using a pre-trained sequence-to-sequence model. We first learn a task-oriented similarity function that we use to pair similar examples. Then, we use these example pairs to train a model to generate examples. Experiments in low resource settings show that augmenting the training material with the proposed strategy systematically improves the results on text classification and natural language inference tasks by up to 10% accuracy, outperforming existing DA approaches.
Anthology ID:
2022.naacl-main.340
Volume:
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
July
Year:
2022
Address:
Seattle, United States
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4587–4601
Language:
URL:
https://aclanthology.org/2022.naacl-main.340
DOI:
10.18653/v1/2022.naacl-main.340
Bibkey:
Cite (ACL):
Danilo Croce, Simone Filice, Giuseppe Castellucci, and Roberto Basili. 2022. Learning to Generate Examples for Semantic Processing Tasks. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4587–4601, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
Learning to Generate Examples for Semantic Processing Tasks (Croce et al., NAACL 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2022.naacl-main.340.pdf
Video:
 https://preview.aclanthology.org/ingestion-script-update/2022.naacl-main.340.mp4
Data
SST