Conditional set generation using Seq2seq models
Aman Madaan, Dheeraj Rajagopal, Niket Tandon, Yiming Yang, Antoine Bosselut
Abstract
Conditional set generation learns a mapping from an input sequence of tokens to a set. Several NLP tasks, such as entity typing and dialogue emotion tagging, are instances of set generation. Seq2Seq models are a popular choice to model set generation but they treat a set as a sequence and do not fully leverage its key properties, namely order-invariance and cardinality. We propose a novel algorithm for effectively sampling informative orders over the combinatorial space of label orders. Further, we jointly model the set cardinality and output by listing the set size as the first element and taking advantage of the autoregressive factorization used by Seq2Seq models. Our method is a model-independent data augmentation approach that endows any Seq2Seq model with the signals of order-invariance and cardinality. Training a Seq2Seq model on this new augmented data (without any additional annotations), gets an average relative improvement of 20% for four benchmarks datasets across models spanning from BART-base, T5-11B, and GPT-3. We will release all code and data upon acceptance.- Anthology ID:
- 2022.emnlp-main.324
- Volume:
- Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, United Arab Emirates
- Editors:
- Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 4874–4896
- Language:
- URL:
- https://aclanthology.org/2022.emnlp-main.324
- DOI:
- 10.18653/v1/2022.emnlp-main.324
- Cite (ACL):
- Aman Madaan, Dheeraj Rajagopal, Niket Tandon, Yiming Yang, and Antoine Bosselut. 2022. Conditional set generation using Seq2seq models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4874–4896, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
- Cite (Informal):
- Conditional set generation using Seq2seq models (Madaan et al., EMNLP 2022)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2022.emnlp-main.324.pdf