Low-Resource Compositional Semantic Parsing with Concept Pretraining

Subendhu Rongali, Mukund Sridhar, Haidar Khan, Konstantine Arkoudas, Wael Hamza, Andrew McCallum


Abstract
Semantic parsing plays a key role in digital voice assistants such as Alexa, Siri, and Google Assistant by mapping natural language to structured meaning representations. When we want to improve the capabilities of a voice assistant by adding a new domain, the underlying semantic parsing model needs to be retrained using thousands of annotated examples from the new domain, which is time-consuming and expensive. In this work, we present an architecture to perform such domain adaptation automatically, with only a small amount of metadata about the new domain and without any new training data (zero-shot) or with very few examples (few-shot). We use a base seq2seq (sequence-to-sequence) architecture and augment it with a concept encoder that encodes intent and slot tags from the new domain. We also introduce a novel decoder-focused approach to pretrain seq2seq models to be concept aware using Wikidata and use it to help our model learn important concepts and perform well in low-resource settings. We report few-shot and zero-shot results for compositional semantic parsing on the TOPv2 dataset and show that our model outperforms prior approaches in few-shot settings for the TOPv2 and SNIPS datasets.
Anthology ID:
2023.eacl-main.103
Volume:
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Andreas Vlachos, Isabelle Augenstein
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1410–1419
Language:
URL:
https://aclanthology.org/2023.eacl-main.103
DOI:
10.18653/v1/2023.eacl-main.103
Bibkey:
Cite (ACL):
Subendhu Rongali, Mukund Sridhar, Haidar Khan, Konstantine Arkoudas, Wael Hamza, and Andrew McCallum. 2023. Low-Resource Compositional Semantic Parsing with Concept Pretraining. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1410–1419, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
Low-Resource Compositional Semantic Parsing with Concept Pretraining (Rongali et al., EACL 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-3/2023.eacl-main.103.pdf
Dataset:
 2023.eacl-main.103.dataset.zip
Video:
 https://preview.aclanthology.org/nschneid-patch-3/2023.eacl-main.103.mp4