Low-Resource Compositional Semantic Parsing with Concept Pretraining
Subendhu Rongali, Mukund Sridhar, Haidar Khan, Konstantine Arkoudas, Wael Hamza, Andrew McCallum
Abstract
Semantic parsing plays a key role in digital voice assistants such as Alexa, Siri, and Google Assistant by mapping natural language to structured meaning representations. When we want to improve the capabilities of a voice assistant by adding a new domain, the underlying semantic parsing model needs to be retrained using thousands of annotated examples from the new domain, which is time-consuming and expensive. In this work, we present an architecture to perform such domain adaptation automatically, with only a small amount of metadata about the new domain and without any new training data (zero-shot) or with very few examples (few-shot). We use a base seq2seq (sequence-to-sequence) architecture and augment it with a concept encoder that encodes intent and slot tags from the new domain. We also introduce a novel decoder-focused approach to pretrain seq2seq models to be concept aware using Wikidata and use it to help our model learn important concepts and perform well in low-resource settings. We report few-shot and zero-shot results for compositional semantic parsing on the TOPv2 dataset and show that our model outperforms prior approaches in few-shot settings for the TOPv2 and SNIPS datasets.- Anthology ID:
- 2023.eacl-main.103
- Volume:
- Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
- Month:
- May
- Year:
- 2023
- Address:
- Dubrovnik, Croatia
- Editors:
- Andreas Vlachos, Isabelle Augenstein
- Venue:
- EACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1410–1419
- Language:
- URL:
- https://aclanthology.org/2023.eacl-main.103
- DOI:
- 10.18653/v1/2023.eacl-main.103
- Cite (ACL):
- Subendhu Rongali, Mukund Sridhar, Haidar Khan, Konstantine Arkoudas, Wael Hamza, and Andrew McCallum. 2023. Low-Resource Compositional Semantic Parsing with Concept Pretraining. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1410–1419, Dubrovnik, Croatia. Association for Computational Linguistics.
- Cite (Informal):
- Low-Resource Compositional Semantic Parsing with Concept Pretraining (Rongali et al., EACL 2023)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2023.eacl-main.103.pdf