Multilingual Code-Switching for Zero-Shot Cross-Lingual Intent Prediction and Slot Filling
Jitin Krishnan, Antonios Anastasopoulos, Hemant Purohit, Huzefa Rangwala
Abstract
Predicting user intent and detecting the corresponding slots from text are two key problems in Natural Language Understanding (NLU). Since annotated datasets are only available for a handful of languages, our work focuses particularly on a zero-shot scenario where the target language is unseen during training. In the context of zero-shot learning, this task is typically approached using representations from pre-trained multilingual language models such as mBERT or by fine-tuning on data automatically translated into the target language. We propose a novel method which augments monolingual source data using multilingual code-switching via random translations, to enhance generalizability of large multilingual language models when fine-tuning them for downstream tasks. Experiments on the MultiATIS++ benchmark show that our method leads to an average improvement of +4.2% in accuracy for the intent task and +1.8% in F1 for the slot-filling task over the state-of-the-art across 8 typologically diverse languages. We also study the impact of code-switching into different families of languages on downstream performance. Furthermore, we present an application of our method for crisis informatics using a new human-annotated tweet dataset of slot filling in English and Haitian Creole, collected during the Haiti earthquake.- Anthology ID:
- 2021.mrl-1.18
- Volume:
- Proceedings of the 1st Workshop on Multilingual Representation Learning
- Month:
- November
- Year:
- 2021
- Address:
- Punta Cana, Dominican Republic
- Venue:
- MRL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 211–223
- Language:
- URL:
- https://aclanthology.org/2021.mrl-1.18
- DOI:
- 10.18653/v1/2021.mrl-1.18
- Cite (ACL):
- Jitin Krishnan, Antonios Anastasopoulos, Hemant Purohit, and Huzefa Rangwala. 2021. Multilingual Code-Switching for Zero-Shot Cross-Lingual Intent Prediction and Slot Filling. In Proceedings of the 1st Workshop on Multilingual Representation Learning, pages 211–223, Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- Multilingual Code-Switching for Zero-Shot Cross-Lingual Intent Prediction and Slot Filling (Krishnan et al., MRL 2021)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2021.mrl-1.18.pdf
- Code
- jitinkrishnan/Multilingual-ZeroShot-SlotFilling