Contextual Semantic Parsing for Multilingual Task-Oriented Dialogues
Mehrad Moradshahi, Victoria Tsai, Giovanni Campagna, Monica Lam
Abstract
Robust state tracking for task-oriented dialogue systems currently remains restricted to a few popular languages. This paper shows that given a large-scale dialogue data set in one language, we can automatically produce an effective semantic parser for other languages using machine translation. We propose automatic translation of dialogue datasets with alignment to ensure faithful translation of slot values and eliminate costly human supervision used in previous benchmarks. We also propose a new contextual semantic parsing model, which encodes the formal slots and values, and only the last agent and user utterances. We show that the succinct representation reduces the compounding effect of translation errors, without harming the accuracy in practice. We evaluate our approach on several dialogue state tracking benchmarks. On RiSAWOZ, CrossWOZ, CrossWOZ-EN, and MultiWOZ-ZH datasets we improve the state of the art by 11%, 17%, 20%, and 0.3% in joint goal accuracy. We present a comprehensive error analysis for all three datasets showing erroneous annotations can lead to misguided judgments on the quality of the model. Finally, we present RiSAWOZ English and German datasets, created using our translation methodology. On these datasets, accuracy is within 11% of the original showing that high-accuracy multilingual dialogue datasets are possible without relying on expensive human annotations. We release our datasets and software open source.- Anthology ID:
- 2023.eacl-main.63
- Volume:
- Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
- Month:
- May
- Year:
- 2023
- Address:
- Dubrovnik, Croatia
- Editors:
- Andreas Vlachos, Isabelle Augenstein
- Venue:
- EACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 902–915
- Language:
- URL:
- https://aclanthology.org/2023.eacl-main.63
- DOI:
- 10.18653/v1/2023.eacl-main.63
- Cite (ACL):
- Mehrad Moradshahi, Victoria Tsai, Giovanni Campagna, and Monica Lam. 2023. Contextual Semantic Parsing for Multilingual Task-Oriented Dialogues. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 902–915, Dubrovnik, Croatia. Association for Computational Linguistics.
- Cite (Informal):
- Contextual Semantic Parsing for Multilingual Task-Oriented Dialogues (Moradshahi et al., EACL 2023)
- PDF:
- https://preview.aclanthology.org/revert-3132-ingestion-checklist/2023.eacl-main.63.pdf