Abstract
Token free approaches have been successfully applied to a series of word and span level tasks. In this work, we evaluate a byte-level sequence to sequence model (ByT5) on the 51 languages in the MASSIVE multilingual semantic parsing dataset. We examine multiple experimental settings: (i) zero-shot, (ii) full gold data and (iii) zero-shot with synthetic data. By leveraging a state-of-the-art label projection method for machine translated examples, we are able to reduce the gap in exact match to only 5 points with respect to a model trained on gold data from all the languages. We additionally provide insights on the cross-lingual transfer of ByT5 and show how the model compares with respect to mT5 across all parameter sizes.- Anthology ID:
- 2022.mmnlu-1.3
- Volume:
- Proceedings of the Massively Multilingual Natural Language Understanding Workshop (MMNLU-22)
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, United Arab Emirates (Hybrid)
- Venue:
- MMNLU
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 25–34
- Language:
- URL:
- https://aclanthology.org/2022.mmnlu-1.3
- DOI:
- Cite (ACL):
- Massimo Nicosia and Francesco Piccinno. 2022. Byte-Level Massively Multilingual Semantic Parsing. In Proceedings of the Massively Multilingual Natural Language Understanding Workshop (MMNLU-22), pages 25–34, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
- Cite (Informal):
- Byte-Level Massively Multilingual Semantic Parsing (Nicosia & Piccinno, MMNLU 2022)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2022.mmnlu-1.3.pdf