Byte-Level Massively Multilingual Semantic Parsing

Massimo Nicosia, Francesco Piccinno


Abstract
Token free approaches have been successfully applied to a series of word and span level tasks. In this work, we evaluate a byte-level sequence to sequence model (ByT5) on the 51 languages in the MASSIVE multilingual semantic parsing dataset. We examine multiple experimental settings: (i) zero-shot, (ii) full gold data and (iii) zero-shot with synthetic data. By leveraging a state-of-the-art label projection method for machine translated examples, we are able to reduce the gap in exact match to only 5 points with respect to a model trained on gold data from all the languages. We additionally provide insights on the cross-lingual transfer of ByT5 and show how the model compares with respect to mT5 across all parameter sizes.
Anthology ID:
2022.mmnlu-1.3
Volume:
Proceedings of the Massively Multilingual Natural Language Understanding Workshop (MMNLU-22)
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates (Hybrid)
Venue:
MMNLU
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
25–34
Language:
URL:
https://aclanthology.org/2022.mmnlu-1.3
DOI:
Bibkey:
Cite (ACL):
Massimo Nicosia and Francesco Piccinno. 2022. Byte-Level Massively Multilingual Semantic Parsing. In Proceedings of the Massively Multilingual Natural Language Understanding Workshop (MMNLU-22), pages 25–34, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
Cite (Informal):
Byte-Level Massively Multilingual Semantic Parsing (Nicosia & Piccinno, MMNLU 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2022.mmnlu-1.3.pdf