Abstract
A Multilingual model relies on language encodings to identify input languages because the multilingual model has to distinguish between the input and output languages or among all the languages for cross-lingual tasks. Furthermore, we find that language encodings potentially refine multiple morphologies of different languages to form a better isomorphic space for multilinguality. To leverage this observation, we present a method to compute a vocabulary-informed language encoding as the language representation, for a required language, considering a local vocabulary covering an acceptable amount of the most frequent word embeddings in this language. In our experiments, our method can consistently improve the performance of multilingual models on unsupervised neural machine translation and cross-lingual embedding.- Anthology ID:
- 2022.coling-1.432
- Volume:
- Proceedings of the 29th International Conference on Computational Linguistics
- Month:
- October
- Year:
- 2022
- Address:
- Gyeongju, Republic of Korea
- Editors:
- Nicoletta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin-Hsi Chen, Lucia Donatelli, Heng Ji, Sadao Kurohashi, Patrizia Paggio, Nianwen Xue, Seokhwan Kim, Younggyun Hahm, Zhong He, Tony Kyungil Lee, Enrico Santus, Francis Bond, Seung-Hoon Na
- Venue:
- COLING
- SIG:
- Publisher:
- International Committee on Computational Linguistics
- Note:
- Pages:
- 4883–4891
- Language:
- URL:
- https://aclanthology.org/2022.coling-1.432
- DOI:
- Cite (ACL):
- Xi Ai and Bin Fang. 2022. Vocabulary-informed Language Encoding. In Proceedings of the 29th International Conference on Computational Linguistics, pages 4883–4891, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
- Cite (Informal):
- Vocabulary-informed Language Encoding (Ai & Fang, COLING 2022)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/2022.coling-1.432.pdf