Abstract
Language modeling is the technique to estimate the probability of a sequence of words. A bilingual language model is expected to model the sequential dependency for words across languages, which is difficult due to the inherent lack of suitable training data as well as diverse syntactic structure across languages. We propose a bilingual attention language model (BALM) that simultaneously performs language modeling objective with a quasi-translation objective to model both the monolingual as well as the cross-lingual sequential dependency. The attention mechanism learns the bilingual context from a parallel corpus. BALM achieves state-of-the-art performance on the SEAME code-switch database by reducing the perplexity of 20.5% over the best-reported result. We also apply BALM in bilingual lexicon induction, and language normalization tasks to validate the idea.- Anthology ID:
- 2020.acl-main.80
- Volume:
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
- Month:
- July
- Year:
- 2020
- Address:
- Online
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 860–870
- Language:
- URL:
- https://aclanthology.org/2020.acl-main.80
- DOI:
- 10.18653/v1/2020.acl-main.80
- Cite (ACL):
- Grandee Lee and Haizhou Li. 2020. Modeling Code-Switch Languages Using Bilingual Parallel Corpus. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 860–870, Online. Association for Computational Linguistics.
- Cite (Informal):
- Modeling Code-Switch Languages Using Bilingual Parallel Corpus (Lee & Li, ACL 2020)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2020.acl-main.80.pdf