Modeling Code-Switch Languages Using Bilingual Parallel Corpus

Grandee Lee, Haizhou Li


Abstract
Language modeling is the technique to estimate the probability of a sequence of words. A bilingual language model is expected to model the sequential dependency for words across languages, which is difficult due to the inherent lack of suitable training data as well as diverse syntactic structure across languages. We propose a bilingual attention language model (BALM) that simultaneously performs language modeling objective with a quasi-translation objective to model both the monolingual as well as the cross-lingual sequential dependency. The attention mechanism learns the bilingual context from a parallel corpus. BALM achieves state-of-the-art performance on the SEAME code-switch database by reducing the perplexity of 20.5% over the best-reported result. We also apply BALM in bilingual lexicon induction, and language normalization tasks to validate the idea.
Anthology ID:
2020.acl-main.80
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2020
Address:
Online
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
860–870
Language:
URL:
https://aclanthology.org/2020.acl-main.80
DOI:
10.18653/v1/2020.acl-main.80
Bibkey:
Cite (ACL):
Grandee Lee and Haizhou Li. 2020. Modeling Code-Switch Languages Using Bilingual Parallel Corpus. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 860–870, Online. Association for Computational Linguistics.
Cite (Informal):
Modeling Code-Switch Languages Using Bilingual Parallel Corpus (Lee & Li, ACL 2020)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2020.acl-main.80.pdf
Video:
 http://slideslive.com/38928986