Open Vocabulary Learning for Neural Chinese Pinyin IME

Zhuosheng Zhang, Yafang Huang, Hai Zhao


Abstract
Pinyin-to-character (P2C) conversion is the core component of pinyin-based Chinese input method engine (IME). However, the conversion is seriously compromised by the ambiguities of Chinese characters corresponding to pinyin as well as the predefined fixed vocabularies. To alleviate such inconveniences, we propose a neural P2C conversion model augmented by an online updated vocabulary with a sampling mechanism to support open vocabulary learning during IME working. Our experiments show that the proposed method outperforms commercial IMEs and state-of-the-art traditional models on standard corpus and true inputting history dataset in terms of multiple metrics and thus the online updated vocabulary indeed helps our IME effectively follows user inputting behavior.
Anthology ID:
P19-1154
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1584–1594
Language:
URL:
https://aclanthology.org/P19-1154
DOI:
10.18653/v1/P19-1154
Bibkey:
Cite (ACL):
Zhuosheng Zhang, Yafang Huang, and Hai Zhao. 2019. Open Vocabulary Learning for Neural Chinese Pinyin IME. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1584–1594, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Open Vocabulary Learning for Neural Chinese Pinyin IME (Zhang et al., ACL 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/auto-file-uploads/P19-1154.pdf
Code
 cooelf/OpenIME