CRASpell: A Contextual Typo Robust Approach to Improve Chinese Spelling Correction
Shulin Liu, Shengkang Song, Tianchi Yue, Tao Yang, Huihui Cai, TingHao Yu, Shengli Sun
Abstract
Recently, Bert-based models have dominated the research of Chinese spelling correction (CSC). These methods have two limitations: (1) they have poor performance on multi-typo texts. In such texts, the context of each typo contains at least one misspelled character, which brings noise information. Such noisy context leads to the declining performance on multi-typo texts. (2) they tend to overcorrect valid expressions to more frequent expressions due to the masked token recovering task of Bert. We attempt to address these limitations in this paper. To make our model robust to contextual noise brought by typos, our approach first constructs a noisy context for each training sample. Then the correction model is forced to yield similar outputs based on the noisy and original contexts. Moreover, to address the overcorrection problem, copy mechanism is incorporated to encourage our model to prefer to choose the input character when the miscorrected and input character are both valid according to the given context. Experiments are conducted on widely used benchmarks. Our model achieves superior performance against state-of-the-art methods by a remarkable gain.- Anthology ID:
- 2022.findings-acl.237
- Volume:
- Findings of the Association for Computational Linguistics: ACL 2022
- Month:
- May
- Year:
- 2022
- Address:
- Dublin, Ireland
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 3008–3018
- Language:
- URL:
- https://aclanthology.org/2022.findings-acl.237
- DOI:
- 10.18653/v1/2022.findings-acl.237
- Cite (ACL):
- Shulin Liu, Shengkang Song, Tianchi Yue, Tao Yang, Huihui Cai, TingHao Yu, and Shengli Sun. 2022. CRASpell: A Contextual Typo Robust Approach to Improve Chinese Spelling Correction. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3008–3018, Dublin, Ireland. Association for Computational Linguistics.
- Cite (Informal):
- CRASpell: A Contextual Typo Robust Approach to Improve Chinese Spelling Correction (Liu et al., Findings 2022)
- PDF:
- https://preview.aclanthology.org/paclic-22-ingestion/2022.findings-acl.237.pdf
- Code
- liushulinle/craspell