Combining ResNet and Transformer for Chinese Grammatical Error Diagnosis

Shaolei Wang, Baoxin Wang, Jiefu Gong, Zhongyuan Wang, Xiao Hu, Xingyi Duan, Zizhuo Shen, Gang Yue, Ruiji Fu, Dayong Wu, Wanxiang Che, Shijin Wang, Guoping Hu, Ting Liu

Abstract
Grammatical error diagnosis is an important task in natural language processing. This paper introduces our system at NLPTEA-2020 Task: Chinese Grammatical Error Diagnosis (CGED). CGED aims to diagnose four types of grammatical errors which are missing words (M), redundant words (R), bad word selection (S) and disordered words (W). Our system is built on the model of multi-layer bidirectional transformer encoder and ResNet is integrated into the encoder to improve the performance. We also explore two ensemble strategies including weighted averaging and stepwise ensemble selection from libraries of models to improve the performance of single model. In official evaluation, our system obtains the highest F1 scores at identification level and position level. We also recommend error corrections for specific error types and achieve the second highest F1 score at correction level.
Anthology ID:
2020.nlptea-1.5
Volume:
Proceedings of the 6th Workshop on Natural Language Processing Techniques for Educational Applications
Month:
December
Year:
2020
Address:
Suzhou, China
Venue:
NLP-TEA
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
36–43
Language:
URL:
https://aclanthology.org/2020.nlptea-1.5
DOI:
Bibkey:
Cite (ACL):
Shaolei Wang, Baoxin Wang, Jiefu Gong, Zhongyuan Wang, Xiao Hu, Xingyi Duan, Zizhuo Shen, Gang Yue, Ruiji Fu, Dayong Wu, Wanxiang Che, Shijin Wang, Guoping Hu, and Ting Liu. 2020. Combining ResNet and Transformer for Chinese Grammatical Error Diagnosis. In Proceedings of the 6th Workshop on Natural Language Processing Techniques for Educational Applications, pages 36–43, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
Combining ResNet and Transformer for Chinese Grammatical Error Diagnosis (Wang et al., NLP-TEA 2020)
Copy Citation:
PDF:
https://preview.aclanthology.org/auto-file-uploads/2020.nlptea-1.5.pdf
PDF Search