Baoyu Hou


2023

pdf
Improving Autoregressive Grammatical Error Correction with Non-autoregressive Models
Hang Cao | Zhiquan Cao | Chi Hu | Baoyu Hou | Tong Xiao | Jingbo Zhu
Findings of the Association for Computational Linguistics: ACL 2023

Grammatical Error Correction (GEC) aims to correct grammatical errors in sentences. We find that autoregressive models tend to assign low probabilities to tokens that need corrections. Here we introduce additional signals to the training of GEC models so that these systems can learn to better predict at ambiguous positions. To do this, we use a non-autoregressive model as an auxiliary model, and develop a new regularization term of training by considering the difference in predictions between the autoregressive and non-autoregressive models. We experiment with this method on both English and Chinese GEC tasks. Experimental results show that our GEC system outperforms the baselines on all the data sets significantly.