Abstract
Grammatical Error Correction (GEC) systems play a vital role in assisting people with their daily writing tasks. However, users may sometimes come across a GEC system that initially performs well but fails to correct errors when the inputs are slightly modified. To ensure an ideal user experience, a reliable GEC system should have the ability to provide consistent and accurate suggestions when encountering irrelevant context perturbations, which we refer to as context robustness. In this paper, we introduce RobustGEC, a benchmark designed to evaluate the context robustness of GEC systems. RobustGEC comprises 5,000 GEC cases, each with one original error-correct sentence pair and five variants carefully devised by human annotators. Utilizing RobustGEC, we reveal that state-of-the-art GEC systems still lack sufficient robustness against context perturbations. Moreover, we propose a simple yet effective method for remitting this issue.- Anthology ID:
- 2023.emnlp-main.1043
- Volume:
- Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 16780–16793
- Language:
- URL:
- https://aclanthology.org/2023.emnlp-main.1043
- DOI:
- 10.18653/v1/2023.emnlp-main.1043
- Cite (ACL):
- Yue Zhang, Leyang Cui, Enbo Zhao, Wei Bi, and Shuming Shi. 2023. RobustGEC: Robust Grammatical Error Correction Against Subtle Context Perturbation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 16780–16793, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- RobustGEC: Robust Grammatical Error Correction Against Subtle Context Perturbation (Zhang et al., EMNLP 2023)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2023.emnlp-main.1043.pdf