Grammatical Error Correction in Low Error Density Domains: A New Benchmark and Analyses
Simon Flachs, Ophélie Lacroix, Helen Yannakoudakis, Marek Rei, Anders Søgaard
Abstract
Evaluation of grammatical error correction (GEC) systems has primarily focused on essays written by non-native learners of English, which however is only part of the full spectrum of GEC applications. We aim to broaden the target domain of GEC and release CWEB, a new benchmark for GEC consisting of website text generated by English speakers of varying levels of proficiency. Website data is a common and important domain that contains far fewer grammatical errors than learner essays, which we show presents a challenge to state-of-the-art GEC systems. We demonstrate that a factor behind this is the inability of systems to rely on a strong internal language model in low error density domains. We hope this work shall facilitate the development of open-domain GEC models that generalize to different topics and genres.- Anthology ID:
- 2020.emnlp-main.680
- Volume:
- Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Editors:
- Bonnie Webber, Trevor Cohn, Yulan He, Yang Liu
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 8467–8478
- Language:
- URL:
- https://aclanthology.org/2020.emnlp-main.680
- DOI:
- 10.18653/v1/2020.emnlp-main.680
- Cite (ACL):
- Simon Flachs, Ophélie Lacroix, Helen Yannakoudakis, Marek Rei, and Anders Søgaard. 2020. Grammatical Error Correction in Low Error Density Domains: A New Benchmark and Analyses. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8467–8478, Online. Association for Computational Linguistics.
- Cite (Informal):
- Grammatical Error Correction in Low Error Density Domains: A New Benchmark and Analyses (Flachs et al., EMNLP 2020)
- PDF:
- https://preview.aclanthology.org/emnlp22-frontmatter/2020.emnlp-main.680.pdf
- Data
- FCE, JFLEG