Abstract
We propose USim, a semantic measure for Grammatical Error Correction (that measures the semantic faithfulness of the output to the source, thereby complementing existing reference-less measures (RLMs) for measuring the output’s grammaticality. USim operates by comparing the semantic symbolic structure of the source and the correction, without relying on manually-curated references. Our experiments establish the validity of USim, by showing that the semantic structures can be consistently applied to ungrammatical text, that valid corrections obtain a high USim similarity score to the source, and that invalid corrections obtain a lower score.- Anthology ID:
- N18-2020
- Volume:
- Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)
- Month:
- June
- Year:
- 2018
- Address:
- New Orleans, Louisiana
- Editors:
- Marilyn Walker, Heng Ji, Amanda Stent
- Venue:
- NAACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 124–129
- Language:
- URL:
- https://aclanthology.org/N18-2020
- DOI:
- 10.18653/v1/N18-2020
- Cite (ACL):
- Leshem Choshen and Omri Abend. 2018. Reference-less Measure of Faithfulness for Grammatical Error Correction. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 124–129, New Orleans, Louisiana. Association for Computational Linguistics.
- Cite (Informal):
- Reference-less Measure of Faithfulness for Grammatical Error Correction (Choshen & Abend, NAACL 2018)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/N18-2020.pdf
- Code
- borgr/USim