Linguistic Corpus Annotation for Automatic Text Simplification Evaluation
Rémi Cardon, Adrien Bibal, Rodrigo Wilkens, David Alfter, Magali Norré, Adeline Müller, Watrin Patrick, Thomas François
Abstract
Evaluating automatic text simplification (ATS) systems is a difficult task that is either performed by automatic metrics or user-based evaluations. However, from a linguistic point-of-view, it is not always clear on what bases these evaluations operate. In this paper, we propose annotations of the ASSET corpus that can be used to shed more light on ATS evaluation. In addition to contributing with this resource, we show how it can be used to analyze SARI’s behavior and to re-evaluate existing ATS systems. We present our insights as a step to improve ATS evaluation protocols in the future.- Anthology ID:
- 2022.emnlp-main.121
- Volume:
- Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, United Arab Emirates
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1842–1866
- Language:
- URL:
- https://aclanthology.org/2022.emnlp-main.121
- DOI:
- Cite (ACL):
- Rémi Cardon, Adrien Bibal, Rodrigo Wilkens, David Alfter, Magali Norré, Adeline Müller, Watrin Patrick, and Thomas François. 2022. Linguistic Corpus Annotation for Automatic Text Simplification Evaluation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1842–1866, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
- Cite (Informal):
- Linguistic Corpus Annotation for Automatic Text Simplification Evaluation (Cardon et al., EMNLP 2022)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2022.emnlp-main.121.pdf