A Benchmark for Neural Readability Assessment of Texts in Spanish

Laura Vásquez-Rodríguez, Pedro-Manuel Cuenca-Jiménez, Sergio Morales-Esquivel, Fernando Alva-Manchego


Abstract
We release a new benchmark for Automated Readability Assessment (ARA) of texts in Spanish. We combined existing corpora with suitable texts collected from the Web, thus creating the largest available dataset for ARA of Spanish texts. All data was pre-processed and categorised to allow experimenting with ARA models that make predictions at two (simple and complex) or three (basic, intermediate, and advanced) readability levels, and at two text granularities (paragraphs and sentences). An analysis based on readability indices shows that our proposed datasets groupings are suitable for their designated readability level. We use our benchmark to train neural ARA models based on BERT in zero-shot, few-shot, and cross-lingual settings. Results show that either a monolingual or multilingual pre-trained model can achieve good results when fine-tuned in language-specific data. In addition, all mod- els decrease their performance when predicting three classes instead of two, showing opportunities for the development of better ARA models for Spanish with existing resources.
Anthology ID:
2022.tsar-1.18
Volume:
Proceedings of the Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022)
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates (Virtual)
Venue:
TSAR
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
188–198
Language:
URL:
https://aclanthology.org/2022.tsar-1.18
DOI:
Bibkey:
Cite (ACL):
Laura Vásquez-Rodríguez, Pedro-Manuel Cuenca-Jiménez, Sergio Morales-Esquivel, and Fernando Alva-Manchego. 2022. A Benchmark for Neural Readability Assessment of Texts in Spanish. In Proceedings of the Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022), pages 188–198, Abu Dhabi, United Arab Emirates (Virtual). Association for Computational Linguistics.
Cite (Informal):
A Benchmark for Neural Readability Assessment of Texts in Spanish (Vásquez-Rodríguez et al., TSAR 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2022.tsar-1.18.pdf