Abstract
We report two essential improvements in readability assessment: 1. three novel features in advanced semantics and 2. the timely evidence that traditional ML models (e.g. Random Forest, using handcrafted features) can combine with transformers (e.g. RoBERTa) to augment model performance. First, we explore suitable transformers and traditional ML models. Then, we extract 255 handcrafted linguistic features using self-developed extraction software. Finally, we assemble those to create several hybrid models, achieving state-of-the-art (SOTA) accuracy on popular datasets in readability assessment. The use of handcrafted features help model performance on smaller datasets. Notably, our RoBERTA-RF-T1 hybrid achieves the near-perfect classification accuracy of 99%, a 20.3% increase from the previous SOTA.- Anthology ID:
- 2021.emnlp-main.834
- Original:
- 2021.emnlp-main.834v1
- Version 2:
- 2021.emnlp-main.834v2
- Volume:
- Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
- Month:
- November
- Year:
- 2021
- Address:
- Online and Punta Cana, Dominican Republic
- Editors:
- Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 10669–10686
- Language:
- URL:
- https://aclanthology.org/2021.emnlp-main.834
- DOI:
- 10.18653/v1/2021.emnlp-main.834
- Cite (ACL):
- Bruce W. Lee, Yoo Sung Jang, and Jason Lee. 2021. Pushing on Text Readability Assessment: A Transformer Meets Handcrafted Linguistic Features. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10669–10686, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- Pushing on Text Readability Assessment: A Transformer Meets Handcrafted Linguistic Features (Lee et al., EMNLP 2021)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2021.emnlp-main.834.pdf
- Code
- brucewlee/lingfeat
- Data
- OneStopEnglish