Abstract
Assessing summaries is a demanding, yet useful task which provides valuable information on language competence, especially for second language learners. We consider automated scoring of college-level summary writing task in English as a second language (EL2). We adopt the Reading-for-Understanding (RU) cognitive framework, extended with the Reading-to-Write (RW) element, and use analytic scoring with six rubrics covering content and writing quality. We show that regression models with reference-based and linguistic features considerably outperform the baselines across all the rubrics. Moreover, we find interesting correlations between summary features and analytic rubrics, revealing the links between the RU and RW constructs.- Anthology ID:
- I17-2031
- Volume:
- Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers)
- Month:
- November
- Year:
- 2017
- Address:
- Taipei, Taiwan
- Editors:
- Greg Kondrak, Taro Watanabe
- Venue:
- IJCNLP
- SIG:
- Publisher:
- Asian Federation of Natural Language Processing
- Note:
- Pages:
- 181–186
- Language:
- URL:
- https://aclanthology.org/I17-2031
- DOI:
- Cite (ACL):
- Tamara Sladoljev-Agejev and Jan Šnajder. 2017. Using Analytic Scoring Rubrics in the Automatic Assessment of College-Level Summary Writing Tasks in L2. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 181–186, Taipei, Taiwan. Asian Federation of Natural Language Processing.
- Cite (Informal):
- Using Analytic Scoring Rubrics in the Automatic Assessment of College-Level Summary Writing Tasks in L2 (Sladoljev-Agejev & Šnajder, IJCNLP 2017)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/I17-2031.pdf