Fine-grained essay scoring of a complex writing task for native speakers
Andrea Horbach, Dirk Scholten-Akoun, Yuning Ding, Torsten Zesch
Abstract
Automatic essay scoring is nowadays successfully used even in high-stakes tests, but this is mainly limited to holistic scoring of learner essays. We present a new dataset of essays written by highly proficient German native speakers that is scored using a fine-grained rubric with the goal to provide detailed feedback. Our experiments with two state-of-the-art scoring systems (a neural and a SVM-based one) show a large drop in performance compared to existing datasets. This demonstrates the need for such datasets that allow to guide research on more elaborate essay scoring methods.- Anthology ID:
- W17-5040
- Volume:
- Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications
- Month:
- September
- Year:
- 2017
- Address:
- Copenhagen, Denmark
- Editors:
- Joel Tetreault, Jill Burstein, Claudia Leacock, Helen Yannakoudakis
- Venue:
- BEA
- SIG:
- SIGEDU
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 357–366
- Language:
- URL:
- https://aclanthology.org/W17-5040
- DOI:
- 10.18653/v1/W17-5040
- Cite (ACL):
- Andrea Horbach, Dirk Scholten-Akoun, Yuning Ding, and Torsten Zesch. 2017. Fine-grained essay scoring of a complex writing task for native speakers. In Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications, pages 357–366, Copenhagen, Denmark. Association for Computational Linguistics.
- Cite (Informal):
- Fine-grained essay scoring of a complex writing task for native speakers (Horbach et al., BEA 2017)
- PDF:
- https://preview.aclanthology.org/teach-a-man-to-fish/W17-5040.pdf