Abstract
Automatically scoring student answers is an important task that is usually solved using instance-based supervised learning. Recently, similarity-based scoring has been proposed as an alternative approach yielding similar perfor- mance. It has hypothetical advantages such as a lower need for annotated training data and better zero-shot performance, both of which are properties that would be highly beneficial when applying content scoring in a realistic classroom setting.In this paper we take a closer look at these alleged advantages by comparing different instance-based and similarity-based methods on multiple data sets in a number of learning curve experiments. We find that both the demand on data and cross-prompt performance is similar, thus not confirming the former two suggested advantages. The by default more straightforward possibility to give feedback based on a similarity-based approach may thus tip the scales in favor of it, although future work is needed to explore this advantage in practice.- Anthology ID:
- 2023.findings-acl.119
- Volume:
- Findings of the Association for Computational Linguistics: ACL 2023
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1892–1903
- Language:
- URL:
- https://aclanthology.org/2023.findings-acl.119
- DOI:
- Cite (ACL):
- Marie Bexte, Andrea Horbach, and Torsten Zesch. 2023. Similarity-Based Content Scoring - A more Classroom-Suitable Alternative to Instance-Based Scoring?. In Findings of the Association for Computational Linguistics: ACL 2023, pages 1892–1903, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Similarity-Based Content Scoring - A more Classroom-Suitable Alternative to Instance-Based Scoring? (Bexte et al., Findings 2023)
- PDF:
- https://preview.aclanthology.org/nodalida-main-page/2023.findings-acl.119.pdf