Abstract
We address the problem of automatic short answer grading, evaluating a collection of approaches inspired by recent advances in distributional text representations. In addition, we propose an unsupervised approach for determining text similarity using one-to-many alignment of word vectors. We evaluate the proposed technique across two datasets from different domains, namely, computer science and English reading comprehension, that additionally vary between highschool level and undergraduate students. Experiments demonstrate that the proposed technique often outperforms other compositional distributional semantics approaches as well as vector space methods such as latent semantic analysis. When combined with a scoring scheme, the proposed technique provides a powerful tool for tackling the complex problem of short answer grading. We also discuss a number of other key points worthy of consideration in preparing viable, easy-to-deploy automatic short-answer grading systems for the real-world.- Anthology ID:
- W16-4904
- Volume:
- Proceedings of the 3rd Workshop on Natural Language Processing Techniques for Educational Applications (NLPTEA2016)
- Month:
- December
- Year:
- 2016
- Address:
- Osaka, Japan
- Editors:
- Hsin-Hsi Chen, Yuen-Hsien Tseng, Vincent Ng, Xiaofei Lu
- Venue:
- NLP-TEA
- SIG:
- Publisher:
- The COLING 2016 Organizing Committee
- Note:
- Pages:
- 20–29
- Language:
- URL:
- https://aclanthology.org/W16-4904
- DOI:
- Cite (ACL):
- Oliver Adams, Shourya Roy, and Raghuram Krishnapuram. 2016. Distributed Vector Representations for Unsupervised Automatic Short Answer Grading. In Proceedings of the 3rd Workshop on Natural Language Processing Techniques for Educational Applications (NLPTEA2016), pages 20–29, Osaka, Japan. The COLING 2016 Organizing Committee.
- Cite (Informal):
- Distributed Vector Representations for Unsupervised Automatic Short Answer Grading (Adams et al., NLP-TEA 2016)
- PDF:
- https://preview.aclanthology.org/emnlp22-frontmatter/W16-4904.pdf