An empirical investigation of neural methods for content scoring of science explanations
Brian Riordan, Sarah Bichler, Allison Bradford, Jennifer King Chen, Korah Wiley, Libby Gerard, Marcia C. Linn
Abstract
With the widespread adoption of the Next Generation Science Standards (NGSS), science teachers and online learning environments face the challenge of evaluating students’ integration of different dimensions of science learning. Recent advances in representation learning in natural language processing have proven effective across many natural language processing tasks, but a rigorous evaluation of the relative merits of these methods for scoring complex constructed response formative assessments has not previously been carried out. We present a detailed empirical investigation of feature-based, recurrent neural network, and pre-trained transformer models on scoring content in real-world formative assessment data. We demonstrate that recent neural methods can rival or exceed the performance of feature-based methods. We also provide evidence that different classes of neural models take advantage of different learning cues, and pre-trained transformer models may be more robust to spurious, dataset-specific learning cues, better reflecting scoring rubrics.- Anthology ID:
- 2020.bea-1.13
- Volume:
- Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications
- Month:
- July
- Year:
- 2020
- Address:
- Seattle, WA, USA → Online
- Editors:
- Jill Burstein, Ekaterina Kochmar, Claudia Leacock, Nitin Madnani, Ildikó Pilán, Helen Yannakoudakis, Torsten Zesch
- Venue:
- BEA
- SIG:
- SIGEDU
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 135–144
- Language:
- URL:
- https://aclanthology.org/2020.bea-1.13
- DOI:
- 10.18653/v1/2020.bea-1.13
- Cite (ACL):
- Brian Riordan, Sarah Bichler, Allison Bradford, Jennifer King Chen, Korah Wiley, Libby Gerard, and Marcia C. Linn. 2020. An empirical investigation of neural methods for content scoring of science explanations. In Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 135–144, Seattle, WA, USA → Online. Association for Computational Linguistics.
- Cite (Informal):
- An empirical investigation of neural methods for content scoring of science explanations (Riordan et al., BEA 2020)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-1/2020.bea-1.13.pdf