Mistake Captioning: A Machine Learning Approach for Detecting Mistakes and Generating Instructive Feedback

Anton Vinogradov, Andrew Miles Byrd, Brent Harrison


Abstract
Giving feedback to students is not just about marking their answers as correct or incorrect, but also finding mistakes in their thought process that led them to that incorrect answer. In this paper, we introduce a machine learning technique for mistake captioning, a task that attempts to identify mistakes and provide feedback meant to help learners correct these mistakes. We do this by training a sequence-to-sequence network to generate this feedback based on domain experts. To evaluate this system, we explore how it can be used on a Linguistics assignment studying Grimm’s Law. We show that our approach generates feedback that outperforms a baseline on a set of automated NLP metrics. In addition, we perform a series of case studies in which we examine successful and unsuccessful system outputs.
Anthology ID:
2021.ranlp-1.163
Volume:
Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)
Month:
September
Year:
2021
Address:
Held Online
Editors:
Ruslan Mitkov, Galia Angelova
Venue:
RANLP
SIG:
Publisher:
INCOMA Ltd.
Note:
Pages:
1455–1462
Language:
URL:
https://aclanthology.org/2021.ranlp-1.163
DOI:
Bibkey:
Cite (ACL):
Anton Vinogradov, Andrew Miles Byrd, and Brent Harrison. 2021. Mistake Captioning: A Machine Learning Approach for Detecting Mistakes and Generating Instructive Feedback. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 1455–1462, Held Online. INCOMA Ltd..
Cite (Informal):
Mistake Captioning: A Machine Learning Approach for Detecting Mistakes and Generating Instructive Feedback (Vinogradov et al., RANLP 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/naacl-24-ws-corrections/2021.ranlp-1.163.pdf