Brent Harrison


pdf bib
Error Causal inference for Multi-Fusion models
Chengxi Li | Brent Harrison
Proceedings of the Second Workshop on Advances in Language and Vision Research

In this paper, we propose an error causal inference method that could be used for finding dominant features for a faulty instance under a well-trained multi-modality input model, which could apply to any testing instance. We evaluate our method using a well-trained multi-modalities stylish caption generation model and find those causal inferences that could provide us the insights for next step optimization.

Mistake Captioning: A Machine Learning Approach for Detecting Mistakes and Generating Instructive Feedback
Anton Vinogradov | Andrew Miles Byrd | Brent Harrison
Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)

Giving feedback to students is not just about marking their answers as correct or incorrect, but also finding mistakes in their thought process that led them to that incorrect answer. In this paper, we introduce a machine learning technique for mistake captioning, a task that attempts to identify mistakes and provide feedback meant to help learners correct these mistakes. We do this by training a sequence-to-sequence network to generate this feedback based on domain experts. To evaluate this system, we explore how it can be used on a Linguistics assignment studying Grimm’s Law. We show that our approach generates feedback that outperforms a baseline on a set of automated NLP metrics. In addition, we perform a series of case studies in which we examine successful and unsuccessful system outputs.