Response Construct Tagging: NLP-Aided Assessment for Engineering Education

Ananya Ganesh, Hugh Scribner, Jasdeep Singh, Katherine Goodman, Jean Hertzberg, Katharina Kann

Abstract
Recent advances in natural language processing (NLP) have greatly helped educational applications, for both teachers and students. In higher education, there is great potential to use NLP tools for advancing pedagogical research. In this paper, we focus on how NLP can help understand student experiences in engineering, thus facilitating engineering educators to carry out large scale analysis that is helpful for re-designing the curriculum. Here, we introduce a new task we call response construct tagging (RCT), in which student responses to tailored survey questions are automatically tagged for six constructs measuring transformative experiences and engineering identity of students. We experiment with state-of-the-art classification models for this task and investigate the effects of different sources of additional information. Our best model achieves an F1 score of 48. We further investigate multi-task training on the related task of sentiment classification, which improves our model’s performance to 55 F1. Finally, we provide a detailed qualitative analysis of model performance.
Anthology ID:
2022.bea-1.29
Volume:
Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022)
Month:
July
Year:
2022
Address:
Seattle, Washington
Editors:
Ekaterina Kochmar, Jill Burstein, Andrea Horbach, Ronja Laarmann-Quante, Nitin Madnani, Anaïs Tack, Victoria Yaneva, Zheng Yuan, Torsten Zesch
Venue:
BEA
SIG:
SIGEDU
Publisher:
Association for Computational Linguistics
Note:
Pages:
250–261
Language:
URL:
https://aclanthology.org/2022.bea-1.29
DOI:
10.18653/v1/2022.bea-1.29
Bibkey:
Cite (ACL):
Ananya Ganesh, Hugh Scribner, Jasdeep Singh, Katherine Goodman, Jean Hertzberg, and Katharina Kann. 2022. Response Construct Tagging: NLP-Aided Assessment for Engineering Education. In Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022), pages 250–261, Seattle, Washington. Association for Computational Linguistics.
Cite (Informal):
Response Construct Tagging: NLP-Aided Assessment for Engineering Education (Ganesh et al., BEA 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/landing_page/2022.bea-1.29.pdf
PDF Search