The Specificity and Helpfulness of Peer-to-Peer Feedback in Higher Education
Roman Rietsche, Andrew Caines, Cornelius Schramm, Dominik Pfütze, Paula Buttery
Abstract
With the growth of online learning through MOOCs and other educational applications, it has become increasingly difficult for course providers to offer personalized feedback to students. Therefore asking students to provide feedback to each other has become one way to support learning. This peer-to-peer feedback has become increasingly important whether in MOOCs to provide feedback to thousands of students or in large-scale classes at universities. One of the challenges when allowing peer-to-peer feedback is that the feedback should be perceived as helpful, and an import factor determining helpfulness is how specific the feedback is. However, in classes including thousands of students, instructors do not have the resources to check the specificity of every piece of feedback between students. Therefore, we present an automatic classification model to measure sentence specificity in written feedback. The model was trained and tested on student feedback texts written in German where sentences have been labelled as general or specific. We find that we can automatically classify the sentences with an accuracy of 76.7% using a conventional feature-based approach, whereas transfer learning with BERT for German gives a classification accuracy of 81.1%. However, the feature-based approach comes with lower computational costs and preserves human interpretability of the coefficients. In addition we show that specificity of sentences in feedback texts has a weak positive correlation with perceptions of helpfulness. This indicates that specificity is one of the ingredients of good feedback, and invites further investigation.- Anthology ID:
- 2022.bea-1.15
- Volume:
- Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022)
- Month:
- July
- Year:
- 2022
- Address:
- Seattle, Washington
- Editors:
- Ekaterina Kochmar, Jill Burstein, Andrea Horbach, Ronja Laarmann-Quante, Nitin Madnani, Anaïs Tack, Victoria Yaneva, Zheng Yuan, Torsten Zesch
- Venue:
- BEA
- SIG:
- SIGEDU
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 107–117
- Language:
- URL:
- https://aclanthology.org/2022.bea-1.15
- DOI:
- 10.18653/v1/2022.bea-1.15
- Cite (ACL):
- Roman Rietsche, Andrew Caines, Cornelius Schramm, Dominik Pfütze, and Paula Buttery. 2022. The Specificity and Helpfulness of Peer-to-Peer Feedback in Higher Education. In Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022), pages 107–117, Seattle, Washington. Association for Computational Linguistics.
- Cite (Informal):
- The Specificity and Helpfulness of Peer-to-Peer Feedback in Higher Education (Rietsche et al., BEA 2022)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2022.bea-1.15.pdf