Reviewriter: AI-Generated Instructions For Peer Review Writing
Xiaotian Su, Thiemo Wambsganss, Roman Rietsche, Seyed Parsa Neshaei, Tanja Käser
Abstract
Large Language Models (LLMs) offer novel opportunities for educational applications that have the potential to transform traditional learning for students. Despite AI-enhanced applications having the potential to provide personalized learning experiences, more studies are needed on the design of generative AI systems and evidence for using them in real educational settings. In this paper, we design, implement and evaluate \texttt{Reviewriter}, a novel tool to provide students with AI-generated instructions for writing peer reviews in German. Our study identifies three key aspects: a) we provide insights into student needs when writing peer reviews with generative models which we then use to develop a novel system to provide adaptive instructions b) we fine-tune three German language models on a selected corpus of 11,925 student-written peer review texts in German and choose German-GPT2 based on quantitative measures and human evaluation, and c) we evaluate our tool with fourteen students, revealing positive technology acceptance based on quantitative measures. Additionally, the qualitative feedback presents the benefits and limitations of generative AI in peer review writing.- Anthology ID:
- 2023.bea-1.5
- Volume:
- Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023)
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Ekaterina Kochmar, Jill Burstein, Andrea Horbach, Ronja Laarmann-Quante, Nitin Madnani, Anaïs Tack, Victoria Yaneva, Zheng Yuan, Torsten Zesch
- Venue:
- BEA
- SIG:
- SIGEDU
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 57–71
- Language:
- URL:
- https://aclanthology.org/2023.bea-1.5
- DOI:
- 10.18653/v1/2023.bea-1.5
- Cite (ACL):
- Xiaotian Su, Thiemo Wambsganss, Roman Rietsche, Seyed Parsa Neshaei, and Tanja Käser. 2023. Reviewriter: AI-Generated Instructions For Peer Review Writing. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 57–71, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Reviewriter: AI-Generated Instructions For Peer Review Writing (Su et al., BEA 2023)
- PDF:
- https://preview.aclanthology.org/naacl-24-ws-corrections/2023.bea-1.5.pdf