Abstract
Reinforcement Learning remains an underutilized method of training and fine-tuning Language Models (LMs) despite recent successes. This paper presents a simple approach of fine-tuning a language model with Reinforcement Learning to achieve competitive performance on the BEA 2023 Shared Task whose goal is to automatically generate teacher responses in educational dialogues. We utilized the novel NLPO algorithm that masks out tokens during generation to direct the model towards generations that maximize a reward function. We show results for both the t5-base model with 220 million parameters from the HuggingFace repository submitted to the leaderboard that, despite its comparatively small size, has achieved a good performance on both test and dev set, as well as GPT-2 with 124 million parameters. The presented results show that despite maximizing only one of the metrics used in the evaluation as a reward function our model scores highly in the other metrics as well.- Anthology ID:
- 2023.bea-1.59
- Volume:
- Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023)
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Ekaterina Kochmar, Jill Burstein, Andrea Horbach, Ronja Laarmann-Quante, Nitin Madnani, Anaïs Tack, Victoria Yaneva, Zheng Yuan, Torsten Zesch
- Venue:
- BEA
- SIG:
- SIGEDU
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 736–744
- Language:
- URL:
- https://aclanthology.org/2023.bea-1.59
- DOI:
- 10.18653/v1/2023.bea-1.59
- Cite (ACL):
- Thomas Huber, Christina Niklaus, and Siegfried Handschuh. 2023. Enhancing Educational Dialogues: A Reinforcement Learning Approach for Generating AI Teacher Responses. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 736–744, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Enhancing Educational Dialogues: A Reinforcement Learning Approach for Generating AI Teacher Responses (Huber et al., BEA 2023)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2023.bea-1.59.pdf