Let GPT be a Math Tutor: Teaching Math Word Problem Solvers with Customized Exercise Generation

Zhenwen Liang, Wenhao Yu, Tanmay Rajpurohit, Peter Clark, Xiangliang Zhang, Ashwin Kalyan


Abstract
In this paper, we present a novel approach for distilling math word problem solving capabilities from large language models (LLMs) into smaller, more efficient student models. Our approach is designed to consider the student model’s weaknesses and foster a tailored learning experience by generating targeted exercises aligned with educational science principles, such as knowledge tracing and personalized learning. Concretely, we let GPT-3 be a math tutor and run two steps iteratively: 1) assessing the student model’s current learning status on a GPT-generated exercise book, and 2) improving the student model by training it with tailored exercise samples generated by GPT-3. Experimental results reveal that our approach outperforms LLMs (e.g., GPT-3 and PaLM) in accuracy across three distinct benchmarks while employing significantly fewer parameters. Furthermore, we provide a comprehensive analysis of the various components within our methodology to substantiate their efficacy.
Anthology ID:
2023.emnlp-main.889
Volume:
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
14384–14396
Language:
URL:
https://aclanthology.org/2023.emnlp-main.889
DOI:
10.18653/v1/2023.emnlp-main.889
Bibkey:
Cite (ACL):
Zhenwen Liang, Wenhao Yu, Tanmay Rajpurohit, Peter Clark, Xiangliang Zhang, and Ashwin Kalyan. 2023. Let GPT be a Math Tutor: Teaching Math Word Problem Solvers with Customized Exercise Generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14384–14396, Singapore. Association for Computational Linguistics.
Cite (Informal):
Let GPT be a Math Tutor: Teaching Math Word Problem Solvers with Customized Exercise Generation (Liang et al., EMNLP 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-2/2023.emnlp-main.889.pdf
Video:
 https://preview.aclanthology.org/nschneid-patch-2/2023.emnlp-main.889.mp4