DSMentor: Curriculum-Guided Inference with Online Memory for Data-Science LLM Agents

He Wang, Alexander Hanbo Li, Yiqun Hu, Sheng Zhang, Hideo Kobayashi, Jiani Zhang, Henghui Zhu, Chung-Wei Hang, Patrick Ng


Abstract
Large language model (LLM) agents have shown strong capabilities in generating code to solve complex data science problems, yet they often overlook the impact of task order during inference. We present DSMentor, an inference-time optimization framework that applies curriculum learning—progressing from easier to harder tasks—to enhance LLM performance on challenging data science tasks. Guided by a mentor and supported by a growing long-term memory, DSMentor organizes problems by difficulty, retains prior experiences, and leverages them to guide subsequent reasoning. Extensive experiments on DSEval and QRData benchmarks show that DSMentor with Claude-3.5-Sonnet improves pass rates by up to 5.2% over baseline agents and achieves an 8.8% gain over GPT-4 with Program-of-Thoughts prompting. These results highlight the effectiveness of curriculum-based inference strategies in advancing LLM agents.
Anthology ID:
2026.surgellm-1.12
Volume:
Proceedings of the First Workshop on Structured Understanding, Retrieval, and Generation in the LLM Era (SURGeLLM 2026)
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Vivek Gupta, Kaize Ding, Harsha Kokel, Yue Zhao, Amit Agarwal, Yu Wang, Michael Glass, Yu Zhang, Kavitha Srinivas, Xiusi Chen, Oktie Hassanzadeh, Qi Zhu, Shuaichen Chang, Yuan Luo
Venues:
SURGeLLM | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
190–208
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.surgellm-1.12/
DOI:
Bibkey:
Cite (ACL):
He Wang, Alexander Hanbo Li, Yiqun Hu, Sheng Zhang, Hideo Kobayashi, Jiani Zhang, Henghui Zhu, Chung-Wei Hang, and Patrick Ng. 2026. DSMentor: Curriculum-Guided Inference with Online Memory for Data-Science LLM Agents. In Proceedings of the First Workshop on Structured Understanding, Retrieval, and Generation in the LLM Era (SURGeLLM 2026), pages 190–208, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
DSMentor: Curriculum-Guided Inference with Online Memory for Data-Science LLM Agents (Wang et al., SURGeLLM 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.surgellm-1.12.pdf