DSMentor: Curriculum-Guided Inference with Online Memory for Data-Science LLM Agents
He Wang, Alexander Hanbo Li, Yiqun Hu, Sheng Zhang, Hideo Kobayashi, Jiani Zhang, Henghui Zhu, Chung-Wei Hang, Patrick Ng
Abstract
Large language model (LLM) agents have shown strong capabilities in generating code to solve complex data science problems, yet they often overlook the impact of task order during inference. We present DSMentor, an inference-time optimization framework that applies curriculum learning—progressing from easier to harder tasks—to enhance LLM performance on challenging data science tasks. Guided by a mentor and supported by a growing long-term memory, DSMentor organizes problems by difficulty, retains prior experiences, and leverages them to guide subsequent reasoning. Extensive experiments on DSEval and QRData benchmarks show that DSMentor with Claude-3.5-Sonnet improves pass rates by up to 5.2% over baseline agents and achieves an 8.8% gain over GPT-4 with Program-of-Thoughts prompting. These results highlight the effectiveness of curriculum-based inference strategies in advancing LLM agents.- Anthology ID:
- 2026.surgellm-1.12
- Volume:
- Proceedings of the First Workshop on Structured Understanding, Retrieval, and Generation in the LLM Era (SURGeLLM 2026)
- Month:
- July
- Year:
- 2026
- Address:
- San Diego, California, United States
- Editors:
- Vivek Gupta, Kaize Ding, Harsha Kokel, Yue Zhao, Amit Agarwal, Yu Wang, Michael Glass, Yu Zhang, Kavitha Srinivas, Xiusi Chen, Oktie Hassanzadeh, Qi Zhu, Shuaichen Chang, Yuan Luo
- Venues:
- SURGeLLM | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 190–208
- Language:
- URL:
- https://preview.aclanthology.org/ingest-acl-workshops/2026.surgellm-1.12/
- DOI:
- Cite (ACL):
- He Wang, Alexander Hanbo Li, Yiqun Hu, Sheng Zhang, Hideo Kobayashi, Jiani Zhang, Henghui Zhu, Chung-Wei Hang, and Patrick Ng. 2026. DSMentor: Curriculum-Guided Inference with Online Memory for Data-Science LLM Agents. In Proceedings of the First Workshop on Structured Understanding, Retrieval, and Generation in the LLM Era (SURGeLLM 2026), pages 190–208, San Diego, California, United States. Association for Computational Linguistics.
- Cite (Informal):
- DSMentor: Curriculum-Guided Inference with Online Memory for Data-Science LLM Agents (Wang et al., SURGeLLM 2026)
- PDF:
- https://preview.aclanthology.org/ingest-acl-workshops/2026.surgellm-1.12.pdf