@inproceedings{dinucu-jianu-etal-2025-problem,
    title = "From Problem-Solving to Teaching Problem-Solving: Aligning {LLM}s with Pedagogy using Reinforcement Learning",
    author = "Dinucu-Jianu, David  and
      Macina, Jakub  and
      Daheim, Nico  and
      Hakimi, Ido  and
      Gurevych, Iryna  and
      Sachan, Mrinmaya",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.15/",
    pages = "272--292",
    ISBN = "979-8-89176-332-6",
    abstract = "Large language models (LLMs) can transform education, but their optimization for direct question-answering often undermines effective pedagogy which requires strategically withholding answers. To mitigate this, we propose an online reinforcement learning (RL)-based alignment framework that can quickly adapt LLMs into effective tutors using simulated student-tutor interactions by emphasizing pedagogical quality and guided problem-solving over simply giving away answers. We use our method to train a 7B parameter tutor model without human annotations which reaches similar performance to larger proprietary models like LearnLM. We introduce a controllable reward weighting to balance pedagogical support and student solving accuracy, allowing us to trace the Pareto frontier between these two objectives. Our models better preserve reasoning capabilities than single-turn SFT baselines and can optionally enhance interpretability through thinking tags that expose the model{'}s instructional planning."
}