@inproceedings{rrv-etal-2025-thinktuning,
    title = "{T}hink{T}uning: Instilling Cognitive Reflections without Distillation",
    author = "Rrv, Aswin  and
      Dineen, Jacob  and
      Handa, Divij  and
      Uddin, Md Nayem  and
      Parmar, Mihir  and
      Baral, Chitta  and
      Zhou, Ben",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.1592/",
    pages = "31236--31250",
    ISBN = "979-8-89176-332-6",
    abstract = "Recent advances in test-time scaling have led to the emergence of thinking LLMs that exhibit self-reflective behaviors and multi-step reasoning. While RL drives this self-improvement paradigm, recent studies show that solely RL does not truly instill these new reasoning abilities - it merely draws out behaviors already present in the base models. This raises a question: How can we train models that don{'}t exhibit such thinking behavior to develop it in the first place? To this end, we propose ThinkTuning, a GRPO-based interactive training approach where we augment the rollouts of a student model with the guidance from a teacher model. A simple idea from classroom practice inspires our method: a teacher poses a problem, lets the student try an answer, then gives corrective feedback{--}enough to point the mind in the right direction and then show the solution. Each feedback reshapes the student{'}s thoughts, leading them to arrive at the correct solution. Similarly, we find that this type of implicit supervision through feedback from a teacher model of the same size improves the reasoning capabilities of the student model. Particularly, on average, our method shows 3.69{\%} improvement over zero-shot baselines across benchmarks, and on MATH-500 and GPQA-Diamond, it shows 2.08{\%} and 3.99{\%} improvement over the vanilla-GRPO baseline."
}