@inproceedings{fan-etal-2026-ketchup,
    title = "{KETCHUP}: K-Step Return Estimation for Sequential Knowledge Distillation",
    author = "Fan, Jiabin  and
      Luo, Guoqing  and
      Bowling, Michael  and
      Mou, Lili",
    editor = "Demberg, Vera  and
      Inui, Kentaro  and
      Marquez, Llu{\'i}s",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {EACL} 2026",
    month = mar,
    year = "2026",
    address = "Rabat, Morocco",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-eacl/2026.findings-eacl.39/",
    pages = "778--796",
    ISBN = "979-8-89176-386-9",
    abstract = "We propose a novel $K$-step return estimation method (called $K$ETCHUP) for Reinforcement Learning (RL)-based knowledge distillation (KD) in text generation tasks. Our idea is to induce a $K$-step return by using the Bellman Optimality Equation for multiple steps. Theoretical analysis shows that this $K$-step formulation reduces the variance of the gradient estimates, thus leading to improved RL optimization, especially when the student model size is large. Empirical evaluation on three text generation tasks demonstrates that our approach yields superior performance in both standard task metrics and large language model (LLM)-based evaluation. These results suggest that our $K$-step return induction offers a promising direction for enhancing RL-based KD in LLM research."
}