@inproceedings{lin-etal-2025-step,
    title = "Step-{KTO}: Optimizing Mathematical Reasoning through Stepwise Binary Feedback",
    author = "Lin, Yen-Ting  and
      Jin, Di  and
      Xu, Tengyu  and
      Wu, Tianhao  and
      Sukhbaatar, Sainbayar  and
      Zhu, Chen  and
      He, Yun  and
      Chen, Yun-Nung  and
      Weston, Jason E  and
      Tian, Yuandong  and
      Rahnama, Arash  and
      Wang, Sinong  and
      Ma, Hao  and
      Fang, Han",
    editor = "Valentino, Marco  and
      Ferreira, Deborah  and
      Thayaparan, Mokanarangan  and
      Ranaldi, Leonardo  and
      Freitas, Andre",
    booktitle = "Proceedings of The 3rd Workshop on Mathematical Natural Language Processing (MathNLP 2025)",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.mathnlp-main.2/",
    pages = "15--33",
    ISBN = "979-8-89176-348-7",
    abstract = "Large language models (LLMs) have recently demonstrated remarkable success in mathematical reasoning. Despite progress in methods like chain-of-thought prompting and self-consistency sampling, these advances often focus on final correctness without ensuring that the underlying reasoning process is coherent and reliable. This paper introduces Step-KTO, a training framework that combines process-level and outcome-level binary feedback to guide LLMs toward more trustworthy reasoning trajectories. By providing binary evaluations for both the intermediate reasoning steps and the final answer, Step-KTO encourages the model to adhere to logical progressions rather than relying on superficial shortcuts. Our experiments on challenging mathematical benchmarks show that Step-KTO significantly improves both final answer accuracy and the quality of intermediate reasoning steps. For example, on the MATH-500 dataset, Step-KTO achieves a notable improvement in Pass@1 accuracy over strong baselines. These results highlight the promise of integrating stepwise process feedback into LLM training, paving the way toward more interpretable and dependable reasoning capabilities."
}