@inproceedings{zhang-etal-2024-learn,
    title = "Learn Beyond The Answer: Training Language Models with Reflection for Mathematical Reasoning",
    author = "Zhang, Zhihan  and
      Ge, Tao  and
      Liang, Zhenwen  and
      Yu, Wenhao  and
      Yu, Dian  and
      Jia, Mengzhao  and
      Yu, Dong  and
      Jiang, Meng",
    editor = "Al-Onaizan, Yaser  and
      Bansal, Mohit  and
      Chen, Yun-Nung",
    booktitle = "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2024",
    address = "Miami, Florida, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/Add-Cong-Liu-Florida-Atlantic-University-author-id/2024.emnlp-main.817/",
    doi = "10.18653/v1/2024.emnlp-main.817",
    pages = "14720--14738",
    abstract = "Supervised fine-tuning enhances the problem-solving abilities of language models across various mathematical reasoning tasks. To maximize such benefits, existing research focuses on *broadening* the training set with various data augmentation techniques, which is effective for standard single-round question-answering settings. Our work introduces a novel technique aimed at cultivating a *deeper* understanding of the training problems at hand, enhancing performance not only in standard settings but also in more complex scenarios that require reflective thinking. Specifically, we propose **reflective augmentation**, a method that embeds problem reflection into each training instance. It trains the model to consider alternative perspectives and engage with abstractions and analogies, thereby fostering a thorough comprehension through reflective reasoning. Extensive experiments validate the achievement of our aim, underscoring the unique advantages of our method and its complementary nature relative to existing augmentation techniques."
}