@inproceedings{yang-etal-2025-llm-reasoning,
    title = "How Is {LLM} Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark",
    author = "Yang, Minglai  and
      Huang, Ethan  and
      Zhang, Liang  and
      Surdeanu, Mihai  and
      Wang, William Yang  and
      Pan, Liangming",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.674/",
    pages = "13340--13358",
    ISBN = "979-8-89176-332-6",
    abstract = "We introduce Grade School Math with Distracting Context (GSM-DC), a synthetic benchmark to evaluate Large Language Models' (LLMs) reasoning robustness against systematically controlled irrelevant context (IC). GSM-DC constructs symbolic reasoning graphs with precise distractor injections, enabling rigorous, reproducible evaluation. Our experiments demonstrate that LLMs are significantly sensitive to IC, affecting both reasoning path selection and arithmetic accuracy. Additionally, training models with strong distractors improves performance in both in-distribution and out-of-distribution scenarios. We further propose a stepwise tree search guided by a process reward model, which notably enhances robustness in out-of-distribution conditions."
}