@inproceedings{chen-etal-2025-exploring,
    title = "Exploring the Hidden Reasoning Process of Large Language Models by Misleading Them",
    author = "Chen, Guanyu  and
      Wang, Peiyang  and
      Jiang, Yizhou  and
      Liu, Yuqian  and
      Zhao, Chujie  and
      Fang, Ying  and
      Zhang, Tianren  and
      Chen, Feng",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.1102/",
    doi = "10.18653/v1/2025.findings-emnlp.1102",
    pages = "20263--20278",
    ISBN = "979-8-89176-335-7",
    abstract = "Large language models (LLMs) have been able to perform various forms of reasoning tasks ina wide range of scenarios, but are they truly engaging in task abstraction and rule-based reasoning beyond mere memorization? To answer this question, we propose a novel experimentalapproach, Misleading Fine-Tuning (MisFT), to examine whether LLMs perform abstract reasoning by altering their original understanding of fundamental rules. In particular, by constructing datasets with math expressions or logical formulas that contradict correct principles, we fine-tune the model to learn those contradictory rules and assess its generalization ability on unseen test domains. Through a series of experiments, we find that current LLMs are capable of applying contradictory rules to solve practical math word problems and natural language reasoning tasks, implying the presence of an internal mechanism in LLMs that abstracts before reasoning."
}