@inproceedings{liang-etal-2026-reasoning,
    title = "Your Reasoning Model is Secretly a Reward Model - Optimization-Free Verification from Experience",
    author = "Liang, Zhenwen  and
      Li, Ruosen  and
      Zhou, Yujun  and
      Song, Linfeng  and
      Yu, Dian  and
      Du, Xinya  and
      Mi, Haitao  and
      Yu, Dong",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.788/",
    pages = "17358--17372",
    ISBN = "979-8-89176-390-6",
    abstract = "Assessing the quality of Large Language Model (LLM) outputs becomes especially challenging in high-branching settings, where a single prompt yields many plausible candidates. Existing verifiers typically operate on the surface text (e.g., reward models, LLM judges, majority voting) or on confidence proxies derived from token probabilities, both of which can be brittle: the former can be influenced by stylistic artifacts, while the latter is often miscalibrated. In this paper, we study a third source of information{---}the model{'}s hidden states{---}for \textit{binary correctness verification} in tasks with a reliable success/failure signal (e.g., deterministic checkers or reference-grounded answers). We find that correct and incorrect solutions exhibit measurable geometric differences in their hidden-state trajectories. To isolate this signal with minimal modeling assumptions, we introduce \textbf{Clue (Clustering and Experience-based Verification)}, a training-free, non-parametric verifier. Clue summarizes each reasoning trace by an \textit{activation delta}{---}the difference between hidden states at the start and end of the explicit reasoning span{---}and predicts correctness by comparing this delta to two class centroids computed from labeled experience. Across math (AIME 24/25), scientific QA (GPQA), and a multi-domain benchmark (WebInstruct-verified), Clue improves selection and reranking, with particularly strong gains on smaller or less-calibrated models. For example, on AIME 24 with a 1.5B model, Clue raises accuracy from 56.7{\%} (majority@64) to 70.0{\%} (top-maj@16)."
}