@inproceedings{zhao-etal-2026-tokens,
    title = "What Tokens Truly Matter? The Logit Conflation Problem in {LLM} Sampling",
    author = "Zhao, Pinlong  and
      Tang, Huijun  and
      Jiao, Pengfei  and
      Li, Mengyang",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.findings-acl.1841/",
    pages = "36943--36961",
    ISBN = "979-8-89176-395-1",
    abstract = "Sampling methods for large language models select candidate tokens based on logit statistics, implicitly assuming that high logits indicate desirable outputs. We identify the Logit Conflation Problem, where a token{'}s logit aggregates prompt-independent factors, including linguistic fluency and parametric associations, with prompt-relevance. However, only prompt-relevance determines instruction-following quality. We propose SEAL-Sampling (Signal Extraction for Active ReLevance) to isolate this component through attention-weighted attribution. Our framework defines prompt-relevance as the causal effect of prompt content on token logits and establishes attention patterns as an efficient proxy. Experiments on LLaMA-3 demonstrate significant improvements over top-n{\ensuremath{\sigma}}, with gains of 1.8{\%} on AlpacaEval 2.0 and 2.2{\%} on IFEval. Furthermore, attribution scores correlate weakly with raw logits, confirming the extraction of an orthogonal signal. The method is training-free and introduces minimal latency, adding less than 12ms overhead per token."
}