@inproceedings{kasnavieh-etal-2026-introlm,
    title = "{I}ntro{LM}: Introspective Language Models via Prefilling-Time Self-Evaluation",
    author = "Kasnavieh, Hossein Hosseini  and
      Haffari, Gholamreza  and
      Leckie, Christopher  and
      Toosi, Adel N.",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.findings-acl.598/",
    pages = "12313--12326",
    ISBN = "979-8-89176-395-1",
    abstract = "A major challenge for the operation of large language models (LLMs) is how to predict whether a specific LLM will produce sufficiently high-quality output for a given query. Existing approaches rely on external classifiers, most commonly BERT-based models, which suffer from limited context windows, constrained representational capacity, and additional computational overhead. We propose IntroLM, a method that enables causal language models to predict their own output quality during the prefilling phase without affecting generation using [CPX] tokens. By introducing token-conditional LoRA that activates only for the introspective [CPX] token, the model learns to predict the output quality for a given query while preserving the original backbone behavior and avoiding external evaluators. On question-answering benchmarks, IntroLM applied to Qwen3-8B achieves a ROC{--}AUC of 90{\%} for success prediction, outperforming a DeBERTa-v3-Large classifier by 14{\%}. When integrated into multi-model routing systems, IntroLM achieves superior cost{--}performance trade-offs, reducing end-to-end latency by up to 33{\%} and large-model usage by up to 50{\%} at matched reliability. Our code is available at https://github.com/hhosseini1377/LLM{\_}routing."
}