@inproceedings{sanz-guerrero-etal-2026-large,
    title = "Large Language Models Are Overconfident in Their Own Responses",
    author = "Sanz-Guerrero, Mario  and
      Mager, Manuel  and
      Von Der Wense, Katharina",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1570/",
    pages = "31406--31418",
    ISBN = "979-8-89176-395-1",
    abstract = "Prior work has shown that instruction-tuned large language models (LLMs) are less well calibrated than their base pre-trained counterparts. However, little is known about the frequently used chat template{'}s effect on the calibration of conversational LLMs. In this work, we investigate the mechanisms driving this miscalibration by decoupling the effects of the post-training algorithm and the chat format. We find that, while instruction tuning fundamentally harms calibration, the chat template aggravates the issue through an ``ownership bias'' {--} models are significantly more confident in their *own* answers than in identical answers provided by a user. Extensive experiments across six recent open-weight LLMs, three benchmarks, and three confidence elicitation methods show that models assign up to 26{\%} higher confidence to their own responses. Leveraging this insight, we propose a simple inference-time strategy: framing the model{'}s answer as user input during confidence elicitation. This approach significantly reduces overconfidence and improves calibration by up to 26{\%} without the need for retraining, narrowing the gap between base and instruction-tuned models."
}