@inproceedings{xu-etal-2025-self-ensemble,
    title = "Self-Ensemble: Mitigating Confidence Distortion for Large Language Models",
    author = "Xu, Zicheng  and
      Wang, Guanchu  and
      Zheng, Guangyao  and
      Chuang, Yu-Neng  and
      Szalay, Alex  and
      Hu, Xia  and
      Braverman, Vladimir",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.902/",
    doi = "10.18653/v1/2025.findings-emnlp.902",
    pages = "16603--16615",
    ISBN = "979-8-89176-335-7",
    abstract = "Although Large Language Models (LLMs) perform well in general fields, they exhibit a **confidence distortion problem** on multi-choice question-answering (MCQA), particularly as the number of answer choices increases. Specifically, on MCQA with many choices, LLMs suffer from under-confidence in correct predictions and over-confidence in incorrect ones, leading to a substantially degraded performance. To solve this problem, we propose Self-Ensemble in this work. Our method splits the choices into several groups and ensembles LLM predictions across these groups to reach a final decision. The advantage of Self-Ensemble is its plug-and-play nature, where it can be integrated into existing LLM architecture based on a designed attention mask and positional encoding, without requiring labeled datasets for parameter tuning. Experimental results on three LLMs and datasets demonstrate that Self-Ensemble comprehensively addresses the confidence distortion problem of LLMs, outperforming standard inference as well as baseline methods."
}