@inproceedings{xu-etal-2024-pride,
    title = "Pride and Prejudice: {LLM} Amplifies Self-Bias in Self-Refinement",
    author = "Xu, Wenda  and
      Zhu, Guanglei  and
      Zhao, Xuandong  and
      Pan, Liangming  and
      Li, Lei  and
      Wang, William",
    editor = "Ku, Lun-Wei  and
      Martins, Andre  and
      Srikumar, Vivek",
    booktitle = "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest_wac_2008/2024.acl-long.826/",
    doi = "10.18653/v1/2024.acl-long.826",
    pages = "15474--15492",
    abstract = "Recent studies show that large language models (LLMs) improve their performance through self-feedback on certain tasks while degrade on others. We discovered that such a contrary is due to LLM`s bias in evaluating their own output. In this paper, we formally define LLM`s self-bias {--} the tendency to favor its own generation {--} using two statistics. We analyze six LLMs (GPT-4, GPT-3.5, Gemini, LLaMA2, Mixtral and DeepSeek) on translation, constrained text generation, and mathematical reasoning tasks. We find that self-bias is prevalent in all examined LLMs across multiple languages and tasks. Our analysis reveals that while the self-refine pipeline improves the fluency and understandability of model outputs, it further amplifies self-bias. To mitigate such biases, we discover that larger model size and external feedback with accurate assessment can significantly reduce bias in the self-refine pipeline, leading to actual performance improvement in downstream tasks. The code and data are released at https://github.com/xu1998hz/llm{\_}self{\_}bias."
}