@inproceedings{huang-etal-2025-alleviating,
    title = "Alleviating Hallucinations from Knowledge Misalignment in Large Language Models via Selective Abstention Learning",
    author = "Huang, Lei  and
      Feng, Xiaocheng  and
      Ma, Weitao  and
      Fan, Yuchun  and
      Feng, Xiachong  and
      Gu, Yuxuan  and
      Ye, Yangfan  and
      Zhao, Liang  and
      Zhong, Weihong  and
      Wang, Baoxin  and
      Wu, Dayong  and
      Hu, Guoping  and
      Kong, Lingpeng  and
      Xiao, Tong  and
      Liu, Ting  and
      Qin, Bing",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.1199/",
    pages = "24564--24579",
    ISBN = "979-8-89176-251-0",
    abstract = "Large language models (LLMs) are known to suffer from severe hallucination issues. One of the main causes lies in the knowledge misalignment between the pre-training stage and the supervised fine-tuning stage. The unfamiliar knowledge encountered during fine-tuning may encourage LLMs to generate facts that are not grounded in parametric knowledge. To address this, we propose Seal, a novel training objective with an abstention mechanism, in which the model learns to selectively reject tokens that misalign with the desired knowledge distribution via a special $\texttt{[REJ]}$ token. This allows the model the option of acknowledging the insufficiency of knowledge rather than blindly assigning high probability to all ground-truth answers. We further propose a regularized decoding objective that penalizes uncertain predictions during inference by using the $\texttt{[REJ]}$ probability learned during training. Extensive experiments on six short-form and long-form QA datasets with three LLMs of different sizes demonstrate that our method effectively alleviates hallucinations caused by knowledge misalignment. Further analysis highlights the adaptations of our method in answer refusal scenarios and its ability to effectively maintain the model{'}s instruction-following capabilities."
}