@inproceedings{liu-etal-2026-pretrainrl,
    title = "{P}retrain{RL}: Alleviating Factuality Hallucination of Large Language Models at the Beginning",
    author = "Liu, Langming  and
      Lv, Kangtao  and
      Chen, Haibin  and
      Zhang, Weidong  and
      Wang, Yejing  and
      Liu, Shilei  and
      Tong, Xin  and
      Yuan, Yujin  and
      Wang, Yongwei  and
      Su, Wenbo  and
      Zheng, Bo",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.910/",
    pages = "18298--18310",
    ISBN = "979-8-89176-395-1",
    abstract = "Large language models (LLMs), despite their powerful capabilities, suffer from factual hallucinations where they generate verifiable falsehoods. We identify a root of this issue: the imbalanced data distribution in the pretraining corpus, which leads to a state of ``low-probability truth'' and ``high-probability falsehood''. Recent approaches, such as teaching models to say ``I don{'}t know'' or post-hoc knowledge editing, either evade the problem or face catastrophic forgetting. To address this issue from its root, we propose PretrainRL, a novel framework that integrates reinforcement learning into the pretraining phase to consolidate factual knowledge. The core principle of PretrainRL is ``debiasing then learning.'' It actively reshapes the model{'}s probability distribution by down-weighting high-probability falsehoods, thereby making ``room'' for low-probability truths to be learned effectively. To enable this, we design an efficient negative sampling strategy to discover these high-probability falsehoods and introduce novel metrics to evaluate the model{'}s probabilistic state concerning factual knowledge. Extensive experiments on three public benchmarks demonstrate that PretrainRL significantly alleviates factual hallucinations and outperforms state-of-the-art methods."
}