@inproceedings{fang-etal-2026-knowledge,
    title = "Knowledge Injection Exists in {M}o{E}? Exploring Expert-Aware Contrast Decoding in {M}o{E} for Mitigating {LLM}s' Hallucinations",
    author = "Fang, Xinyue  and
      Tian, Zhiliang  and
      Huang, Zhen  and
      Pan, Ziyi  and
      Wen, Zhihua  and
      Wang, Xi  and
      Fang, Quntian  and
      Li, Dongsheng",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1824/",
    pages = "39326--39343",
    ISBN = "979-8-89176-390-6",
    abstract = "Existing LLM hallucination mitigation methods, including prompt engineering and model optimization, either hardly alter models' internal knowledge or have poor cross-domain generalization. Contrastive decoding mitigates hallucinations by using layer-wise differences in LLMs. However, prior studies only explore transformer-based models (e.g., GPT), ignoring other effective frameworks like mixture-of-experts (MoE) models. Since MoE alters the traditional transformer architecture, we conduct empirical studies to investigate whether similar layer-wise differences exist in MoEs. Our results show that they do not exist in MoE with shared experts; nevertheless, across different MoEs, higher layers exhibit distinct expert activation patterns between factual and non-factual outputs. Building on these, we propose EAACD, an expert-aware adaptive contrast decoding that uses expert differences in MoE{'}s higher layers to mitigate hallucinations on QA tasks. EAACD splits high-layer experts into a higher-reliability group and several lower-reliability groups based on their confidence and consistency. It contrasts the higher-reliability group{'}s prediction with each lower-reliability group{'}s prediction to calibrate the model{'}s original predictions. To strengthen this contrast, EAACD amplifies hallucinations from lower-reliability experts via attention and masking to provide stronger negative references. EAACD outperforms all baselines on four datasets"
}