@inproceedings{fu-etal-2025-quantized,
    title = "Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized {LLM}s",
    author = "Fu, Yao  and
      Long, Xianxuan  and
      Li, Runchao  and
      Yu, Haotian  and
      Sheng, Mu  and
      Han, Xiaotian  and
      Yin, Yu  and
      Li, Pan",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.emnlp-main.1548/",
    doi = "10.18653/v1/2025.emnlp-main.1548",
    pages = "30423--30446",
    ISBN = "979-8-89176-332-6",
    abstract = "Quantization enables efficient deployment of large language models (LLMs) in resource-constrained environments by significantly reducing memory and computation costs. While quantized LLMs often maintain performance on perplexity and zero-shot tasks, their impact on truthfulness{---}whether generating truthful or deceptive responses{---}remains largely unexplored. In this work, we introduce TruthfulnessEval, a comprehensive evaluation framework for assessing the truthfulness of quantized LLMs across three dimensions: (1) Truthfulness on Logical Reasoning; (2) Truthfulness on Common Sense; and (3) Truthfulness on Imitative Falsehoods. Using this framework, we examine mainstream quantization techniques (ranging from 4-bit to extreme 2-bit) across several open-source LLMs. Surprisingly, we find that while quantized models retain internally truthful representations, they are more susceptible to producing false outputs under misleading prompts. To probe this vulnerability, we test 15 rephrased variants of ``honest'', ``neutral'' and ``deceptive'' prompts and observe that ``deceptive'' prompts can override truth-consistent behavior, whereas ``honest'' and ``neutral'' prompts maintain stable outputs. Further, we reveal that quantized models ``know'' the truth internally yet still produce false outputs when guided by ``deceptive'' prompts via layer-wise probing and PCA visualizations. Our findings provide insights into future designs of quantization-aware alignment and truthfulness interventions."
}