Yuan Cao
Other people with similar names: Yuan Cao
Unverified author pages with similar names: Yuan Cao
2026
Through a Compressed Lens: Investigating The Impact of Quantization on Factual Knowledge Recall
Qianli Wang | Mingyang Wang | Nils Feldhus | Simon Ostermann | Yuan Cao | Hinrich Schuetze | Sebastian Möller | Vera Schmitt
Proceedings of the 6th Workshop on Trustworthy NLP (TrustNLP 2026)
Qianli Wang | Mingyang Wang | Nils Feldhus | Simon Ostermann | Yuan Cao | Hinrich Schuetze | Sebastian Möller | Vera Schmitt
Proceedings of the 6th Workshop on Trustworthy NLP (TrustNLP 2026)
Quantization methods are widely used to accelerate inference and streamline the deployment of large language models (LLMs). Although quantization’s effects on various LLM capabilities have been extensively studied, one critical area remains underexplored: factual knowledge recall (FKR), the process by which LLMs access stored knowledge. To this end, we conduct comprehensive experiments using three common quantization techniques at distinct bit widths, in conjunction with interpretability-driven analyses on two tasks, knowledge memorization and latent multi-hop reasoning. We show that quantization typically results in information loss within LLMs, consequently diminishing their capacity for FKR. This effect is particularly amplified in smaller models within the same architectural families. However, models quantized at reduced bit precision do not consistently exhibit inferior performance and occasionally quantization may even enhance model FKR. We find that BitSandBytes demonstrates highest preservation of the original full-precision model’s FKR. Despite variability across models and methods, quantization causes modest performance degradation and remains an effective compression strategy.