Anchoring Depends on Confidence and Post-Training in Language Models

Hillary N. Owusu, Naomi H. Feldman


Abstract
Anchoring bias causes large language models (LLMs) to shift quantitative judgments in response to irrelevant numerical primes. We analyze this bias as a function of model confidence and accuracy in base, instruction-tuned, and distilled variants of Llama and Qwen models. We find that anchoring susceptibility is negatively correlated with model confidence without regard to accuracy: confidently incorrect models resist anchoring as effectively as accurate ones, provided their internal priors are sufficiently strong. We further show that post-training impacts the strength of this relationship, and that models are more susceptible to high anchors than to low anchors. Our findings suggest anchoring resistance is a structural property of distributional concentration (certainty) rather than knowledge correctness (factual accuracy), with implications for deploying LLMs in numerical reasoning tasks.
Anthology ID:
2026.acl-short.16
Volume:
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
174–180
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.acl-short.16/
DOI:
Bibkey:
Cite (ACL):
Hillary N. Owusu and Naomi H. Feldman. 2026. Anchoring Depends on Confidence and Post-Training in Language Models. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 174–180, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
Anchoring Depends on Confidence and Post-Training in Language Models (Owusu & Feldman, ACL 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.acl-short.16.pdf
Checklist:
 2026.acl-short.16.checklist.pdf