Diagnosing and Mitigating Sycophancy and Skepticism in LLM Causal Judgment

Edward Y Chang


Abstract
Do frontier LLMs reason causally, or do they pattern-match, yielding under pressure and hedging under uncertainty? We frame causal judgment as evaluation along three axes, Utility, Safety, and Wise Refusal, across Pearl’s Ladder. We introduce Recursive Causal Audit (RCA), a process-integrity evaluator whose Judge has no access to gold labels: it checks whether a model’s answer is entailed by itsown derivation, internally consistent, and not dominated by user hints under pressure. RCA unifies persona and pressure: prompt tone is the intervention that regulates pressure-induced drift. For fine diagnostic resolution we use CAUSALT3, with explicit trap families and standardized pressure protocols. CAUSALT3 reveals a Skepticism Trap (Claude Haiku rejects 60% of valid L1 links) and a Scaling Paradox (GPT-5.2 underperforms GPT-4-Turbo by 55 points on L3, driven by paralysis rather than hallucination). Under RCA, operating points shift toward the high-Utility, high-Safety quadrant without retraining, consistent with much of the observed failure arising from how answers are rendered under pressure rather than from missing causal knowledge.
Anthology ID:
2026.findings-acl.427
Volume:
Findings of the Association for Computational Linguistics: ACL 2026
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
8769–8789
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.427/
DOI:
Bibkey:
Cite (ACL):
Edward Y Chang. 2026. Diagnosing and Mitigating Sycophancy and Skepticism in LLM Causal Judgment. In Findings of the Association for Computational Linguistics: ACL 2026, pages 8769–8789, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
Diagnosing and Mitigating Sycophancy and Skepticism in LLM Causal Judgment (Chang, Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.427.pdf
Checklist:
 2026.findings-acl.427.checklist.pdf