Zhiyu Lu
2026
Immediate Inference: The Missing Foundation in Large Language Model Logical Reasoning
Sihang Jiang | Zhiyu Lu | Keyi Wang | Jiaqing Liang | Yanghua Xiao | Xiaojun Meng | Jiansheng Wei
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Sihang Jiang | Zhiyu Lu | Keyi Wang | Jiaqing Liang | Yanghua Xiao | Xiaojun Meng | Jiansheng Wei
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
While extensive research has evaluated LLMs on complex reasoning tasks, the foundational building blocks of logical reasoning remain underexplored. We introduce IIBench, a benchmark evaluating immediate inference (elementary operations over categorical propositions). Our evaluation reveals that even SoTA models exhibit systematic deficiencies in immediate inference, and establishes immediate inference as foundational: it mediates approximately 40% of the effect on syllogistic reasoning, with near-perfect correlation ( = 0.98) across reasoning benchmarks. Our analysis reveals that models lack robust operator grounding, oscillating between structural reasoning and surface pattern matching with inconsistent handling of quantifiers and negation.