Lakksh at SemEval-2026 Task 11(1 2): Neuro-Symbolic Decomposition to Mitigate Content Bias in Syllogistic Reasoning

Lakksh Sharma, Krish Sharma, Jatin Bedi


Abstract
Syllogistic reasoning is the ability to distinguish logical validity from semantic plausibility — a setting in which LLMs succumb to frequent content bias by conflating the two. The result is a characteristic failure to recognize logically valid arguments with highly implausible conclusions and logically invalid but semantically plausible arguments. This paper introduces a neuro-symbolic system that avoids this behavior by design: neural structure extraction is strictly separated from symbolic validity checking. A T5-Small parser is trained only on synthetic nonsense-symbol syllogisms, ensuring that the structural parse is learned in the absence of real-world semantics. Validity checking is performed by a deterministic symbolic kernel operating on extracted logical form alone, ensuring that semantic content cannot influence the final call. In binary validity classification, the system achieves 97.38% accuracy with a Total Content Effect of 3.10; in the retrieval setting, it achieves 82.11% accuracy with 99.47% F1 on premise identification. Ablation experiments show that formal theorem proving via NL-to-Z3 translation actually increases content bias due to leakage in intermediate representations. The results recommend architectural separation as a promising content-robustness strategy for syllogistic reasoning.
Anthology ID:
2026.semeval-1.17
Volume:
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
115–120
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.17/
DOI:
Bibkey:
Cite (ACL):
Lakksh Sharma, Krish Sharma, and Jatin Bedi. 2026. Lakksh at SemEval-2026 Task 11(1 2): Neuro-Symbolic Decomposition to Mitigate Content Bias in Syllogistic Reasoning. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 115–120, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
Lakksh at SemEval-2026 Task 11(1 2): Neuro-Symbolic Decomposition to Mitigate Content Bias in Syllogistic Reasoning (Sharma et al., SemEval 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.17.pdf
Supplementarymaterial:
 2026.semeval-1.17.SupplementaryMaterial.zip