HHU-SyLo at SemEval-2026 Task 11: Logic in the Loop – Hybridizing LLMs and Theorem Provers for Robust Formal Reasoning

Wiebke Petersen, Cherine Jaziri, Diem Tran


Abstract
We present our system for SemEval-2026 Task 11 on reasoning disentanglement, separating syllogistic validity from semantic plausibility. We compare direct neural inference against two neuro-symbolic pipelines: translation to first-order logic and to syllogistic triples. By offloading inference to symbolic theorem provers, these hybrid models effectively mitigate content bias and improve logical fidelity.
Anthology ID:
2026.semeval-1.400
Volume:
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3188–3198
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.400/
DOI:
Bibkey:
Cite (ACL):
Wiebke Petersen, Cherine Jaziri, and Diem Tran. 2026. HHU-SyLo at SemEval-2026 Task 11: Logic in the Loop – Hybridizing LLMs and Theorem Provers for Robust Formal Reasoning. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 3188–3198, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
HHU-SyLo at SemEval-2026 Task 11: Logic in the Loop – Hybridizing LLMs and Theorem Provers for Robust Formal Reasoning (Petersen et al., SemEval 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.400.pdf
Supplementarymaterial:
 2026.semeval-1.400.SupplementaryMaterial.zip