@inproceedings{akinfaderin-diallo-2026-fregelogic,
    title = "{F}rege{L}ogic at {S}em{E}val 2026 Task 11: A Hybrid Neuro-Symbolic Architecture for Content-Robust Syllogistic Validity Prediction",
    author = "Akinfaderin, Adewale  and
      Diallo, Nafi",
    editor = "Kochmar, Ekaterina  and
      Ghosh, Debanjan  and
      North, Kai  and
      Komachi, Mamoru",
    booktitle = "Proceedings of the 20th {I}nternational {W}orkshop on {S}emantic {E}valuation (2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.329/",
    pages = "2611--2620",
    ISBN = "979-8-89176-414-9",
    abstract = "We present FregeLogic, a hybrid neuro-symbolic system for SemEval-2026 Task 11 (Subtask 1), which addresses syllogistic validity prediction while reducing content effects on predictions. Our approach combines an ensemble of five LLM classifiers, spanning three open-weights models (Llama 4 Maverick, Llama 4 Scout, and Qwen3-32B) paired with varied prompting strategies, with a Z3 SMT solver that serves as a formal logic tiebreaker. The central hypothesis is that LLM disagreement within the ensemble signals likely content-biased errors, where real-world believability interferes with logical judgment. By deferring to Z3{'}s structurally-grounded formal verification on these disputed cases, our system achieves 94.3{\%} accuracy with a content effect of 2.85 and a combined score of 41.88 in nested 5-fold cross-validation on the dataset (N = 960). This represents a 2.76-point improvement in combined score over the pure ensemble (39.12), with a 0.9{\%} accuracy gain, driven by a 16{\%} reduction in content effect (3.39{\textrightarrow}2.85). Adopting structured-output API calls for Z3 extraction reduced failure rates from {\ensuremath{\sim}}22{\%} to near zero, and an Aristotelian encoding with existence axioms was validated against task annotations. Our results suggest that targeted neuro-symbolic integration, applying formal methods precisely where ensemble consensus is lowest, can improve the combined accuracy-plus-content-effect metric used by this task."
}