YNU-NLP at SemEval-2026 Task 11: A Neuro-Symbolic Approach with Reflexion Mechanism Disentangling Content and Formal Reasoning in Language Models

Yu Wang, You Zhang, Hao Zhang, Dan Xu


Abstract
This paper describes our systems for SemEval-2026 Task 11, Disentangling Content and Formal Reasoning in Language Models. We participated in all four subtasks, addressing the Content Effect-a phenomenon where models rely on real-world plausibility rather than logical validity. Existing methods, such as standard Chain-of-Thought (CoT) prompting or single-task Supervised Fine-Tuning (SFT), often struggle to completely decouple content from reasoning due to the inherent probabilistic biases in pre-trained models. To address these limitations, a hybrid neuro-symbolic framework based on the Qwen2.5-14B architecture is proposed, integrating multi-task instruction tuning with a robust neuro-symbolic pipeline. The principal innovation lies in the deployment of a Reflexion mechanism coupled with formal verification: natural language arguments are parsed into First-Order Logic (FOL) and subsequently verified by the Z3 Theorem Prover. Parsing anomalies are automatically rectified through an iterative self-correction module. The proposed system ranked 1st in Subtasks 1 & 2, 2nd in Subtask 4, and 9th in Subtask 3, validating its ability to decouple logical validity from content plausibility.
Anthology ID:
2026.semeval-1.126
Volume:
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
919–926
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.126/
DOI:
Bibkey:
Cite (ACL):
Yu Wang, You Zhang, Hao Zhang, and Dan Xu. 2026. YNU-NLP at SemEval-2026 Task 11: A Neuro-Symbolic Approach with Reflexion Mechanism Disentangling Content and Formal Reasoning in Language Models. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 919–926, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
YNU-NLP at SemEval-2026 Task 11: A Neuro-Symbolic Approach with Reflexion Mechanism Disentangling Content and Formal Reasoning in Language Models (Wang et al., SemEval 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.126.pdf