dutirshlee at SemEval-2026 Task 11: Symbolic Augmentation for Content-Bias-Resistant Syllogistic Reasoning

Songhuan Li, Liang Yang, Shengdi Yin, Qiang Zhang, Hongfei Lin


Abstract
We describe our system for SemEval-2026 Task 11 Subtask 1 (English syllogistic validity). Our approach fine-tunes Qwen2.5-7B-Instruct with LoRA and a symbolic data augmentation (SDA) scheme that replaces real-world entities with abstract placeholders, explicitly decoupling logical form from content. The resulting model achieves 96.34% accuracy and a total content effect (TCE) of 2.15, yielding a primary score of 44.86. We provide detailed ablations and negative results (prompting, self consistency, contrastive decoding, structured chain-of-thought, andDPO)tocharacterizewhy direct LoRA training with SDA is the most ro bust configuration for this task. Finally, we use a specialist–generalist complementarity setting where a strong API model (ACC 99.48, TCE 1.06, score 57.68) is corrected by the SDA spe cialist on a single disagreement, producing a merged output with ACC 100 and TCE 0.
Anthology ID:
2026.semeval-1.73
Volume:
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
509–513
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.73/
DOI:
Bibkey:
Cite (ACL):
Songhuan Li, Liang Yang, Shengdi Yin, Qiang Zhang, and Hongfei Lin. 2026. dutirshlee at SemEval-2026 Task 11: Symbolic Augmentation for Content-Bias-Resistant Syllogistic Reasoning. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 509–513, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
dutirshlee at SemEval-2026 Task 11: Symbolic Augmentation for Content-Bias-Resistant Syllogistic Reasoning (Li et al., SemEval 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.73.pdf
Supplementarymaterial:
 2026.semeval-1.73.SupplementaryMaterial.tex