Yam at SemEval-2026 Task 4: Failure-Driven Prompt Evolution for Narrative Comparison

Yen Yee Yam, Hong Meng Yam


Abstract
We present a structured, parameter-free system for SemEval-2026 Task 4 on Narrative Story Similarity. Instead of treating similarity as scalar embedding proximity, we align model reasoning with the task ontology by decomposing each story into abstract theme, course of action, and outcome, and performing contrastive comparison over these dimensions. Our primary contribution is a closed-loop, failure-driven prompt optimization procedure that iteratively refines concise guideline documents while keeping model parameters fixed and reverting updates that degrade performance, thereby isolating improvements attributable to structured reasoning rather than representation learning. Ontology-aligned decomposition alone achieves 70% accuracy on both train and test sets; with controlled guideline evolution, performance improves to 76% on train and 73% on test without additional supervision or fine-tuning. These results demonstrate that structured prompt optimization can meaningfully enhance contrastive narrative reasoning in a fully parameter-free setting.
Anthology ID:
2026.semeval-1.421
Volume:
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3394–3398
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.421/
DOI:
Bibkey:
Cite (ACL):
Yen Yee Yam and Hong Meng Yam. 2026. Yam at SemEval-2026 Task 4: Failure-Driven Prompt Evolution for Narrative Comparison. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 3394–3398, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
Yam at SemEval-2026 Task 4: Failure-Driven Prompt Evolution for Narrative Comparison (Yam & Yam, SemEval 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.421.pdf
Supplementarymaterial:
 2026.semeval-1.421.SupplementaryMaterial.zip