hermeneutichools at SemEval-2026 Task 4: Multiperspectivity as a Resource for Narrative Similarity Prediction

Max Upravitelev, Veronika Solopova, Jing Yang, Charlott Jakob, Premtim Sahitaj, Ariana Sahitaj, Vera Schmitt


Abstract
Predicting narrative similarity can be under-stood as an inherently interpretive task: differ-ent, equally valid readings of the same text canproduce divergent interpretations and thus dif-ferent similarity judgments, posing a fundamen-tal challenge for semantic evaluation bench-marks that encode a single ground truth. Ratherthan treating this multiperspectivity as a chal-lenge to overcome, we propose to incorporateit in the decision making process of predic-tive systems. To explore this strategy, we cre-ated an ensemble of 31 LLM personas. Theserange from practitioners following interpretiveframeworks to more intuitive, lay-style charac-ters. Our experiments were conducted on theSemEval-2026 Task 4 dataset, where the sys-tem ranked 13th out of 47 teams and achievedan accuracy score of 0.705. Accuracy improveswith ensemble size, consistent with CondorcetJury Theorem-like dynamics under weakenedindependence. Practitioner personas performworse individually but produce less correlatederrors, yielding larger ensemble gains undermajority voting. Our error analysis reveals aconsistent negative association between gender-focused interpretive vocabulary and accuracyacross all persona categories, suggesting ei-ther attention to dimensions not relevant for thebenchmark or valid interpretations absent fromthe ground truth. This finding underscores theneed for evaluation frameworks that accountfor interpretive plurality.
Anthology ID:
2026.semeval-1.230
Volume:
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1817–1831
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.230/
DOI:
Bibkey:
Cite (ACL):
Max Upravitelev, Veronika Solopova, Jing Yang, Charlott Jakob, Premtim Sahitaj, Ariana Sahitaj, and Vera Schmitt. 2026. hermeneutichools at SemEval-2026 Task 4: Multiperspectivity as a Resource for Narrative Similarity Prediction. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 1817–1831, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
hermeneutichools at SemEval-2026 Task 4: Multiperspectivity as a Resource for Narrative Similarity Prediction (Upravitelev et al., SemEval 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.230.pdf
Supplementarymaterial:
 2026.semeval-1.230.SupplementaryMaterial.zip