hermeneutichools at SemEval-2026 Task 4: Multiperspectivity as a Resource for Narrative Similarity Prediction
Max Upravitelev, Veronika Solopova, Jing Yang, Charlott Jakob, Premtim Sahitaj, Ariana Sahitaj, Vera Schmitt
Abstract
Predicting narrative similarity can be under-stood as an inherently interpretive task: differ-ent, equally valid readings of the same text canproduce divergent interpretations and thus dif-ferent similarity judgments, posing a fundamen-tal challenge for semantic evaluation bench-marks that encode a single ground truth. Ratherthan treating this multiperspectivity as a chal-lenge to overcome, we propose to incorporateit in the decision making process of predic-tive systems. To explore this strategy, we cre-ated an ensemble of 31 LLM personas. Theserange from practitioners following interpretiveframeworks to more intuitive, lay-style charac-ters. Our experiments were conducted on theSemEval-2026 Task 4 dataset, where the sys-tem ranked 13th out of 47 teams and achievedan accuracy score of 0.705. Accuracy improveswith ensemble size, consistent with CondorcetJury Theorem-like dynamics under weakenedindependence. Practitioner personas performworse individually but produce less correlatederrors, yielding larger ensemble gains undermajority voting. Our error analysis reveals aconsistent negative association between gender-focused interpretive vocabulary and accuracyacross all persona categories, suggesting ei-ther attention to dimensions not relevant for thebenchmark or valid interpretations absent fromthe ground truth. This finding underscores theneed for evaluation frameworks that accountfor interpretive plurality.- Anthology ID:
- 2026.semeval-1.230
- Volume:
- Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
- Month:
- July
- Year:
- 2026
- Address:
- San Diego, California, USA
- Editors:
- Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
- Venues:
- SemEval | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1817–1831
- Language:
- URL:
- https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.230/
- DOI:
- Cite (ACL):
- Max Upravitelev, Veronika Solopova, Jing Yang, Charlott Jakob, Premtim Sahitaj, Ariana Sahitaj, and Vera Schmitt. 2026. hermeneutichools at SemEval-2026 Task 4: Multiperspectivity as a Resource for Narrative Similarity Prediction. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 1817–1831, San Diego, California, USA. Association for Computational Linguistics.
- Cite (Informal):
- hermeneutichools at SemEval-2026 Task 4: Multiperspectivity as a Resource for Narrative Similarity Prediction (Upravitelev et al., SemEval 2026)
- PDF:
- https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.230.pdf