Narrative Team at SemEval-2026 Task 5: Rating Plausibility of Word Senses in Ambiguous Sentences through Narrative Understanding

Valentin Istrate, Mocanu Octavian, Tatiana Khaidukova


Abstract
This paper describes our system for SemEval-2026 Task 5, which focuses on predicting the plausibility of word senses in ambiguous narrative contexts. The task requires assigning a real-valued plausibility score to candidate word senses based on aggregated human judgments. Our approach compares two modeling paradigms: (i) a pretrained transformer-based regression model using DistilBERT fine-tuned on the task data, and (ii) a lightweight neural baseline based on a bidirectional LSTM trained either from scratch or initialized with GloVe embeddings. Input representations combine a candidate sense definition with the narrative context and target sentence, separated by a special token. On the official test set, the DistilBERT model achieves the strongest result among our submissions, with an Acc@SD score of 0.54 and Spearman correlation of 0.17, while the best BiLSTM submission reaches 0.52 Acc@SD and 0.02 Spearman correlation. Although DistilBERT performs best in our experiments, the recurrent baseline remains competitive under the tolerance-based metric. We discuss model variants, reproducibility details, and limitations of our analysis.
Anthology ID:
2026.semeval-1.148
Volume:
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1089–1093
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.148/
DOI:
Bibkey:
Cite (ACL):
Valentin Istrate, Mocanu Octavian, and Tatiana Khaidukova. 2026. Narrative Team at SemEval-2026 Task 5: Rating Plausibility of Word Senses in Ambiguous Sentences through Narrative Understanding. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 1089–1093, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
Narrative Team at SemEval-2026 Task 5: Rating Plausibility of Word Senses in Ambiguous Sentences through Narrative Understanding (Istrate et al., SemEval 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.148.pdf
Supplementarymaterial:
 2026.semeval-1.148.SupplementaryMaterial.zip