BertKittens at SemEval-2026 Task 3: Multi-Domain Aspect Sentiment with BERT/DeBERTa Ensembles for VA Regression and Aspect–Opinion–VA Triplets

Arseny Sukhodolsky, Ruslan Salimgareev, Tatiana Ianshina


Abstract
Our system is built on transformer encoders (BERT and DeBERTa) fine-tuned in a multi-task learning framework. For the regression subtask (evaluated with RMSE), we jointly predict Valence–Arousal (VA) scores and token-level opinion spans using a shared encoder with task-specific output heads. This formulation introduces auxiliary supervision at the token level, which stabilizes training and improves regression accuracy compared to single-task optimization.When gold abstracts and opinion annotations are provided, our models achieve strong performance. However, in fully end-to-end settings requiring automatic span extraction, performance degrades substantially due to error propagation from token-level predictions.Our findings highlight the benefits of joint affective regression and span modeling, while exposing the limitations of transformer-based sequence labeling under strict end-to-end evaluation constraints.
Anthology ID:
2026.semeval-1.380
Volume:
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3027–3034
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.380/
DOI:
Bibkey:
Cite (ACL):
Arseny Sukhodolsky, Ruslan Salimgareev, and Tatiana Ianshina. 2026. BertKittens at SemEval-2026 Task 3: Multi-Domain Aspect Sentiment with BERT/DeBERTa Ensembles for VA Regression and Aspect–Opinion–VA Triplets. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 3027–3034, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
BertKittens at SemEval-2026 Task 3: Multi-Domain Aspect Sentiment with BERT/DeBERTa Ensembles for VA Regression and Aspect–Opinion–VA Triplets (Sukhodolsky et al., SemEval 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.380.pdf
Supplementarymaterial:
 2026.semeval-1.380.SupplementaryMaterial.zip