Sahara Tokenizers at PARSEME 2.0 Subtask 1: Combining Contextual Embeddings with Structural Decoding for Multi-Word Expression Detection
Yunus Karatepe, Mert Sülük, Zeynep Tuğçe Kırımlı, Begüm Özbay
Abstract
Multi-Word Expressions (MWEs) pose a significant challenge for natural language processing systems due to their idiosyncratic semantic and syntactic properties. This paper describes our system for the PARSEME 2.0 Shared Task on automatic identification of verbal MWEs across 17 typologically diverse languages. Our approach combines multilingual BERT with explicit Part-of-Speech (POS) feature injection through a dual-head architecture that jointly performs BIO-based identification and category classification. We further investigate extensions, including Conditional Random Field (CRF) decoding for structured prediction, focal loss for addressing class imbalance, and model ensembling for improving discontinuous MWE detection. Our official submission achieves a global MWE-based F1 score of 48.39%, securing second place in the shared task. Ablation studies reveal a strong synergy between POS features and CRF decoding, with the combined approach yielding the best single-model performance. Furthermore, ensembling models trained with different objectives improves both overall F1 score and discontinuous MWE scores, demonstrating the importance of training diversity for capturing non-adjacent syntactic patterns.- Anthology ID:
- 2026.mwe-1.20
- Volume:
- Proceedings of the 22nd Workshop on Multiword Expressions (MWE 2026)
- Month:
- March
- Year:
- 2026
- Address:
- Rabat, Marocco
- Editors:
- Atul Kr. Ojha, Verginica Barbu Mititelu, Mathieu Constant, Ivelina Stoyanova, A. Seza Doğruöz, Alexandre Rademaker
- Venues:
- MWE | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 154–159
- Language:
- URL:
- https://preview.aclanthology.org/ingest-eacl/2026.mwe-1.20/
- DOI:
- Cite (ACL):
- Yunus Karatepe, Mert Sülük, Zeynep Tuğçe Kırımlı, and Begüm Özbay. 2026. Sahara Tokenizers at PARSEME 2.0 Subtask 1: Combining Contextual Embeddings with Structural Decoding for Multi-Word Expression Detection. In Proceedings of the 22nd Workshop on Multiword Expressions (MWE 2026), pages 154–159, Rabat, Marocco. Association for Computational Linguistics.
- Cite (Informal):
- Sahara Tokenizers at PARSEME 2.0 Subtask 1: Combining Contextual Embeddings with Structural Decoding for Multi-Word Expression Detection (Karatepe et al., MWE 2026)
- PDF:
- https://preview.aclanthology.org/ingest-eacl/2026.mwe-1.20.pdf