Abstract
Dialectal variation is present in many human languages and is attracting a growing interest in NLP. Most previous work concentrated on either (1) classifying dialectal varieties at the document or sentence level or (2) performing standard NLP tasks on dialectal data. In this paper, we propose the novel task of token-level dialectal feature prediction. We present a set of fine-grained annotation guidelines for Norwegian dialects, expand a corpus of dialectal tweets, and manually annotate them using the introduced guidelines. Furthermore, to evaluate the learnability of our task, we conduct labeling experiments using a collection of baselines, weakly supervised and supervised sequence labeling models. The obtained results show that, despite the difficulty of the task and the scarcity of training data, many dialectal features can be predicted with reasonably high accuracy.- Anthology ID:
- 2023.nodalida-1.16
- Volume:
- Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa)
- Month:
- May
- Year:
- 2023
- Address:
- Tórshavn, Faroe Islands
- Editors:
- Tanel Alumäe, Mark Fishel
- Venue:
- NoDaLiDa
- SIG:
- Publisher:
- University of Tartu Library
- Note:
- Pages:
- 146–158
- Language:
- URL:
- https://aclanthology.org/2023.nodalida-1.16
- DOI:
- Cite (ACL):
- Jeremy Barnes, Samia Touileb, Petter Mæhlum, and Pierre Lison. 2023. Identifying Token-Level Dialectal Features in Social Media. In Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa), pages 146–158, Tórshavn, Faroe Islands. University of Tartu Library.
- Cite (Informal):
- Identifying Token-Level Dialectal Features in Social Media (Barnes et al., NoDaLiDa 2023)
- PDF:
- https://preview.aclanthology.org/corrections-2024-05/2023.nodalida-1.16.pdf