@inproceedings{torge-etal-2023-named,
    title = "Named Entity Recognition for Low-Resource Languages - Profiting from Language Families",
    author = "Torge, Sunna  and
      Politov, Andrei  and
      Lehmann, Christoph  and
      Saffar, Bochra  and
      Tao, Ziyan",
    editor = "Piskorski, Jakub  and
      Marci{\'n}czuk, Micha{\l}  and
      Nakov, Preslav  and
      Ogrodniczuk, Maciej  and
      Pollak, Senja  and
      P{\v{r}}ib{\'a}{\v{n}}, Pavel  and
      Rybak, Piotr  and
      Steinberger, Josef  and
      Yangarber, Roman",
    booktitle = "Proceedings of the 9th Workshop on Slavic Natural Language Processing 2023 (SlavicNLP 2023)",
    month = may,
    year = "2023",
    address = "Dubrovnik, Croatia",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.bsnlp-1.1",
    doi = "10.18653/v1/2023.bsnlp-1.1",
    pages = "1--10",
    abstract = "Machine learning drives forward the development in many areas of Natural Language Processing (NLP). Until now, many NLP systems and research are focusing on high-resource languages, i.e. languages for which many data resources exist. Recently, so-called low-resource languages increasingly come into focus. In this context, multi-lingual language models, which are trained on related languages to a target low-resource language, may enable NLP tasks on this low-resource language. In this work, we investigate the use of multi-lingual models for Named Entity Recognition (NER) for low-resource languages. We consider the West Slavic language family and the low-resource languages Upper Sorbian and Kashubian. Three RoBERTa models were trained from scratch, two mono-lingual models for Czech and Polish, and one bi-lingual model for Czech and Polish. These models were evaluated on the NER downstream task for Czech, Polish, Upper Sorbian, and Kashubian, and compared to existing state-of-the-art models such as RobeCzech, HerBERT, and XLM-R. The results indicate that the mono-lingual models perform better on the language they were trained on, and both the mono-lingual and language family models outperform the large multi-lingual model in downstream tasks. Overall, the study shows that low-resource West Slavic languages can benefit from closely related languages and their models.",
}