@inproceedings{lendvai-etal-2023-domain-adapting,
    title = "Domain-Adapting {BERT} for Attributing Manuscript, Century and Region in Pre-{M}odern {S}lavic Texts",
    author = "Lendvai, Piroska  and
      Reichel, Uwe  and
      Jouravel, Anna  and
      Rabus, Achim  and
      Renje, Elena",
    editor = "Tahmasebi, Nina  and
      Montariol, Syrielle  and
      Dubossarsky, Haim  and
      Kutuzov, Andrey  and
      Hengchen, Simon  and
      Alfter, David  and
      Periti, Francesco  and
      Cassotti, Pierluigi",
    booktitle = "Proceedings of the 4th Workshop on Computational Approaches to Historical Language Change",
    month = dec,
    year = "2023",
    address = "Singapore",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.lchange-1.2",
    doi = "10.18653/v1/2023.lchange-1.2",
    pages = "15--21",
    abstract = "Our study presents a stratified dataset compiled from six different Slavic bodies of text, for cross-linguistic and diachronic analyses of Slavic Pre-Modern language variants. We demonstrate unsupervised domain adaptation and supervised finetuning of BERT on these low-resource, historical Slavic variants, for the purposes of provenance attribution in terms of three downstream tasks: manuscript, century and copying region classification.The data compilation aims to capture diachronic as well as regional language variation and change: the texts were written in the course of roughly a millennium, incorporating language variants from the High Middle Ages to the Early Modern Period, and originate from a variety of geographic regions. Mechanisms of language change in relatively small portions of such data have been inspected, analyzed and typologized by Slavists manually; our contribution aims to investigate the extent to which the BERT transformer architecture and pretrained models can benefit this process. Using these datasets for domain adaptation, we could attribute temporal, geographical and manuscript origin on the level of text snippets with high F-scores. We also conducted a qualitative analysis of the models{'} misclassifications.",
}