@inproceedings{nawaz-etal-2026-towards,
    title = "Towards Benchmarking {O}ld {C}hurch {S}lavonic Lemmatization",
    author = "Nawaz, Usman  and
      Napolitano, Marianna  and
      Karafillidis, Iris  and
      Lo Presti, Liliana  and
      Cascia, Marco",
    editor = "Bernard, Timoth{\'e}e  and
      Chersoni, Emmanuele  and
      Rambelli, Giulia",
    booktitle = "Proceedings of the Third Workshop on the Bridges and Gaps between Formal and Computational Linguistics ({B}ri{G}ap-3)",
    month = jul,
    year = "2026",
    address = "Paris, France",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-brigap/2026.brigap-1.9/",
    pages = "82--93",
    ISBN = "XXX-X-XXXXXX-XX-X",
    abstract = "Lemmatization is an important preprocessing step in Natural Language Processing (NLP); however, annotated resources for medieval languages such as Old Church Slavonic (OCS) are limited in scope, size, and diversity. This paper presents the annotated resources for OCS lemmatization, including annotation process, design choices and non-standard Unicode related issues. The annotated corpus is used to evaluate existing lemmatization tools (Stanza and UDPipe-2 models trained on the UD 2.12 treebank, and a dictionary-based approach) both in cross-dataset and on a corpus obtained by merging the new annotations with existing UD V2.12 OCS data. Pretrained models perform poorly ({\ensuremath{\approx}} 15{--}16{\%}), below a dictionary baseline ({\ensuremath{\approx}} 38{\%}), while retraining on the new data improves performance (up to {\ensuremath{\approx}} 51{\%}) and shows different cross-dataset generalization. Experiments in cross-dataset and on the combined corpus demonstrate that lemmatization performance depends strongly on dataset similarity, annotation conventions, and orthographic mismatch. Overall, the findings show the value of the newly annotated resources and the importance of extending OCS lemmatization benchmarks for historical Slavic NLP."
}