@inproceedings{chirkin-etal-2025-ruscontext,
    title = "{R}us{C}on{T}ext Benchmark: A {R}ussian Language Evaluation Benchmark for Understanding Context",
    author = "Chirkin, Andrey  and
      Kuznetsova, Svetlana  and
      Volina, Maria  and
      Dengina, Anna",
    editor = "Zhao, Jin  and
      Wang, Mingyang  and
      Liu, Zhu",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/landing_page/2025.acl-srw.91/",
    pages = "1158--1170",
    ISBN = "979-8-89176-254-1",
    abstract = "This paper represents an implementation of an approach rather similar to that of (Zhu et al., 2024), adapted for the Russian-language data. We introduce the RusConText Benchmark for evaluating short-context understanding in Russian, comprising four distinct yet interrelated tasks: coreference resolution, discourse understanding, idiom interpretation and ellipsis resolution. Each task targets a specific aspect of linguistic processing, challenging a large language model to recover omitted information, resolve referential dependencies, interpret idioms and discourse. The RusConText Benchmark is an additional resource beyond standard benchmarks, designed to assess model performance from a specific perspective. In addition, we present the results of scoring 4 models on our benchmark."
}