\section{Datasets}
\label{sec:data}

\begin{table*}[!htb]
  \begin{center}
    \resizebox{\textwidth}{!}{
\begin{tabular}{@{}l rrrr rrrr r rrrr @{}}\toprule
                              & \MC{4}{text size}                  & \MC{4}{entities}              &          & \MC{4}{mentions}              \\\cmidrule(lr){2-5}\cmidrule(lr){6-10}\cmidrule(lr){11-14}
document                      & \MC{4}{total number of}            &  total &per 1k &\MC{2}{length}& range    &  total &per 1k &\MC{2}{length}\\\cmidrule(lr){2-5}\cmidrule(lr){8-9}\cmidrule(lr){13-14}
                              &  docs &  sents &    words &empty n.&  count & words &   max &  avg & p95      &  count & words &   max &  avg \\\midrule
Ancient\_Greek-PROIEL         &    19 &  6,475 &   64,111 &  6,283 &  3,215 &    50 &   332 &  6.6 &      290 & 21,354 &   333 &    52 &  1.7 \\
Ancient\_Hebrew-PTNK          &    40 &  1,161 &   28,485 &      0 &    870 &    31 &   102 &  7.2 &      744 &  6,247 &   219 &    22 &  1.5 \\
Catalan-AnCora                & 1,298 & 13,613 &  429,313 &  6,377 & 17,558 &    41 &   101 &  3.6 &      538 & 62,417 &   145 &   141 &  4.8 \\
Czech-PCEDT                   & 2,312 & 49,208 &1,155,755 & 35,315 & 49,036 &    42 &   236 &  3.4 &      663 &167,418 &   145 &    79 &  3.6 \\
Czech-PDT                     & 3,165 & 49,419 &  834,707 & 21,092 & 46,460 &    56 &   173 &  3.3 &      430 &154,437 &   185 &    99 &  3.1 \\
\NEW Czech-PDTSC              & 1,553 & 73,802 &  743,963 & 62,130 & 44,940 &    60 &    81 &  4.1 &      400 &185,512 &   249 &    57 &  1.6 \\
\NEW Dutch-OpenBoek           &     9 &  5,704 &  103,518 &      0 &  2,287 &    22 &   780 &  7.4 &\bf 8,708 & 16,995 &   164 &    20 &  1.5 \\
\NEW English-FantasyCoref     &   211 & 12,126 &  340,545 &      0 &  5,829 &    17 &   339 &  9.8 &\bf 2,087 & 56,963 &   167 &    50 &  1.9 \\
English-GUM                   &   237 & 13,263 &  233,926 &    121 &  9,266 &    40 &   131 &  4.4 &      899 & 40,918 &   175 &    95 &  2.6 \\
English-LitBank               &   100 &  8,560 &  210,530 &      0 &  2,164 &    10 &   261 & 10.8 &\bf 1,939 & 23,340 &   111 &   129 &  1.6 \\
English-ParCorFull            &    19 &    543 &   10,798 &      0 &    188 &    17 &    38 &  4.4 &      556 &    835 &    77 &    37 &  2.1 \\
French-ANCOR                  &   455 & 31,761 &  454,577 &      0 & 13,204 &    29 &   103 &  4.3 &\bf 2,073 & 56,459 &   124 &    17 &  1.9 \\
French-Democrat               &   126 & 13,057 &  284,883 &      0 &  7,162 &    25 &   895 &  6.5 &\bf 7,440 & 46,487 &   163 &    71 &  1.7 \\
\NEW French-LitBankFr         &    29 & 11,312 &  276,985 &      0 &  2,023 &     7 &   934 & 16.9 &\bf11,583 & 34,228 &   124 &    22 &  1.5 \\
German-ParCorFull             &    19 &    543 &   10,602 &      0 &    243 &    23 &    43 &  3.7 &      487 &    896 &    85 &    30 &  2.0 \\
German-PotsdamCC              &   176 &  2,238 &   33,222 &      0 &    880 &    26 &    15 &  2.9 &      165 &  2,519 &    76 &    34 &  2.6 \\
Hindi-HDTB                    &   271 &  3,479 &   76,282 &      0 &  3,148 &    41 &    36 &  3.8 &      358 & 12,082 &   158 &    43 &  1.8 \\
Hungarian-KorKor              &    94 &  1,351 &   24,568 &  1,569 &  1,122 &    46 &    41 &  3.6 &      244 &  4,091 &   167 &    42 &  2.2 \\
Hungarian-SzegedKoref         &   400 &  8,820 &  123,968 &  4,857 &  4,769 &    38 &    36 &  3.2 &      327 & 15,165 &   122 &    36 &  1.6 \\
Korean-ECMT                   & 1,470 & 30,784 &  482,986 &      0 & 16,536 &    34 &    55 &  3.4 &      448 & 56,538 &   117 &    12 &  1.3 \\
\NEW Latin-CorefLat           &    16 &  1,741 &   25,965 &      0 &    734 &    28 &    14 &  2.8 &       82 &  2,075 &    80 &    70 &  2.5 \\
Lithuanian-LCC                &   100 &  1,714 &   37,014 &      0 &  1,087 &    29 &    23 &  4.0 &      385 &  4,337 &   117 &    19 &  1.5 \\
Norwegian-BokmaalNARC         &   346 & 15,742 &  245,515 &      0 &  5,658 &    23 &   298 &  4.7 &    1,042 & 26,611 &   108 &    51 &  1.9 \\
Norwegian-NynorskNARC         &   394 & 12,481 &  206,660 &      0 &  5,079 &    25 &    84 &  4.3 &      852 & 21,847 &   106 &    57 &  2.1 \\
Old\_Church\_Slavonic-PROIEL  &    26 &  6,832 &   61,759 &  6,289 &  3,396 &    55 &   134 &  6.5 &      279 & 22,116 &   358 &    52 &  1.5 \\
Polish-PCC                    & 1,828 & 35,874 &  538,885 & 18,615 & 22,143 &    41 &   135 &  3.7 &      276 & 82,706 &   153 &   108 &  1.9 \\
Russian-RuCor                 &   181 &  9,035 &  156,636 &      0 &  3,515 &    22 &   141 &  4.6 &    1,064 & 16,193 &   103 &    18 &  1.7 \\
Spanish-AnCora                & 1,356 & 14,159 &  458,418 &  8,112 & 19,445 &    42 &   110 &  3.6 &      530 & 70,663 &   154 &   101 &  4.8 \\
Turkish-ITCC                  &    24 &  4,732 &   55,358 & 11,584 &  4,019 &    73 &   369 &  5.4 &    1,032 & 21,569 &   390 &    31 &  1.1 \\
\bottomrule\end{tabular}
}
    \caption{CorefUD~1.4 data sizes in terms of the total number of documents, sentences,
      words (i.e.\ non-empty nodes), empty nodes (empty words),
      coreference entities
      (total count, relative count per 1000 words, average and maximal length in number of mentions and 95th percentile of range, i.e. distance from the first to the last mention in words)
      and coreference mentions
      (total count, relative count per 1000 words, average and maximal length in number of words).
      All the counts are excluding singletons and for the concatenation of train+dev+test.
      Train/dev/test splits of these datasets roughly follow the 8/1/1 ratio,
      but for the shared task we used reduced versions: mini-dev and mini-test.
      Newly added datasets have a gray background
      and values of p95 range$>$1500 are in bold.
      }
    \label{tab:sizes}
  \end{center}
\end{table*}

As in previous years, the shared task uses training and evaluation data from
the public part of the CorefUD collection
\cite{corefud2022lrec},\footnote{\url{https://ufal.mff.cuni.cz/corefud}} now
available in its latest release
(1.4).\footnote{\url{https://hdl.handle.net/11234/1-6108}} The public edition
of CorefUD~1.4 contains 29 datasets%
\footnote{For the shared task, we used only 27 of them (see
Section~\ref{sec:data-preproc}).} covering \nlanguages{} languages. Compared
to CorefUD~1.3, used last year \cite{oursharedtask2025}, the new release adds
five datasets and two languages. Several of the additions emphasize
long-document collections (primarily literary texts), which can challenge
%unconstrained
%\todo{Zakomentoval jsem zde "unconstrained"}
% protože se ukázalo, že long-doc je problematické pro některé LLM i UNC systémy
% a naopak jiné LLM i UNC to zvládají lépe, tedy nemůžeme tvrdit, že to nějak souvisí s LLM-vs-UNC.
systems that are often optimized for short-range coreference.
In addition to English LitBank and French Democrat\&ANCOR
%\todo{připsal jsem Democrat\&ANCOR}
% protože p95 range mají 7440 a 2073, tedy vyšší než En LitBank (1939) dle Table 1.
from previous editions, long-entity\footnote{
 The entity length can be defined in various ways, e.g., by the number of mentions.
 However, here we focus rather on the \emph{range} of the entity,
  i.e., the distance from the first to the last mention of a given entity in words.
 We report this entity range in Table~\ref{tab:sizes} as the 95th percentile for each dataset,
  i.e., the longest range after excluding top 5~\% (possible outliers) and also excluding singletons.
 When the p95 range is higher than 1500, we consider a given dataset a long-entity dataset.
 % Martin: Těch 1500 je samozřejmě arbitrární threshold.
 % Může to být cokoli mezi 1064 (RuCor, který asi nechceme považovat za long-entity) a 1939 (En LitBank).
} 
data are now represented by French LitBankFr, Dutch OpenBoek, and English FantasyCoref.
With OpenBoek, we also add Dutch to the language set.
The set of languages is further expanded with Latin via CorefLat, and we broaden
the coverage of spoken language by including Czech PDTSC. Overall, the datasets
span diverse domains, including news, fiction, Bible texts, and Wikipedia
articles. Table~\ref{tab:sizes} provides an overview of the datasets and
their sizes; see Appendix~\ref{sec:data-references} for references to the
individual datasets.

A key aim of the CorefUD project is to stimulate research on coreference resolution beyond English, especially for languages that exhibit zero anaphora.
Zero anaphora (\emph{zero mentions}) arises when a referent (e.g., a subject or object) is recoverable from context but not overtly realized. This is typical of pro-drop languages, where verbal morphology often provides sufficient cues to reconstruct the omitted pronoun.
In CorefUD, such cases are encoded as \emph{empty nodes} inserted into Universal Dependencies (UD) trees, allowing them to participate in coreference chains alongside overt mentions.
Compared to the previous version, the set of datasets with zero mentions has been extended by introducing Czech PDTSC.

Our shared task is restricted to identity coreference. The CorefUD datasets, however, may also annotate other relations (e.g., bridging), and some corpora additionally mark phenomena such as event or abstract anaphora, while others do not. Since CorefUD is not fully harmonized with respect to annotation guidelines, the exact interpretation of these anaphoric labels can differ slightly across corpora. In converting resources to the CorefUD format, we aim to extract identity coreference\footnote{We are aware that complete isolation is not possible due to near-identity relations; see \citet{recasensNearIdentity2010}.} while otherwise preserving the original annotation as much as possible.


\subsection{New Resources}
\label{sec:data-new}

\paragraph{English FantasyCoref} English FantasyCoref \citep[\enfantasycoref{};][]{han-etal-2021-fantasycoref} is a conversion of the FantasyCoref corpus consisting of English fantasy texts. It provides full entity level coreference annotation and follows an entity cluster model. Annotations are made from an omniscient writer’s point of view, extending OntoNotes guidelines to account for literary specific phenomena such as asymmetry of knowledge, transformations or status changes of entities, foretold or wished entities that later materialize, and extensive lexical or metaphorical variation. The corpus is larger and structurally more complex than \enlitbank, with longer documents and longer range coreference chains.
% \todo{Martin: FantasyCoref má v průměru kratší dokumenty než EnLitBank: 1,614 vs  2,105.
%Je ale pravda, že nejdelší dokument má delší: 10,312 vs 3,419.}
% Asi zde nemusíme nic měnit. To "longer documents" se asi chápe jako max, nikoli avg.
% Jen na to upozorňuji. Žádná lepší formulace mě nenapadá.

\paragraph{French LitBank} \citep[\frlitbankfr;][]{jcls3924} is a corpus of 19th--20th century French literary prose. The texts consist mainly of novel excerpts originating from the French-Democrat corpus (\frdemocrat), and are annotated with literary entities and coreference relations. Entity annotation follows principles adapted from the English-LitBank corpus and is limited to selected categories relevant for literary analysis (e.g., persons, locations, facilities, geopolitical entities, vehicles, and time expressions). Coreference annotation targets these entities and is designed to support long-distance reference tracking typical of literary texts.
%The original corpus was not annotated in the UD framework; therefore, morpho-syntactic annotation in the CorefUD version is produced by UDPipe 2.
% Martin: přesunul jsem tuto informaci na konec celé podsekce 2.1,
%         protože FantasyCoref má taky UDPipe anotaci.

\paragraph{Dutch OpenBoek} \citep[\nlopenboek;][]{Cranenburgh_vanNoord_2022} is a corpus of Dutch literary prose from the 19th and early 20th centuries, consisting of long fragments (typically over 10\,000 tokens) from public-domain novels. The corpus is annotated for nominal and pronominal coreference, including singletons, using a single identity-based coreference category.
%Bridging relations and discontinuous mentions are not annotated.
%\todo{Martin: zakomentoval jsem zde větu o absenci Bridging a discontinuous a větu o meta-anotacích, které jsme smazali v konverzi.}
%A ve FantasyCoref a FrLitBank jsou bridgingy? Discontinuous tam nejsou.
%Tato věta uvedená jen zde je zavádějící, navrhuji ji smazat.}
%
%In addition to coreference, entity-level attributes such as gender and number are annotated. A distinctive feature of the Dutch-OpenBoek corpus is the inclusion of historical spelling normalization as a separate meta-annotation layer.
% Martin: Jenže i ty entity gender+number, i ty meta-anotace jsme v konverzi smazali, že?
%To zde musíme napsat, nebo smazat ty poslední dvě věty.}
%
%Morphosyntactic annotation in the CorefUD conversion is produced by UDPipe~2.
% Martin: přesunul jsem tuto informaci na konec celé podsekce 2.1.

% https://ufal.mff.cuni.cz/pdtsc2.0
% https://ufal.mff.cuni.cz/pdtsl/cz/TR-2006-33.pdf je pouze česky
\paragraph{Czech PDTSC} \citep[\cspdtsc;][]{pdtsc}, i.e.
Prague Dependency Treebank of Spoken Czech, contains
transcriptions of spontaneous dialogues from the projects Malach (testimonies
of Shoah survivors) and Companions (a virtual companion chatbot for lonely
people). In both cases, the dialogues are asymmetric, with one party speaking
most of the time and the other party inserting short comments and questions.
The speech is transcribed using standard Czech orthography, including
punctuation.
For technical reasons, the conversations are split into documents of 50 sentences, and coreference is only annotated inside each document.


\paragraph{Latin CorefLat} \citep[\lacoreflat;][]{delfino2024building}
is a converted version of CorefLat, a coreference-annotated corpus of Latin. It comprises four literary texts: two prose works (Book~I of \textit{De bello Gallico} by Gaius Julius Caesar; Book~I of \textit{Confessiones} by Augustine of Hippo) and two theatrical plays (\textit{Curculio} by Titus Maccius Plautus; \textit{Medea} by Lucius Annaeus Seneca).

Unlike other CorefUD datasets, CorefLat spans several centuries. The texts range from Curculio (2nd century BCE), the oldest, to Confessiones (4th century CE), the newest, which may already allow for the study of coreference from a diachronic perspective. For the purposes of the CorefUD collection, however, we adopted a train/dev/test split that preserves the overall proportion of each text across sections: roughly the first 10~\% of each of the four documents is allocated to the test set, another 10~\% to the development set, and the remaining 80~\% to the training set.

The CorefLat dataset appears to underperform compared to the results reported for other CorefUD datasets later in this paper (see Table~\ref{tab:all-langs}), most notably for the baseline system, which achieves only 6.8 on the primary metric. In a post--shared task investigation, we identified a flaw in the conversion pipeline. In addition to the link-based coreference representation, which was correctly imported into CorefUD~1.4, CorefLat also uses a cluster-based representation for certain groups of mentions; this representation was not handled by the converter. After extending the converter to combine both representation types, a preliminary experiment with the baseline system reached a CoNLL F1 score of 37.
We therefore expect that releasing an improved version of the dataset will also substantially improve the performance of other systems. Nevertheless, CorefLat may still pose specific challenges for coreference resolution because of its diachronic character and genre diversity.


%The cause is not yet clear. One possible explanation is the diachronic nature and genre diversity of the %underlying texts; however, the discrepancy may also indicate imperfections in the conversion from the native %CorefLat format to the CorefUD format, suggesting that the implementation requires further refinement. 
%We plan to examine it in the future.
%This issue will be examined in detail prior to the release of the next version of CorefUD.
%\todo{navrhuji nahradit poslední větu za kratší: We plan to examine it in the future.}


\bigskip
The original \enfantasycoref, \frlitbankfr, \nlopenboek, and \lacoreflat{} corpora were not annotated in the UD framework; therefore, morpho-syntactic annotation in the CorefUD version is produced by UDPipe~2.

\subsection{Updated Resources}
\label{sec:data-update}

The English GUM corpus (\engum) contains the same sentences as in CorefUD~1.3, but with minor corrections in the annotations from UD~2.17.

All three Czech datasets (PDT, PCEDT, and PDTSC) are subsets of the Prague
Dependency Treebank-Consolidated, PDT-C~2.0, converted to UD \cite{udpdtc}.
While PDTSC is new in CorefUD, the other two were included in previous
versions of CorefUD. Changes since the last version are in morphosyntactic
annotation and its PDT-to-UD conversion. Coreference annotation is unchanged,
except for a few corrections.

\paragraph{Updated morphosyntax prediction}
For datasets without manual morphosyntactic annotation, we re-predicted UD relations, tags, and features using newer UDPipe models based on UD release 2.17 (previously 2.15).
Czech PCEDT was excluded from this subset because the latest version now includes manual morphosyntactic annotation.



\subsection{Data for the Shared Task}
\label{sec:data-preproc}

Relative to the public CorefUD~1.4 release, the dataset distributed to shared task participants was subject to minor adjustments.%
\footnote{
Both the shared task data and submissions are available at \url{http://hdl.handle.net/11234/1-6160}.
}

\paragraph{Data reduction}
As in last year's shared task, the English and German ParCorFull datasets were excluded due to their very small size and high variance, which disproportionately affected macro-averaged results. Moreover, the development and test sets were reduced to \emph{mini-dev} and \emph{mini-test} sets by capping each split at 25k words via random sampling of complete documents to lower evaluation costs (details in \citet{oursharedtask2025}).

New this year, we do not apply the 25k-word cap to the literary datasets with long documents, as capping would leave too few documents per split and reduce diversity across documents. In practice, this exception affects only \enfantasycoref{} and \frlitbankfr{}.
% Martin změnil: too little data -> too few documents


\begin{figure*}
    \centering
    \includegraphics[width=\linewidth]{figures/plaintext-json/format-example.pdf}
    \caption{Serialization of a Spanish example sentence from \esancora{} to our plaintext and JSON format. For clarity, mention spans are highlighted by colored underlining, where two coreferential entities share the same color. A zero mention labeled on an empty node is grayed. Note that multi-word tokens are split in the plaintext format into syntactic words (e.g., the Spanish \emph{``del''} appears as \emph{``de el''})}
    \label{fig:plaintext-example}
\end{figure*}


\paragraph{Plaintext format}
For the LLM track, we provide a conversion of the data into a simplified plaintext representation, together with both the conversion utility and the already converted dataset files.

In this plaintext format (see Figure~\ref{fig:plaintext-example}), each document is stored as a single line of text, with tokens separated by spaces. Coreference information is attached to each token following the `\texttt{|}' symbol. Mentions, including singletons, are specified by their span boundaries, indicated using opening and closing square brackets combined with an entity identifier. Empty nodes are marked with the prefix '\texttt{\#\#}'. Since empty nodes are defined by syntactic structure rather than linear token order, they are inserted directly after their syntactic parent. This representation does not capture the type of dependency relation to the parent, and therefore cannot differentiate between multiple empty nodes sharing the same parent (see Section~\ref{sec:evaluation}). Although this restriction may have a minor influence on evaluation, we consider it a reasonable compromise to maintain the format’s simplicity.

Compared to CoNLL-U, the plaintext format is deliberately less expressive and does not retain enough information for certain evaluation metrics (for example, head match relies on identifying mention heads via syntactic trees). To address this, we provide a reverse conversion tool that reconstructs CoNLL-U files from the plaintext annotations, along with an output cleaning utility.%
\footnote{The conversion tool and cleaner are distributed as a unified application/Python library on GitHub: \url{https://github.com/ondfa/text2text-coref}}

The cleaner is designed to fix common issues in LLM-generated outputs, such as malformed annotation structures (e.g., missing mention boundaries) or alterations to the token sequence. It first enforces correct opening and closing of all mentions, and then aligns the output text with the original input using word-level edit distance. During this alignment, empty nodes are excluded, as systems are expected to generate them independently. After achieving an exact match between token sequences, the annotations can be reliably projected back onto the original CoNLL-U data.


\paragraph{JSON format}

In order to facilitate the development of participants' systems
%\todo{Martin dopsal unconstrained}
% Nevím, jestli je potřeba vysvětlovat, proč JSON není vhodný pro LLM,
% ale aspoň takto naznačme, že to nebyl náš cíl. Předejdeme tak mnoha nedorozuměním.
in the unconstrained track,
we also provide data in a much simpler JSON format (see Figure~\ref{fig:plaintext-example}). We also offer tools for exporting coreference annotations from CoNLL-U into JSON and importing them back. The structure of the JSON format used in the Shared task is inspired by the structure of the output provided by the Maverick Coref package.\footnote{\url{https://github.com/SapienzaNLP/maverick-coref}}

 A file in JSON format is a list of documents, where each document is represented as a JSON object with four fields:
\begin{itemize}
    \item \textit{doc\_id}: the document ID
    \item \textit{tokens}: a list of tokens in the document.
    \item \textit{clusters\_token\_offsets}: a list of coreference clusters, where each cluster is represented as a list of mentions, and each mention is represented as a list of two integers indicating the start and end token offsets (inclusive) of the mention in the tokens list.
    \item \textit{clusters\_text\_mentions}: a list of coreference clusters, where each cluster is represented as a list of textual mentions corresponding to the mentions in clusters\_token\_offsets.
\end{itemize}

Conversion is very similar to the plaintext format. Empty nodes are represented exactly the same way; prefixed with `\textit{\#\#}', placed right after their syntactic head.

\paragraph{Data variants and starting points}
For each track, we release two versions of the data: \emph{gold data} (for training and evaluation against gold annotations) and \emph{input data} (the version intended to be processed by participants' systems and submitted).
For the Unconstrained track, we additionally provide three alternative starting points that differ in how much of the pipeline is already pre-filled with baseline predictions.

\emph{Gold data} contains gold-standard coreference and gold empty nodes, and is meant for training and evaluation.
It follows the CorefUD~1.4 release and preserves manually annotated morpho-syntactic features whenever they are available in the original dataset.
All CoNLL-U fields are kept as in the source data, including the forms of empty nodes.\footnote{Earlier data versions removed them to match the output of the baseline empty-node predictor. This is no longer necessary, since the baseline now predicts all CoNLL-U fields for empty nodes (see Section~\ref{sec:baseline}).}
The gold train and mini-dev sets were available for download; the gold mini-test set remained private and was used internally in Codabench for evaluation.

\emph{Input data} is derived from the gold data by applying preprocessing that mimics a setting without manual linguistic annotation.
This preprocessing was performed only for the mini-dev and mini-test sets.
Specifically, we replaced the original morpho-syntactic features with the outputs of UD~2.17 models across all datasets (including those that originally contained human-annotated features) and removed gold empty nodes and gold coreference annotations.
This yields the input data for the LLM track; for the Unconstrained track, participants could choose one of three \emph{starting points}: (1) \emph{Coreference and zeros from scratch} (no predictions of empty nodes or coreference; identical to the LLM-track input), (2) \emph{Coreference from scratch} (baseline empty-node predictions provided, no coreference), and (3) \emph{Refine the baseline} (baseline predictions of both empty nodes and coreference provided).
