
\section{Results and Comparison}
\label{sec:results}

%\formn{MOR u LLM systemov je horsi nez u Unconstrained. Pripadne zmerat korelaciu s CoNLL score, ci to nie je len dosledok celkovej kvality. UWB pouzivali iba hlavy pri niektorych datasetoch.}
%\formn{z porovnania LLM-NUST-FewShot a CorPipeEnsemble na \hboptnk{} je vidiet, ze najvacsi rozdiel medzi nimi v F1 je u B-cubed F1 a CEAF-e F1. Zatial co LLM-NUST-FewShot dosahuje pomerne zhodne hodnoty P a R, u CorPipeEnsemble je P znacne vyssie ako R u MUC (6b) a hlavne B-cubed (11b), naopak u CEAF-e je P nizsie nez R (o 5b)}
%\formp{Mame nejake zaujimave pozorovania pre \enlitbank{} a \hboptnk{}, na ktorych vyhrali LLM systemy?}

\begin{table*}\centering
\begin{tabular}{@{}l r r@{~~}l r@{~~}l r@{~~}l @{}}\toprule
                    & \MC{5}{excluding singletons} & \MC{2}{with singletons}\\\cmidrule(lr){2-6}\cmidrule(l){7-8}
\bf system          & \bf head-match & \MC{2}{\bf partial-match} & \MC{2}{\bf exact-match} & \MC{2}{\bf head-match}\\\midrule
LLM-LatticeNLP      & \bf 74.32 & \bf 74.32 & (-0.00) &     41.83 & (-32.48) & \bf 76.09 & (+1.77)\\
LLM-UWB             &     73.83 &     73.83 & (+0.00) &     40.76 & (-33.08) &     75.59 & (+1.75)\\
LLM-PortNLP         &     68.69 &     67.51 & (-1.18) & \bf 65.30 &  (-3.39) &     70.98 & (+2.29)\\
LLM-Landcore        &     46.19 &     44.79 & (-1.40) &     40.47 &  (-5.73) &     47.78 & (+1.59)\\\midrule
CorPipeEnsemble     & \bf 77.11 & \bf 76.30 & (-0.81) & \bf 74.07 &  (-3.04) & \bf 79.11 & (+2.00)\\
CorPipeXXL          &     76.18 &     75.32 & (-0.86) &     72.90 &  (-3.28) &     78.15 & (+1.97)\\
CorPipeLarge        &     72.32 &     71.16 & (-1.16) &     68.79 &  (-3.53) &     74.52 & (+2.20)\\
DAggerCoref         &     67.56 &     67.56 & (+0.00) &     37.63 & (-29.94) &     58.75 & (-8.81)\\
Stanza              &     67.00 &     65.87 & (-1.13) &     63.32 & (-3.68) &     68.53 & (+1.54)\\
\baselinegz         &     55.39 &     55.06 & (-0.33) &     53.91 & (-1.49) &     48.28 & (-7.11)\\
\baseline           &     54.54 &     54.16 & (-0.38) &     52.96 & (-1.58) &     47.45 & (-7.09)\\
AUKBC-MULCRF        &     35.24 &     35.21 & (-0.03) &     20.65 & (-14.59) &     33.10 & (-2.14)\\\midrule
\sys{Winner-2023}   &     74.90 &     73.33 & (-1.57) &     71.46 & (-3.44) &     76.82 & (+1.91)\\
\sys{Winner-2024}   &     73.90 &     72.19 & (-1.71) &     69.86 & (-4.04) &     75.65 & (+1.75)\\
\sys{Winner-2025}   &     75.84 &     74.90 & (-0.94) &     72.76 & (-3.08) &     78.33 & (+2.49)\\
\sys{Baseline-2023} &     56.96 &     56.28 & (-0.68) &     54.75 & (-2.21) &     49.32 & (-7.64)\\
\sys{Baseline-2024} &     53.16 &     52.48 & (-0.68) &     51.26 & (-1.90) &     46.45 & (-6.71)\\
\sys{Baseline-2025} &     56.01 &     55.58 & (-0.43) &     54.24 & (-1.77) &     47.88 & (-8.13)\\
\bottomrule\end{tabular}

\caption{\textbf{Main results}: the CoNLL F$_1$ score macro-averaged over all datasets.
The table shows the primary metric (head-match excluding singletons)
 and three alternative metrics:
 partial-match excluding singletons,
 exact-match excluding singletons and
 head-match with singletons.
A difference relative to the primary metric is reported in parenthesis.
The top section shows the LLM track, below is the Unconstrained track.
The best score in each column and each of these two sections is in bold.
The systems are ordered by the primary metric.
Similar notes apply to the following tables.
The last six rows showing the winner and baseline results from CRAC~2023--2025
 are copied from the last year Findings \citep{oursharedtask2025},
 and thus are not directly comparable with the rest of the table
 because both the test and training data have been changed (CorefUD~1.1 vs. 1.2 vs. 1.3 vs. 1.4).
}
\label{tab:main-results}
\end{table*}

%Table~\ref{tab:secondary-metrics} shows recall, precision, and F1 for six
%metrics.
%The F1 scores of the first five metrics (MUC. B$^3$, BLANC, and LEA)
% result in the same ordering of systems (same as the primary metric)
% except for RitwikmishraFix, which is slightly better than DFKI-CorefGen in BLANC and LEA.
%Most of the systems have higher precision than recall for all the metrics,
% but the highest disbalance is in the \baseline{} system.
%CorPipe* are the only systems that have higher recall than precision
% at least for CEAF-e (but other metrics have similar precision and recall).

%The MOR metric (mention overlap ratio) measures only the mention matching quality,
% while ignoring the coreference, but even then
% the ordering of systems is similar to the primary metric
% (Ondfa is the fourth worst according to MOR,
%  again because it does not predict full spans for most datasets).

\begin{table*}\centering
\resizebox{\textwidth}{!}{
\begin{tabular}{@{}l ccccccc @{}}\toprule
\bf system        & \bf MUC &\bf B$^3$ &\bf CEAF-e &\bf BLANC &\bf LEA &\bf MOR &\bf MD-h\\\midrule
CorPipeEnsemble   &  {\bf 82} / {\bf 84} / {\bf 83}  &       73  / {\bf 78} / {\bf 75}  &  {\bf 74} /      72  / {\bf 73}  &  {\bf 74} / {\bf 78} / {\bf 75}  &  {\bf 71} / {\bf 76} / {\bf 73}  &       81  /      85  / {\bf 82}  &  {\bf 87} / {\bf 88} / {\bf 87} \\
CorPipeXXL        &       82  /      83  /      82   &  {\bf 73} /      76  /      74   &       73  /      72  /      72   &       73  /      76  /      74   &       71  /      73  /      72   &  {\bf 81} /      84  /      82   &       86  /      87  /      87  \\
LLM-LatticeNLP    &       79  /      83  /      81   &       72  /      76  /      74   &       65  / {\bf 73} /      69   &       72  /      76  /      74   &       70  /      74  /      71   &       42  / {\bf 88} /      55   &       81  /      86  /      84  \\
LLM-UWB           &       80  /      82  /      81   &       71  /      73  /      72   &       68  /      71  /      69   &       72  /      74  /      73   &       68  /      71  /      69   &       42  /      84  /      55   &       81  /      83  /      82  \\
CorPipeLarge      &       80  /      80  /      80   &       69  /      71  /      70   &       69  /      67  /      68   &       70  /      71  /      70   &       66  /      68  /      67   &       79  /      80  /      79   &       85  /      85  /      85  \\
LLM-PortNLP       &       74  /      82  /      78   &       64  /      71  /      67   &       58  /      69  /      62   &       64  /      73  /      67   &       61  /      69  /      65   &       68  /      82  /      74   &       74  /      83  /      78  \\
DAggerCoref       &       73  /      80  /      77   &       60  /      71  /      65   &       59  /      65  /      62   &       61  /      71  /      65   &       57  /      68  /      62   &       38  /      83  /      51   &       75  /      82  /      78  \\
Stanza            &       74  /      79  /      76   &       63  /      68  /      65   &       61  /      62  /      61   &       63  /      69  /      65   &       60  /      65  /      62   &       72  /      81  /      75   &       79  /      84  /      81  \\
\baselinegz       &       60  /      75  /      66   &       46  /      61  /      51   &       46  /      56  /      49   &       47  /      62  /      51   &       42  /      56  /      47   &       55  /      86  /      65   &       69  /      87  /      76  \\
\baseline         &       60  /      73  /      65   &       45  /      60  /      50   &       45  /      55  /      49   &       45  /      61  /      50   &       41  /      55  /      46   &       54  /      85  /      65   &       68  /      85  /      74  \\
LLM-Landcore      &       48  /      66  /      54   &       38  /      56  /      43   &       39  /      52  /      42   &       33  /      54  /      37   &       34  /      52  /      39   &       49  /      72  /      56   &       54  /      74  /      61  \\
AUKBC-MULCRF      &       46  /      43  /      44   &       31  /      34  /      31   &       36  /      28  /      31   &       31  /      30  /      28   &       24  /      30  /      26   &       35  /      60  /      43   &       63  /      56  /      59  \\
\bottomrule\end{tabular}
}
\caption{Recall / Precision / F1 for individual \textbf{secondary metrics}.
All scores macro-averaged over all datasets.
Both MOR and MD-h measure mention detection quality, while ignoring the coreference.
The MD-h metric (mention detection score on heads) measures only the mention matching quality,
while MOR considers also the whole span detection.
%while ignoring the coreference and mention spans
}
\label{tab:secondary-metrics}
\end{table*}


\paragraph{Main results}
%\formp{MN pozorovania:
%\begin{itemize}
%\item celkovo vyhral CorPipe, ale kvalita LLM systemov je uz velmi podobna, pricom 3/4 LLM systemov je lepsich ako ne-CorPipe unc systemy
%\item zaujimave je porovnanie algoritmov spred 20 rokov s tymi sucasnymi, kedy AUKBC-MULCRF nedokazali porazit ani len Baseline pristup s obrovskym odstupom (napisat nejak menej znevazujuco)
%\end{itemize}
%}

The main results of the shared task are presented in Table~\ref{tab:main-results}.
Systems LLM-LatticeNLP and CorPipeEnsemble won the LLM and Unconstrained tracks, achieving primary scores of 74.32 and 77.11, respectively.
Generative LLMs thus still could not outperform the best discriminative encoder-based systems, but the gap has narrowed to less than 3 points (compared to 13 points last year), see Figure~\ref{fig:scores-over-years}.

Looking at the LLM track only, one can clearly see the difference made by fine-tuning.
Even though LLM-Landcore uses a frontier-level closed-source model, relying on in-context learning alone was not sufficient to surpass \baseline{} performance.
By contrast, the fine-tuned open-weight LLM approaches remained highly competitive up to the very last moment (see Appendix~\ref{sec:codalab-evol}) and achieved performance comparable to the encoder-based models.
%\todo{Zvážit to up to the very last moment}
% Martin: Souhlasím, že na Appendix~\ref{sec:codalab-evol} je vhodné se někde odkázat.
% Nevím ale, je-li to vhodné zde. To competetive bych si představoval trochu jinak,
% než že někdo posílá spoustu submissions během test phase (to zavání optimalizací na test setu).

As in previous editions, the CorPipe team and their architecture dominate the Unconstrained track, outperforming the other encoder-based systems by roughly 10 points, including the widely used Stanza system.
AUKBC-MULCRF ranks lower in absolute terms, but offers a notably lightweight alternative to large neural models.
Although the authors do not report the exact number of trainable parameters, the system is likely, together with Stanza (Table~\ref{tab:context-size}), among the most parameter-efficient submissions. This observation broadly aligns with the overall trend that larger models tend to achieve higher scores.


\paragraph{Secondary metrics}
Table~\ref{tab:main-results} also reports CoNLL F$_1$ scores under alternative mention-matching strategies.
While the system ranking under partial match broadly mirrors head match, exact match more clearly distinguishes systems that resolve only mention heads rather than full spans (LLM-LatticeNLP, LLM-UWB, and DAggerCoref), as indicated by the absence of mentions longer than one token in Table~\ref{tab:stats-mentions-nonsingleton} in Appendix~\ref{sec:stats-concat}.
For these systems, exact-match CoNLL scores are about 30 points lower than the corresponding head-match scores.
This aligns with the MOR results in Table~\ref{tab:secondary-metrics}, which fall well below \baseline{} for these systems; we do not observe comparable drops in MD-h.

Including singletons (single-mention entities) in the head-match evaluation in Table~\ref{tab:main-results} increases CoNLL scores for all participating systems except DAggerCoref and AUKBC-MULCRF, suggesting that these systems do not model singletons.
This is confirmed by the absence (or only a marginal number) of singletons in Table~\ref{tab:stats-entities} in Appendix~\ref{sec:stats-concat}.

Table~\ref{tab:secondary-metrics} reports additional coreference measures; aside from minor differences, they yield the same ranking as the primary score
(except for MOR, where LLM-LatticeNLP, LLM-UWB, and DAggerCoref are penalized in recall for predicting only mention heads, as mentioned above).
The interpretation of recall and precision in coreference evaluation is tricky and differs for each metric.
That said, we can notice that the top-scoring systems (both LLM and unconstrained)
maintain relatively small gaps between precision and recall.
The other systems mostly show higher precision than recall,
but the difference is not as striking as it was last year.


\paragraph{Comparison across datasets}
Table~\ref{tab:all-langs} and Figure~\ref{fig:alllangs} both show CoNLL F$_1$ scores of all systems across the datasets.
To make patterns more visible, the datasets in Figure~\ref{fig:alllangs} are ordered from left to right by the decreasing performance of the top system, CorPipeEnsemble.
Last year, the CorPipe team was outperformed on two datasets only:
on \hboptnk{} by LLM-NUST-FewShot (+10.4) and on \enlitbank{} by LLM-UWB (+2.2).
This year, it has been outperformed on 7 datasets, most notably on the following three:
on \hboptnk{} by LLM-LatticeNLP (+5.3) and LLM-UWB (+2.3),
on \nonynorsknarc{} by LLM-UWB (+2.7) and LLM-LatticeNLP (+0.8), and
on \nlopenboek{} by LLM-LatticeNLP (+2.7).
% on \nobokmaalnarc{} by LLM-LatticeNLP (+2.3) and LLM-UWB (+1.5),
% on \frdemocrat{} by LLM-LatticeNLP (+1.1),
% on \francor{} by LLM-UWB (+1.0) and LLM-LatticeNLP (+0.8), and
% on \enlitbank{} by LLM-LatticeNLP (+0.1).
%Similarly to last year, the difference on Ancient Hebrew is suspicious, but we are not sure what the cause is.
Similarly to last year, we are not sure about the cause of these differences.
Figure~\ref{fig:alllangs} shows that all systems, except for the three CorPipes and LLM-LatticeNLP, struggled with \frlitbankfr{} (relative to CorPipeEnsemble)
, and most of them also struggled with \grcproiel{}.

%\formp{MN pozorovania:
%\begin{itemize}
%    \item mozno by pri analyze pomohlo este ine radenie datasetov, napr. podla rozptylu alebo podla priemerneho rozdielu vysledku na datasete od celkoveho vysledku
%    \item na \nobokmaalnarc{}, \nonynorsknarc{}, \francor{}, \nlopenboek{}, \hboptnk{} a \frdemocrat{} su niektore LLMs lepsie ako CorPipe. Mam podozrenie, ze to moze byt kontaminaciou, pretoze u viacerych z tychto korpusov su k dispozicii testsety na GitHube (ale to iste plati aj pre datasety, kde sa toto neudialo)
%\end{itemize}}

\paragraph{Performance on zero mentions}

Table~\ref{tab:all-langs-zero} reports system performance on datasets containing zero mentions, evaluated with the anaphor-decomposable score for zero anaphora.
In CorefUD 1.4, we extended this set of datasets by \cspdtsc{}.

Looking at the F1 scores, CorPipe submissions dominate in most languages.
The only exception is \hukorkor{}, where LLM-UWB and LLM-LatticeNLP score about 10 points higher.
We do not observe this pattern on \huszegedkoref{}, where empty nodes are less frequent (relative to the number of words) and were annotated less rigorously.
Annotation guidelines have a strong impact on zero-mention evaluation, as also seen in the Czech datasets, where none of the systems surpass \baselinegz{}.
Although pro-drop in Czech is not, from a linguistic perspective, considered more complex than in Hungarian or Turkish, the Czech annotation guidelines make resolution harder.
%\todo{Zde jsem zakomentoval nepravdivé tvrzení o LLM-Landcore and AUKBC-MULCRF}
%The authors of LLM-Landcore and AUKBC-MULCRF stated that they omitted zero mentions from their predictions (see Table~\ref{tab:train-config}).
%However, zero mentions were not completely absent: we still observe non-zero scores on most datasets, even though these systems perform worse than \baseline{}.
LLM-Landcore and AUKBC-MULCRF have notably worse results on zero anaphors.\footnote{
 AUKBC-MULCRF used the empty-nodes baseline starting point,
  so empty nodes are presented in its predictions for all the 11 datasets,
  but for \hukorkor{}, \huszegedkoref{}, and \plpcc{}, there are no zero anaphora predicted.
 LLM-Landcore had to predict the empty nodes from scratch (the empty-nodes baseline is allowed only in the unconstrained track), and in \caancora{} it predicted no empty nodes.
}

Compared to last year, we observe positive progress: this year's winning system performs similarly, or even better (e.g., on \cspcedt{} and \grcproiel{}) than last year's winner.

%\formp{tabulka kolko percent empty nodes je aj zero mentions a ci to koreluje s rozdielmi medzi Baseline-GZ a Baseline?}

%Table~\ref{tab:all-langs-zero} shows the performance of the systems on zero anaphora resolution on datasets with annotated zeros.
%Let us start with a comparison of the \baseline{} and \baselinegz{} systems, which differ only in the nature of the empty nodes (predicted vs. gold).\footnote{
%The gold empty nodes in the testset were not available to the participants,
% thus \baselinegz{} is not directly comparable with the other systems;
% it serves as a comparison with the previous year,
%  when all empty nodes were gold.
%}
%It confirms that by moving to the realistic setup for zeros the task became much more challenging, illustrated by the performance drop in the F1 score by 5-19 points for most of the datasets.



\paragraph{Comparison over years}
With five iterations of the shared task now completed, we can examine longitudinal trends in multilingual coreference resolution.
Although the data and some task details have changed from year to year, the same baseline approach has been retained throughout: a fixed baseline system retrained each year on the current release.
This provides a consistent reference point for measuring progress by relating each year's top submissions to \baseline{}.

Figure~\ref{fig:scores-over-years} tracks the best unconstrained and best LLM-based submissions over time.
%, using \baseline{} as a shared reference point.
% Martin: to už je v předchozím odstavci
After the dip observed last year (likely driven in part by the removal of the ParCorFull datasets), the top unconstrained system rebounds and now surpasses \baseline{} by more than 41~\%.
The LLM Track also improves markedly: the strongest LLM-based system increases its gain over \baseline{} from 12~\% in the last year to 36~\% this year.
Overall, the trend is clear: despite the benchmark becoming tougher as new datasets are added, state-of-the-art systems keep pushing performance upward year after year.

\begin{table*}\centering
%\scriptsize
\resizebox{\textwidth}{!}{
\begin{tabular}{@{}l r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{}}\toprule
\bf system        & \rot\caancora & \rot\cspcedt & \rot\cspdt & \rot\cspdtsc & \rot\cuproiel & \rot\depotsdamcc & \rot\enfantasycoref & \rot\engum & \rot\enlitbank & \rot\esancora & \rot\francor & \rot\frdemocrat & \rot\frlitbankfr & \rot\grcproiel & \rot\hboptnk & \rot\hihdtb & \rot\hukorkor & \rot\huszegedkoref & \rot\koecmt & \rot\lacoreflat & \rot\ltlcc & \rot\nlopenboek & \rot\nobokmaalnarc & \rot\nonynorsknarc & \rot\plpcc & \rot\rurucor & \rot\tritcc\\\midrule
CorPipeEnsemble   &\bf  84.4 &\bf  79.3 &\bf  81.8 &\bf  76.7 &\bf  68.8 &\bf  75.0 &\bf  81.1 &\bf  77.4 &     85.3 &\bf  85.3 &     76.5 &     73.4 &\bf  82.5 &\bf  79.0 &     74.5 &\bf  78.4 &\bf  68.7 &\bf  72.6 &\bf  70.3 &\bf  62.6 &\bf  76.1 &     74.7 &     78.9 &     76.8 &\bf  82.1 &\bf  86.2 &     73.5\\
CorPipeXXL        &     83.0 &     78.8 &     81.6 &     76.7 &     67.9 &     74.8 &     80.8 &     76.7 &     83.7 &     84.2 &     75.5 &     73.8 &     81.5 &     77.9 &     72.0 &     77.8 &     68.2 &     71.4 &     69.7 &     58.7 &     75.2 &     73.1 &     77.4 &     76.4 &     81.5 &     84.5 &\bf  74.0\\
LLM-LatticeNLP    &     77.0 &     72.8 &     76.3 &     70.9 &     60.1 &     71.3 &     78.8 &     76.9 &\bf  85.4 &     78.3 &     77.3 &\bf  74.5 &     80.2 &     74.9 &\bf  79.8 &     76.8 &     65.5 &     66.6 &     68.8 &     58.7 &     65.3 &\bf  77.4 &\bf  81.2 &     77.6 &     78.3 &     82.6 &     73.1\\
LLM-UWB           &     82.7 &     75.8 &     80.0 &     73.8 &     59.6 &     73.3 &     80.2 &     75.9 &     84.6 &     83.1 &\bf  77.5 &     66.7 &     60.5 &     75.3 &     76.8 &     77.0 &     65.3 &     69.0 &     66.9 &     56.2 &     73.5 &     66.1 &     80.4 &\bf  79.5 &     80.1 &     84.7 &     69.1\\
CorPipeLarge      &     79.7 &     73.8 &     77.9 &     72.3 &     56.3 &     69.9 &     74.9 &     73.5 &     79.4 &     82.3 &     71.5 &     71.0 &     76.5 &     69.0 &     66.4 &     76.3 &     64.6 &     67.7 &     68.6 &     57.5 &     75.7 &     69.9 &     74.4 &     74.0 &     78.0 &     82.2 &     69.5\\
LLM-PortNLP       &     73.7 &     71.5 &     74.1 &     69.8 &     57.9 &     67.9 &     74.8 &     70.6 &     78.4 &     75.4 &     70.5 &     53.4 &     54.9 &     71.1 &     72.7 &     75.6 &     59.1 &     62.5 &     69.5 &     44.8 &     68.1 &     72.9 &     72.1 &     72.1 &     76.8 &     81.2 &     63.0\\
DAggerCoref       &     76.8 &     70.8 &     74.6 &     68.2 &     57.2 &     71.9 &     73.0 &     68.6 &     73.6 &     77.4 &     70.2 &     69.0 &     66.9 &     65.2 &     57.9 &     72.8 &     60.5 &     61.0 &     61.1 &     56.0 &     66.5 &     64.5 &     72.9 &     72.0 &     73.7 &     79.1 &     42.9\\
Stanza            &     77.7 &     74.0 &     76.4 &     71.2 &     38.9 &     70.4 &     69.9 &     72.7 &     73.3 &     80.1 &     69.5 &     57.1 &     64.5 &     53.9 &     61.2 &     75.5 &     59.9 &     66.9 &     67.1 &     36.9 &     73.0 &     60.0 &     72.8 &     70.8 &     73.7 &     80.4 &     60.9\\
\baselinegz       &     68.8 &     68.8 &     67.9 &     69.3 &     26.9 &     52.4 &     65.1 &     62.0 &     66.2 &     70.7 &     61.8 &     55.6 &     46.1 &     31.1 &     31.7 &     66.6 &     44.6 &     55.0 &     65.0 &      6.8 &     62.4 &     40.6 &     61.4 &     61.1 &     68.9 &     68.2 &     50.8\\
\baseline         &     67.9 &     63.4 &     66.2 &     66.1 &     24.7 &     52.4 &     65.1 &     61.9 &     66.2 &     70.3 &     61.8 &     55.6 &     46.1 &     30.6 &     31.7 &     66.6 &     42.2 &     54.3 &     65.0 &      6.8 &     62.4 &     40.6 &     61.4 &     61.1 &     67.5 &     68.2 &     46.8\\
LLM-Landcore      &     41.3 &     36.7 &     40.3 &     40.9 &     29.4 &     49.3 &     58.1 &     55.3 &     63.0 &     47.3 &     46.4 &     20.7 &     46.7 &     43.4 &     61.4 &     60.6 &     41.6 &     37.3 &     59.5 &     32.8 &     52.8 &     39.3 &     54.5 &     53.5 &     43.2 &     52.3 &     39.8\\
AUKBC-MULCRF      &     38.8 &     25.8 &     36.8 &     37.9 &     34.8 &     29.8 &     41.4 &     44.2 &     34.4 &     37.4 &     37.4 &     35.5 &     28.6 &     43.4 &     48.2 &     46.2 &     29.5 &     31.6 &     21.6 &     16.2 &     28.0 &     34.5 &     42.9 &     39.6 &     36.2 &     33.7 &     37.2\\
\bottomrule\end{tabular}
}
\caption{Results for \textbf{individual languages} in the primary metric (CoNLL F$_1$).
%Best score in each column is in bold.
}
\label{tab:all-langs}
\end{table*}


\begin{figure*}\centering
\resizebox{!}{23cm}{
\begin{tikzpicture}
  \begin{axis}[
    ylabel={CoNLL F$_1$},
    ymin=15, ymax=90,
    width=\textwidth, height=20cm,
    xtick={1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27},
    xticklabels={\rot\rurucor, \rot\enlitbank, \rot\esancora, \rot\caancora, \rot\frlitbankfr, \rot\plpcc, \rot\cspdt, \rot\enfantasycoref, \rot\cspcedt, \rot\grcproiel, \rot\nobokmaalnarc, \rot\hihdtb, \rot\engum, \rot\nonynorsknarc, \rot\cspdtsc, \rot\francor, \rot\ltlcc, \rot\depotsdamcc, \rot\nlopenboek, \rot\hboptnk, \rot\tritcc, \rot\frdemocrat, \rot\huszegedkoref, \rot\koecmt, \rot\cuproiel, \rot\hukorkor, \rot\lacoreflat},
    legend style={at={(0.5, -0.2)}, anchor=north},
    legend columns=3,
    ymajorgrids=true,
    grid style=dashed
  ]
    \addplot [solid, color=blue, mark=*] coordinates {
      (1, 86.2)
      (2, 85.3)
      (3, 85.3)
      (4, 84.4)
      (5, 82.5)
      (6, 82.1)
      (7, 81.8)
      (8, 81.1)
      (9, 79.3)
      (10, 79.0)
      (11, 78.9)
      (12, 78.4)
      (13, 77.4)
      (14, 76.8)
      (15, 76.7)
      (16, 76.5)
      (17, 76.1)
      (18, 75.0)
      (19, 74.7)
      (20, 74.5)
      (21, 73.5)
      (22, 73.4)
      (23, 72.6)
      (24, 70.3)
      (25, 68.8)
      (26, 68.7)
      (27, 62.6)
    };
    \addlegendentry{CorPipeEnsemble}
    \addplot [solid, color=blue!60!black, mark=o] coordinates {
      (1, 84.5)
      (2, 83.7)
      (3, 84.2)
      (4, 83.0)
      (5, 81.5)
      (6, 81.5)
      (7, 81.6)
      (8, 80.8)
      (9, 78.8)
      (10, 77.9)
      (11, 77.4)
      (12, 77.8)
      (13, 76.7)
      (14, 76.4)
      (15, 76.7)
      (16, 75.5)
      (17, 75.2)
      (18, 74.8)
      (19, 73.1)
      (20, 72.0)
      (21, 74.0)
      (22, 73.8)
      (23, 71.4)
      (24, 69.7)
      (25, 67.9)
      (26, 68.2)
      (27, 58.7)
    };
    \addlegendentry{CorPipeXXL}
    \addplot [dashed, color=blue!60!green, mark=square*] coordinates {
      (1, 82.6)
      (2, 85.4)
      (3, 78.3)
      (4, 77.0)
      (5, 80.2)
      (6, 78.3)
      (7, 76.3)
      (8, 78.8)
      (9, 72.8)
      (10, 74.9)
      (11, 81.2)
      (12, 76.8)
      (13, 76.9)
      (14, 77.6)
      (15, 70.9)
      (16, 77.3)
      (17, 65.3)
      (18, 71.3)
      (19, 77.4)
      (20, 79.8)
      (21, 73.1)
      (22, 74.5)
      (23, 66.6)
      (24, 68.8)
      (25, 60.1)
      (26, 65.5)
      (27, 58.7)
    };
    \addlegendentry{LLM-LatticeNLP}
    \addplot [dashed, color=red, mark=triangle*] coordinates {
      (1, 84.7)
      (2, 84.6)
      (3, 83.1)
      (4, 82.7)
      (5, 60.5)
      (6, 80.1)
      (7, 80.0)
      (8, 80.2)
      (9, 75.8)
      (10, 75.3)
      (11, 80.4)
      (12, 77.0)
      (13, 75.9)
      (14, 79.5)
      (15, 73.8)
      (16, 77.5)
      (17, 73.5)
      (18, 73.3)
      (19, 66.1)
      (20, 76.8)
      (21, 69.1)
      (22, 66.7)
      (23, 69.0)
      (24, 66.9)
      (25, 59.6)
      (26, 65.3)
      (27, 56.2)
    };
    \addlegendentry{LLM-UWB}
    \addplot [solid, color=green!60!black, mark=diamond*] coordinates {
      (1, 82.2)
      (2, 79.4)
      (3, 82.3)
      (4, 79.7)
      (5, 76.5)
      (6, 78.0)
      (7, 77.9)
      (8, 74.9)
      (9, 73.8)
      (10, 69.0)
      (11, 74.4)
      (12, 76.3)
      (13, 73.5)
      (14, 74.0)
      (15, 72.3)
      (16, 71.5)
      (17, 75.7)
      (18, 69.9)
      (19, 69.9)
      (20, 66.4)
      (21, 69.5)
      (22, 71.0)
      (23, 67.7)
      (24, 68.6)
      (25, 56.3)
      (26, 64.6)
      (27, 57.5)
    };
    \addlegendentry{CorPipeLarge}
    \addplot [dashed, color=orange, mark=pentagon*] coordinates {
      (1, 81.2)
      (2, 78.4)
      (3, 75.4)
      (4, 73.7)
      (5, 54.9)
      (6, 76.8)
      (7, 74.1)
      (8, 74.8)
      (9, 71.5)
      (10, 71.1)
      (11, 72.1)
      (12, 75.6)
      (13, 70.6)
      (14, 72.1)
      (15, 69.8)
      (16, 70.5)
      (17, 68.1)
      (18, 67.9)
      (19, 72.9)
      (20, 72.7)
      (21, 63.0)
      (22, 53.4)
      (23, 62.5)
      (24, 69.5)
      (25, 57.9)
      (26, 59.1)
      (27, 44.8)
    };
    \addlegendentry{LLM-PortNLP}
    \addplot [solid, color=green!60!black, mark=*] coordinates {
      (1, 79.1)
      (2, 73.6)
      (3, 77.4)
      (4, 76.8)
      (5, 66.9)
      (6, 73.7)
      (7, 74.6)
      (8, 73.0)
      (9, 70.8)
      (10, 65.2)
      (11, 72.9)
      (12, 72.8)
      (13, 68.6)
      (14, 72.0)
      (15, 68.2)
      (16, 70.2)
      (17, 66.5)
      (18, 71.9)
      (19, 64.5)
      (20, 57.9)
      (21, 42.9)
      (22, 69.0)
      (23, 61.0)
      (24, 61.1)
      (25, 57.2)
      (26, 60.5)
      (27, 56.0)
    };
    \addlegendentry{DAggerCoref}
    \addplot [solid, color=purple, mark=o] coordinates {
      (1, 80.4)
      (2, 73.3)
      (3, 80.1)
      (4, 77.7)
      (5, 64.5)
      (6, 73.7)
      (7, 76.4)
      (8, 69.9)
      (9, 74.0)
      (10, 53.9)
      (11, 72.8)
      (12, 75.5)
      (13, 72.7)
      (14, 70.8)
      (15, 71.2)
      (16, 69.5)
      (17, 73.0)
      (18, 70.4)
      (19, 60.0)
      (20, 61.2)
      (21, 60.9)
      (22, 57.1)
      (23, 66.9)
      (24, 67.1)
      (25, 38.9)
      (26, 59.9)
      (27, 36.9)
    };
    \addlegendentry{Stanza}
    \addplot [solid, color=teal, mark=square*] coordinates {
      (1, 68.2)
      (2, 66.2)
      (3, 70.7)
      (4, 68.8)
      (5, 46.1)
      (6, 68.9)
      (7, 67.9)
      (8, 65.1)
      (9, 68.8)
      (10, 31.1)
      (11, 61.4)
      (12, 66.6)
      (13, 62.0)
      (14, 61.1)
      (15, 69.3)
      (16, 61.8)
      (17, 62.4)
      (18, 52.4)
      (19, 40.6)
      (20, 31.7)
      (21, 50.8)
      (22, 55.6)
      (23, 55.0)
      (24, 65.0)
      (25, 26.9)
      (26, 44.6)
      (27, 6.8)
    };
    \addlegendentry{\baselinegz}
    \addplot [solid, color=gray, mark=triangle*] coordinates {
      (1, 68.2)
      (2, 66.2)
      (3, 70.3)
      (4, 67.9)
      (5, 46.1)
      (6, 67.5)
      (7, 66.2)
      (8, 65.1)
      (9, 63.4)
      (10, 30.6)
      (11, 61.4)
      (12, 66.6)
      (13, 61.9)
      (14, 61.1)
      (15, 66.1)
      (16, 61.8)
      (17, 62.4)
      (18, 52.4)
      (19, 40.6)
      (20, 31.7)
      (21, 46.8)
      (22, 55.6)
      (23, 54.3)
      (24, 65.0)
      (25, 24.7)
      (26, 42.2)
      (27, 6.8)
    };
    \addlegendentry{\baseline}
    \addplot [dashed, color=black, mark=diamond*] coordinates {
      (1, 52.3)
      (2, 63.0)
      (3, 47.3)
      (4, 41.3)
      (5, 46.7)
      (6, 43.2)
      (7, 40.3)
      (8, 58.1)
      (9, 36.7)
      (10, 43.4)
      (11, 54.5)
      (12, 60.6)
      (13, 55.3)
      (14, 53.5)
      (15, 40.9)
      (16, 46.4)
      (17, 52.8)
      (18, 49.3)
      (19, 39.3)
      (20, 61.4)
      (21, 39.8)
      (22, 20.7)
      (23, 37.3)
      (24, 59.5)
      (25, 29.4)
      (26, 41.6)
      (27, 32.8)
    };
    \addlegendentry{LLM-Landcore}
    \addplot [solid, color=blue, mark=pentagon*] coordinates {
      (1, 33.7)
      (2, 34.4)
      (3, 37.4)
      (4, 38.8)
      (5, 28.6)
      (6, 36.2)
      (7, 36.8)
      (8, 41.4)
      (9, 25.8)
      (10, 43.4)
      (11, 42.9)
      (12, 46.2)
      (13, 44.2)
      (14, 39.6)
      (15, 37.9)
      (16, 37.4)
      (17, 28.0)
      (18, 29.8)
      (19, 34.5)
      (20, 48.2)
      (21, 37.2)
      (22, 35.5)
      (23, 31.6)
      (24, 21.6)
      (25, 34.8)
      (26, 29.5)
      (27, 16.2)
    };
    \addlegendentry{AUKBC-MULCRF}
  \end{axis}
\end{tikzpicture}
}
\caption{Plot with results for individual languages in the primary metric (CoNLL F$_1$).
This plot shows the same information as Table~\ref{tab:all-langs},
but languages are sorted according to the performance of the best system
and LLM-based systems are shown with dashed lines.}
\label{fig:alllangs}
\end{figure*}


\begin{table*}\centering
\resizebox{\textwidth}{!}{
\begin{tabular}{@{}l r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{~~~}r@{}}\toprule
\bf system        & \rot\caancora & \rot\cspcedt & \rot\cspdt & \rot\cspdtsc & \rot\cuproiel & \rot\esancora & \rot\grcproiel & \rot\hukorkor & \rot\huszegedkoref & \rot\plpcc & \rot\tritcc\\\midrule
CorPipeEnsemble   &       90  /      87  /      88   &       69  /      79  /      74   &       82  /      86  /      84   &       86  /      86  /      86   &       79  /      80  /      80   &       94  /      93  /      93   &  {\bf 88} /      93  / {\bf 90}  &       65  /      80  /      72   &  {\bf 86} /      74  / {\bf 79}  &  {\bf 94} /      86  / {\bf 90}  &       88  /      84  /      86  \\
CorPipeXXL        &       89  /      86  /      88   &       69  /      77  /      73   &       82  /      86  /      84   &       86  /      85  /      86   &  {\bf 79} / {\bf 81} / {\bf 80}  &       94  /      92  /      93   &       88  /      91  /      89   &       63  /      81  /      71   &       85  /      73  /      79   &       93  /      85  /      89   &  {\bf 89} /      84  / {\bf 87} \\
LLM-LatticeNLP    &       83  /      83  /      83   &       64  /      78  /      70   &       77  /      87  /      82   &       78  /      83  /      80   &       77  /      78  /      78   &       89  /      89  /      89   &       87  / {\bf 93} /      90   &       74  / {\bf 86} /      80   &       74  /      73  /      74   &       89  /      85  /      87   &       87  /      85  /      86  \\
LLM-UWB           &  {\bf 90} /      87  /      88   &       70  /      75  /      72   &       78  / {\bf 92} /      85   &       84  /      83  /      84   &       72  /      76  /      74   &       92  / {\bf 94} /      93   &       87  /      88  /      87   &  {\bf 77} /      85  / {\bf 81}  &       78  /      77  /      78   &       89  / {\bf 89} /      89   &       87  /      79  /      83  \\
CorPipeLarge      &       89  /      82  /      86   &       56  /      80  /      66   &       81  /      84  /      82   &       82  /      86  /      84   &       71  /      71  /      71   &  {\bf 94} /      93  / {\bf 93}  &       83  /      88  /      86   &       70  /      74  /      72   &       84  /      65  /      73   &       93  /      82  /      87   &       88  /      80  /      84  \\
LLM-PortNLP       &       89  / {\bf 89} / {\bf 89}  &       62  /      77  /      69   &       80  /      85  /      82   &       81  /      82  /      82   &       76  /      79  /      77   &       91  /      93  /      92   &       88  /      91  /      89   &       57  /      86  /      69   &       63  / {\bf 78} /      70   &       88  /      86  /      87   &       81  / {\bf 87} /      84  \\
DAggerCoref       &       84  /      86  /      85   &       48  /      80  /      60   &       73  /      80  /      76   &       69  /      79  /      74   &       70  /      76  /      73   &       88  /      92  /      90   &       78  /      82  /      80   &       49  /      86  /      63   &       45  /      59  /      51   &       87  /      87  /      87   &       41  /      79  /      54  \\
Stanza            &       88  /      82  /      85   &       56  / {\bf 82} /      67   &       81  /      83  /      82   &       82  /      83  /      82   &       59  /      65  /      62   &       92  /      90  /      91   &       74  /      84  /      79   &       60  /      78  /      68   &       81  /      74  /      77   &       93  /      81  /      87   &       78  /      79  /      79  \\
\baselinegz       &       83  /      82  /      83   &  {\bf 76} /      81  / {\bf 78}  &  {\bf 85} /      87  / {\bf 86}  &  {\bf 89} / {\bf 88} / {\bf 89}  &       57  /      69  /      63   &       89  /      89  /      89   &       63  /      69  /      66   &       59  /      75  /      66   &       51  /      57  /      54   &       90  /      88  /      89   &       76  /      81  /      79  \\
\baseline         &       81  /      74  /      77   &       50  /      74  /      60   &       75  /      78  /      77   &       79  /      83  /      81   &       49  /      58  /      53   &       87  /      86  /      86   &       61  /      69  /      65   &       48  /      60  /      53   &       50  /      53  /      52   &       89  /      79  /      83   &       72  /      71  /      71  \\
LLM-Landcore      &        0  /       0  /       0   &       10  /      39  /      16   &       12  /      46  /      19   &       42  /      44  /      43   &        6  /      10  /       8   &        3  /      39  /       5   &       29  /      61  /      40   &       24  /      51  /      33   &        2  /       9  /       3   &        4  /      40  /       7   &       41  /      55  /      47  \\
AUKBC-MULCRF      &       47  /      50  /      49   &       26  /      37  /      30   &       50  /      54  /      52   &       71  /      65  /      68   &       62  /      61  /      62   &       53  /      58  /      56   &       71  /      72  /      72   &        0  /       0  /       0   &        0  /       0  /       0   &        0  /       0  /       0   &       74  /      65  /      70  \\
\midrule
\sys{Winner-2025} &       91  /      87  /      89   &       61  /      79  /      69   &       82  /      86  /      84   &  --                              &       77  /      80  /      79   &       93  /      92  /      92   &       87  /      87  /      87   &       65  /      81  /      72   &       85  /      73 /       78   &       93  /      84  /      89   &       84  /      83  /      84  \\
\sys{Baseline-2025} &     79  /      75  /      77   &        9  /      93  /      17   &       34  /      82  /      48   &  --                              &       52  /      62  /      57   &       88  /      87  /      87   &       62  /      67  /      64   &       56  /      63  /      59   &       54  /      57  /      55   &       86  /      78  /      82   &       71  /      73  /      72  \\
\bottomrule\end{tabular}
}
\caption{Recall / Precision / F1 for anaphor-decomposable score of coreference resolution on \textbf{zero anaphors} across individual languages.
Only datasets containing anaphoric zeros are listed (\engum{} excluded as all zeros in its test set are non-anaphoric).
Note that these scores are directly comparable to neither the CoNLL score nor the supplementary scores calculated with respect to whole entities in Table~\ref{tab:secondary-metrics}.
}
\label{tab:all-langs-zero}
\end{table*}
%\formn{pozriet sa preco su LLM lepsie na hu-korkor, vyselektovat Anji priklady}


\input{figures/tikz-plots/scores-over-years}



%\input{plot-zero-anaphor}

%Annual comparison of the results performed by baselines run in the gold zero setup (\baselinegz{} and \baselinelast{}) shows similar scores on zeros, which confirms that these baselines are comparable.
%The only exception is \plpcc{}, on which \baselinegz{} improved by 25 percentage points.
%This can be explained by the fixes in the CorefUD conversion pipeline from the source corpus that focused on zeros.
%The annual comparison of relative improvements of the best systems over these baselines in terms of the zero anaphora score reveals that the improvements are much lower than they were last year, again confirming the more difficult nature of this year's setup for zeros.


\paragraph{Further analysis}
As in previous years, we provide several additional tables in the
appendices to shed more light on the differences between the submitted
systems. Discussions of these tables are provided in their captions.

Tables~\ref{tab:upos-entity}--\ref{tab:upos-mention} in Appendix~\ref{sec:stats-upos} show results factorized
according to the different universal part-of-speech tags (UPOS) in the
mention heads. 

Tables \ref{tab:stats-entities}--\ref{tab:stats-details} in Appendix~\ref{sec:stats-concat} show various
statistics on the entities and mentions in a concatenation of all the test
sets. Note that such statistics are mostly influenced by larger datasets.

%\paragraph{Differences between LLM and Unconstrained}
% Martin: Loni jsme zde psali, že je to nové a že gap je velký. Ani jedno letos není pravda.
% LLM a UNC porovnáváme v textu několikrát, třeba hned na začátku Sekce 5.
% Já bych tedy tento odstavec zde vynechal. V Conclusions můžeme ještě něco o LLM napsat.

\paragraph{Performance on long-range entities}
Appendices \ref{sec:stats-litbankfr} and \ref{sec:stats-openboek} show the same type of tables as Appendix~\ref{sec:stats-concat},
but for the top two long-entity datasets (with the highest p95 range according to Table~\ref{tab:sizes}),
French-LitBankFr and Dutch-OpenBoek, respectively.

In French-LitBankFr, we can see that LLM-UWB, LLM-PortNLP, and Stanza
have p95 ranges (9370, 6729, and 6336, respectively) that are higher than the gold data (4435),
while LLM-Landcore's p95 range is the closest (3575).
All other systems generate entities that are much shorter than those in the gold data.

In Dutch-OpenBoek, LLM-Landcore is again the closest;
Stanza generates longer entities than the gold data;
and all other systems generate much shorter entities.

To better understand how systems handle long-range entities, we conduct a document-level analysis.
We split the official mini-test set into individual documents, compute the primary score for each document, and examine performance as the document p95 range increases.
To reduce noise, we report a sliding-window average of CoNLL F1.
Because short-range documents contain fewer entities, a fixed number of documents per window would yield unstable estimates.
We therefore define windows by a fixed maximum number of tokens.
This procedure is only an approximation: documents with a high p95 range still contain many mid- and short-range entities, which can influence the aggregate score.\footnote{
 We have tried to design a metric that would focus on ``long'' entities only
  (or, in general, on entities of a given range), but we were not successful
  -- e.g., if a system-predicted entity is slightly shorter than the threshold,
  it would be counted as completely missing, which would be too harsh and lead to non-intuitive (unfair) evaluation results.
}
Finally, entity range is not the only driver of document-level performance; language, genre, and annotation guidelines also matter.

Figure~\ref{fig:scores-long-range} presents results for documents whose p95 range exceeds 100 tokens, computed with a 50k-token sliding window.
Systems vary markedly in their ability to handle long-range entities.
Among the best-performing systems, CorPipe and LLM-LatticeNLP remain robust as the range increases, whereas the other systems deteriorate noticeably.
The contrast between LLM-UWB and LLM-LatticeNLP is particularly informative: despite similar overall performance (Table~\ref{tab:main-results}) and a largely shared setup (Table~\ref{tab:context-size}), their behavior diverges across entity ranges.
LLM-UWB performs slightly better on mid-range documents, and the two systems are comparable for p95 ranges between 1k and 2k tokens.
For the longest-range documents, however, LLM-LatticeNLP maintains performance and achieves the best scores overall, while LLM-UWB drops by roughly 15 F1 points.\footnote{
As an alternative to entity range, we have also sorted the documents by the maximum distance between adjacent mentions (in any entity in a given document).
We omit the resulting graph because it is very similar to Figure~\ref{fig:scores-long-range}
-- the CoNLL F1 scores for each document are the same, and the ordering of documents is similar because p95 entity range correlates with the maximum distance between adjacent mentions.
}

\input{figures/tikz-plots/scores-long-range}
%\input{figures/tikz-plots/scores-max-ante-dist}
