\section{Results}
\label{sec:results}

\begin{table}[b]
\centering
\small
\setlength{\tabcolsep}{4pt}
\begin{tabular}{|l|c|c|c|}
\hline
 & \textbf{Level-1} & \textbf{Level-2} & \textbf{Level-3} \\
\hline
\multicolumn{4}{|l|}{SG} \\
\hline
Agree coef. & $0.210$ & $0.240$ & $0.219$ \\
Freq. coef.  & $0.420$ & $-0.457$ & $-0.445$ \\
$R^2$       & $0.219$ & $0.244$ & $0.137$ \\
\hline
\multicolumn{4}{|l|}{AE} \\
\hline
Agree coef. & $-0.101$ & $0.334$ & $0.338$ \\
Freq. coef.  & $1.900$ & $1.061$ & $2.362$ \\
$R^2$       & $0.575$ & $0.470$ & $0.455$ \\
\hline
\end{tabular}
\caption{Regression coefficients and $R^2$ for OLS predicting confidence from agreement and label frequency. Both variables are significant across all levels with $p \le 0.01$.}
\label{tab:ols_coefs}
\end{table}

\subsection{Cartography plots}
\label{sec:results:plots}

Data maps project each training data point onto the \textbf{variability} / \textbf{confidence} coordinate plane. As the definitions suggest (see Section \ref{sec:background}), \textit{easy-to-learn} items occupy the upper left corner of the map, while \textit{hard-to-learn} occupy the lower left part. Highly variable data, which belongs to neither category, forms the right part of the map. The data points can be colored to encode additional variables, such as their class labels or binned agreement scores, i.e., the proportion of votes that agrees with the majority.

The visual analysis of the obtained data maps reveals several trends. 
First, in  Figures \ref{fig:AE_agr_small}, \ref{fig:sg_agr}, and \ref{fig:AE_agr}, the data maps all exhibit the commonly seen wedge shape. Another global trend, characteristic of data maps \cite{swayamdipta_dataset_2020}, is the interaction between agreement and \textbf{confidence}, apparent in the \textit{easy-to-learn} region. However, the maps for AE exhibit clearer \textit{easy-} and \textit{hard-to-learn} regions with low variability.

On the other hand, \textbf{confidence} visibly interacts with the class structure. Independently of the training method, \textit{expansion} as the most frequent label dominates the \textit{easy-to-learn} region at level-1, even surpassing the temporal relations. On the other hand, at levels 2 and 3, where \textit{expansion} splits up into smaller classes, \textit{asynchronous} and \textit{precedence} consistently emerge as the easiest to learn and visibly coincide with high agreement. 

For AE, the data maps offer an insight into the composition of the \textit{hard-to-learn} region, as it can be seen that infrequent labels, such as \textit{no relation} on level-1, largely have low \textbf{confidence} and \textbf{variability}. Comparing AE with the majority-label classifier, we note that the latter yields a sparsely populated \textit{hard-to-learn} region that has no predominant label at any of the 3 levels. Likewise, due to high \textbf{variability}, no item at levels 2 and 3 can be categorized as strictly \textit{easy-to-learn}. At the same time, AE displays clearer \textit{easy/hard-to-learn} subsets with fewer data points concentrated in the highly variable region. The quantitative results also reveal a drastic difference between SG and AE.

\subsection{Cartography statistics}
\label{sec:results:stats}

To complement the results of visual interpretation in Section \ref{sec:results:plots}, we quantify how \textbf{confidence} relates to item-wise agreement and label frequency, two properties that emerged as impactful in our visual analysis. In particular, we roughly define the \textit{hard-to-learn} subset as the bottom quartile of \textbf{confidence} values and report enrichment as an odds ratio (OR) relating the prevalence of low agreement items/rare labels inside vs.\ outside that quartile.

As shown in Table \ref{tab:OR}, low agreement is only weakly enriched among low-confidence items across both model types, SG and AE (ORs 1.13--1.75), suggesting that disagreement alone does not define the \textit{hard-to-learn} subset. In contrast, label rarity shows a much stronger effect, especially for AE. For SG, rare labels are not consistently concentrated in low confidence (OR = 1.74, 0.54, and 3.75 across levels 1-3), whereas for AE they are strongly enriched at every level, with the effect monotonically increasing with label granularity (OR = 7.47, 17.21, and 25.8). This pattern is also reflected in Figure \ref{fig:freq_boxplots}, where rare labels occupy markedly lower confidence ranges, particularly for AE; the Mann-Whitney test shows a significant difference in how the \textbf{confidence} values are distributed.

Elaborating on this result, we further model \textbf{confidence} globally as a function of agreement and frequency using OLS regression (Table \ref{tab:ols_coefs}). Both predictors are significant across all levels, but their contribution differs sharply depending on model type. For SG, agreement has a small positive coefficient throughout, whereas frequency turns negative at levels 2 and 3, suggesting unstable interactions between majority-label training and fine-grained class structure. For AE, frequency is the dominant predictor, with large positive coefficients across all levels and substantially higher explained variance ($R^2 = 0.455$-$0.575$ vs.\ $0.137$-$0.244$ for SG). Overall, low confidence in DiscoGeM appears to reflect sparsity and ambiguity more than annotation error.

\subsection{Auditing high-confidence labels}
\label{sec:results:lexical}

The cartography statistics point at temporal labels as the most consistent \textit{easy-to-learn} cluster in DiscoGeM; particularly, \textit{precedence} at level-3. Meanwhile, they offer no explanation for the trend. One likely reason is that models may exploit local lexical cues, and the same cues are also available to annotators.

In a further experiment, we address this hypothesis by training a lexical probe: a one-vs-all logistic regression over unigram and bigram features. We insert a special boundary token, \textbf{sep}, between Arg1 and Arg2, thus enabling the bigram model to capture Arg2-initial keywords. This setup is repeated for raw labels and majority-aggregated labels across 5 cross-validation folds. We report the results in Table~\ref{tab:bag-of-words}.

\begin{table}[]
    \small
    \centering
    \begin{tabular}{|l|c|c|c|}
        \hline
        Metric & \textbf{Level-1} & \textbf{Level-2} & \textbf{Level-3} \\
        \hline
        Acc.Maj & $0.86 \pm 0.02$ & $0.90 \pm 0.02$ & $0.90 \pm 0.01$ \\
        F1.Maj & $0.62 \pm 0.03$ & $0.60 \pm 0.06$ & $0.61 \pm 0.02$ \\
         \hline
        Acc.Raw & $0.87 \pm 0.00$ & $0.93 \pm 0.00$ & $0.93 \pm 0.00$ \\
        F1.Raw & $0.67 \pm 0.00$ & $0.71 \pm 0.01$ & $0.68 \pm 0.02$ \\
        \hline
    \end{tabular}
    \caption{Results of the lexical probe for \textit{temporal}, \textit{asynchronous}, and \textit{precedence} for levels 1, 2, and 3 respectively. Metrics averaged across 5 cross-validation folds. Macro-F1 $\ge 0.5$ surpasses the majority label baseline.}
    \label{tab:bag-of-words}
\end{table}

Across the three granularity levels, the lexical probe retains high accuracy along with Macro-F1 that surpasses the majority label baseline ($\approx0.5$ for all settings). The accuracy shifts across the three levels likely reflect the changes in label balance, as the \textit{temporal} class is more populated than \textit{asynchronous} or \textit{precedence}: $13$ vs $9\%$. On the other hand, the F1 score is greater or equal at granular levels, especially for unaggregated labels; this aligns with the clearer clustering of temporal items in the data maps, especially at finer sense levels. Overall, the metrics suggest that temporal relations in DiscoGeM are at least partly recoverable from surface features.

By inspecting the highest positive regression coefficients, we identify the ngrams that are most predictive of the temporal class: \textbf{sep after}, \textbf{sep when}, \textbf{sep suddenly}, \textbf{sep as}, and \textbf{after that} (see Figure \ref{fig:feats} for an overview). Three of them are intra-argument explicit connectives that introduce a dependent clause within Arg2. As their scope is restricted to Arg2, the relation between Arg1 and Arg2 \textbf{remains implicit}.  However, the annotators prompted to insert a connective between Arg1 and Arg2 were not instructed on such cases, which may be the reason why many of them converged on the \textit{precedence} label.

To test whether these ngrams correspond to a distinct cartographic subset, we split temporal instances into cue-rich and cue-light groups. Cue-rich instances contain one of the boundary-adjacent ngrams at the start of Arg2, while cue-light instances do not. Table~\ref{tab:cue-rich-temporal} shows that cue-rich temporal items have the highest agreement and slightly higher confidence than the non-temporal data, while cue-light temporal items have lower confidence and agreement. This suggests that the high-confidence temporal cluster is concentrated in a cue-rich subset.

We also compute the normalized pointwise mutual information (nPMI) between the relevant ngrams and 5-10 most frequent relation types for each level (see Appendix \ref{sec:app2}, Figure~\ref{fig:bigram_npmi}). At each level, the ngrams are positively associated with the temporal target -- \textit{temporal}, \textit{asynchronous}, and \textit{precedence}; conversely, other labels have no positive association. This trend provides an explanation for the temporal subset coinciding with high model confidence and high annotator agreement, as the same local cues are available both to the model and to annotators.



\begin{table}[t]
\centering
\small
\begin{tabular}{|l|c|c|c|}
\hline
Subset & Var. & Conf. & Agreement \\
\hline
Non-temporal & 0.306 & 0.730 & 0.559 \\
Temporal, cue+ & 0.330 & 0.742 & 0.686 \\
Temporal, cue- & 0.339 & 0.637 & 0.560 \\
\hline
\end{tabular}
\caption{
Cartography statistics for cue-rich and cue-light temporal instances as opposed to non-temporal instances.
}
\label{tab:cue-rich-temporal}
\end{table}