\section{Background}
\label{sec:background}

Dataset cartography was the first method in the larger training dynamics family; these methods track how specific examples affect the trained model across epochs \cite{swayamdipta_dataset_2020}. 
%Dataset 
Cartography statistics describe each training example and derive from the predicted probability of the correct class, $P_{corr}$.
This translates into several variables: 
\textbf{confidence} (mean $P_{corr}$ across epochs), 
\textbf{variability} (standard deviation of $P_{corr}$), 
\textbf{correctness} (ratio of epochs where $P_{corr}$ > 0.5).
%As per this scheme, 
All training data fall into one of the three categories: \textit{hard-to-learn} (low confidence \& variability), \textit{easy-to-learn} (high confidence, low variability), \textit{ambiguous} (high variability). 
Crucially, \citet{klie_annotation_2023} found that data map \textbf{confidence} can reliably estimate annotation quality across a variety of NLP tasks, often outperforming other error detectors.

At the same time, \citet{anand_dont_2024} found that data maps learned from majority labels tend to categorize items with high annotator disagreement -- i.e., genuinely ambiguous data -- as \textit{hard-to-learn}. As a result, filtering based on this categorization ignores valuable information. This claim is in line with prior research showing that aggregation by majority voting can lead to information loss \cite{pavlick2019inherent,larimore-etal-2021-reconsidering,plank-2022-problem} and that models that learn from raw annotations may be able to mitigate these effects \cite{uma2021learning,frenda2024perspectivist}.

Such disagreement-aware methods have also been shown to extend well to PDTB-style discourse parsing and even offer advantages over majority-label training 
\cite{yung-etal-2022-label,pyatkin-etal-2023-design,costa-kosseim-2024-exploring,long-etal-2024-multi,yung_human_2026}. These findings can be grounded in the relevant theory, as PDTB allows for concurrent readings \cite{prasad2017penn}, and, in practice, annotators tend to infer several concurrent relations \cite{rohde_discourse_2018} or pick diverging, but mutually compatible labels \cite{yung2019crowdsourcing}.

Our analysis of the \textit{easy-to-learn} subset deals with cue-based biases, which have been studied in detail for natural language inference and SNLI specifically \cite{bowman_large_2015}. These include both lexical \cite{gururangan_annotation_2018,feng_misleading_2019} and syntactic \cite{mccoy_right_2019} heuristics that the models acquire over the course of training.

\section{Experimental setup}
\label{sec:dataset}
DiscoGeM~1.5 \cite{scholman-etal-2022-discogem,yung-demberg-2025-crowdsourcing} is the largest crowdsourced corpus of PDTB-style implicit discourse relations that retains unaggregated crowd labels. The data collection procedure relied on the well-documented connective insertion method \cite{yung2019crowdsourcing}.

Version 1.5 focuses on English, the data amounting to $6489$ text pairs labeled by $164$ crowd workers with each pair having $10$ crowd labels on the average. We focus on the canonical train subset of $4560$ items.
% \citealp{yung_human_2026} note that the crowd labels include some hard errors stemming from cognitive load, but also largely comprise mutually compatible individual readings.

We report our results at three levels of sense granularity in PDTB: level-3 (29 senses), level-2 (17 senses) and level-1 (5) \citep{webber2019pdtb3}. Disagreement is especially visible on level-3 ($\kappa=0.404$), since the workers have to choose between semantically similar alternatives: for example, \textit{precedence}, \textit{succession} or \textit{synchronous} within the coarse \textit{temporal} group. Although the coarser levels are assumed to be easier to learn \cite{xiang2023survey}, they also group heterogeneous data, as each granular class is associated with its own signals.

Our models for data mapping are based on DeBERTa-v3-base \cite{he_debertav3_2023}, an encoder-type model. For each of the three PDTB levels, we train two models: \textbf{SG}, a standard majority-label classifier, and \textbf{AE}, an instance of the Annotator Embeddings model \cite{deng-etal-2023-annotate}, which learns annotator-specific labeling behavior from raw labels paired with annotator ids. We employ each model to obtain \textbf{confidence}, \textbf{correctness} and \textbf{variability}.

% As our interpretation of the data maps points at strong \textit{temporal} cues in the dataset, we attempt to isolate them and analyze their interaction with the granularity levels. We do so by predicting the \textit{temporal}/\textit{asynchronous}/\textit{precedence} label in the one-vs-all setting using logistic regression on top of bag-of-words representations as implemented in \texttt{scikit-learn} \cite{scikit-learn} with 5-fold cross-validation. To make this model sensitive to argument boundaries, we inject the word \textbf{sep} between Arg1 and Arg2 and condition the model on bigrams. Lexical features with high positive coefficients can then reveal the likely cues. Upon identification, we compute the normalized Pointwise Mutual Information (nPMI) between the cues and the most frequent labels to verify that these are consistently associated with temporal labels.

\begin{table}[b]
\centering
\small
\setlength{\tabcolsep}{4pt}
\begin{tabular}{|l|c|c|c|}
\hline
 & \textbf{Level-1} & \textbf{Level-2} & \textbf{Level-3} \\
\hline
LowAgree.OR.SG & $1.75$ & $1.48$ & $1.51$ \\
RareLabel.OR.SG    & $1.74$ & $0.54$ & $3.75$ \\
\hline
LowAgree.OR.AE & $1.13$ & $1.31$ & $1.46$ \\
RareLabel.OR.AE     & $7.47$ & $17.21$ & $25.8$ \\
\hline
\end{tabular}
\caption{Odds ratios for enrichment in the low-confidence (\textit{hard-to-learn}) quartile.}
\label{tab:OR}
\end{table}
