\section{Conclusion}

Overall, our results point to two conclusions. First, \textit{hard-to-learn} DiscoGeM instances cannot be reliably interpreted as erroneous, as agreement level and label frequency are two factors that strongly impact the \textbf{confidence} parameter while not directly translating into annotation quality. Rather, they point to item ambiguity, and filtering out \textit{hard-to-learn} items would thus mean removing the ambiguous part, which dataset cartography normally suggests to keep for training.

On the other hand, the \textit{easy-to-learn} region can pose a problem for robust learning, since for some classes, the right label can often be recovered from cues. nPMI and accuracy show that this trend is exacerbated on more granular levels. In general, this shows that models optimized for discourse labeling can be prone to acquired bias -- similarly to NLI, where models are known to learn lexical or syntactic shortcuts during large-scale training \cite{feng_misleading_2019,mccoy_right_2019}. In contrast to NLI, however, relatively few studies have focused on these artifacts and on the robustness of predictions. As data maps can reliably and interpretably identify some of such shortcuts, we hope this study can make a case for qualitative evaluation of models in discourse parsing, regardless of the specific annotation standard.

Lastly, future collection of IDRR data as well as model analysis may need to treat cue-rich arguments explicitly. DiscoGeM notably includes cue-light classes -- particularly, \textit{no relation}; as a result, cue-driven items may introduce an imbalance, since some relations require the model to interpret the underlying pragmatics and some can be inferred from strong lexical cues. 
% In this sense, cartography does not solve the problem of ambiguity; it helps reveal which parts of the dataset are driven by ambiguity, and which by more accessible surface signals.

% This study attempts to shed more light on how the practice of crowdsourcing linguistic data interacts with model training in the challenging field of IDRR. 
% Specifically, we analyze whether dataset cartography \cite{swayamdipta_dataset_2020} can help identify errors and interpretations with the help of annotator-aware architectures \cite{frenda2024perspectivist}. We find that error identification based on low confidence is infeasible for IDRR datasets; however, high confidence in models coincides with strong heuristics and with high item-wise agreement, pointing at shortcut usage by both models and crowd workers. Likely stemming from the crowd workers' limited training, this case needs to be treated explicitly to ensure robust model training.