\subsection{Causes of disagreement}
\label{sec:qualitative}

%\subsection{Taxonomy of Disagreements}
%Our analysis of disagreements in DiscoGEM is an attempt to address the question why learning the individualized labels from this dataset appears harder compared to other subjective tasks.  In the meantime, none of the NLP tasks covered in these studies are sufficiently similar to IDRR, where the lexical properties of connectives and the sentence syntax may influence the final result. This means that the existing classifications need to be extended and reframed, so as to address our specific problem.

%Our classification was constructed in the following way: 1 annotator inspected a representative subset of the dataset to identify the possible disagreement heuristics. Drawing on the example of \citet{jiang-marneffe-2022-investigating}, we employed the absolute number of distinct labels (level 2) as a criterion for initial data sampling. Out of 6440 total instances in DiscoGEM 1.5, 10\% most diversely and least diversely annotated items (N labels \~ 10 and \~5 respectively) were sampled for inspection. We acknowledge that this methodology does not ensure the completeness of our taxonomy, especially given the complexity of the task. However, we argue that the identified types of disagreements are representative of a large portion of data and that our suggestions on how disagreement should be handled may result in a more accurate model of the crowd knowledge.

%
% iteratively updated . higher quality scores

%In the previous section, we observed that workers exhibit both systematic biases toward certain relations, which the perspectivist model can learn, and random, noisy behaviors that the model fails to predict. In this section, we conduct a qualitative analysis to identify concrete examples illustrating these phenomena.

We selected a subset of $100$ samples that were consistently annotated by $3$ workers, referred to here as Annotators X, Y, and Z. %\footnote{These three workers have the largest number of overlapping annotations among all workers. Out of 144 overlapping samples annotated by all three, we analyzed 50 due to time constraints.} 
For each Level-2 label assigned by the workers, we examined the nature of the disagreement and categorized it according to the potential underlying reason for divergence. When a worker’s label matched one or more of the others, it was counted as a case of agreement. Table~\ref{tab:reasons} presents the distribution of disagreement categories, while Table~\ref{tab:top3} summarizes the three most frequently assigned labels by each worker within this subset.  We discuss these categories and their implications in detail below.

\begin{table}[ht]
    \centering \small
    \begin{tabular}{lll|l}
        \hline
         \multicolumn{3}{l|}{agreement} & $35\%$  \\
         \hline
         \multicolumn{3}{l|}{disagreement} &\\
         &\multicolumn{2}{l|}{inherent ambiguity} & $31\%$\\
         &\multicolumn{2}{l|}{cognitive demands:} &\\
         && unidentified cues & $5\%$\\
         && sentence complexity & $10\%$\\
         && annotation error & $11\%$\\
         &\multicolumn{2}{l|}{task design} & $8\%$\\
         \hline      
    \end{tabular}
    \caption{Distribution of disagreement categories in $300$ annotations ($100$ items $\times$ $3$ workers)}
    \label{tab:reasons}
\end{table}

% \begin{table}[ht]
%     \centering \small
%     \begin{tabular}{l|lll}
%     \hline
%         Ann. & top1 & top2 & top3 \\
%         \hline
%         A & asynch. & conjunction  & cause \\
%          &  ($18\%$)&  ($16\%$) &  ($12\%$) \\
%         B & lev-of-det& cause  & condition \\
%          & ($28\%$)&  ($14\%$) &  ($10\%$) \\
%         C & lev-of-det & cause  & concession \\
%          & ($16\%$)&  ($16\%$) &  ($10\%$) \\
%         \hline
%     \end{tabular}
%     \caption{Top $3$ annotated labels by Annotators X, Y and Z in the $50$ analyzed items}
%     \label{tab:top3}
%\end{table}
%
\begin{table}[ht]
    \centering \small
    \begin{tabular}{@{}l|lll@{}}
    \hline
        & top1 & top2 & top3 \\
        \hline
        X & conj.($20\%$) & asynch.($13\%$)  & inst. ($10\%$) \\
        Y & lev-of-det ($22\%$)& cond. ($11\%$)  & synch.  ($9\%$) \\
        Z & cause ($23\%$) & lev-of-det ($11\%$) & goal ($9\%$) \\
        \hline
    \end{tabular}
    \caption{Top $3$ annotated labels by Annotators X, Y and Z in the $100$ analyzed items}
    \label{tab:top3}
\end{table}
%
Table~\ref{tab:reasons} shows that over one-third of the annotations fall under \textbf{agreement}, where the assigned label matches at least one of the other workers for the same item. Another one-third of the annotations were found to differ due to the \textbf{inherent ambiguity} of the DRs. This ambiguity arises from a variety of causes.
%
A primary source of ambiguity stems from differences in how workers interpret or emphasize \textbf{different spans} within the relation arguments.
In Example (2), Annotator Z focused only on the beginning of Arg2, interpreting the relation as \textit{concession} (i.e., the man opened the window even though it was raining). In contrast, Annotator Y considered the full span of Arg2 and identified the DR as \textit{synchrony}, emphasizing the simultaneous nature of the two events.

\ex {Arg1: \textit{What was the man about to do? He lowered the window on our side.}
Arg2: A heavy rain was now falling, and, by a gesture, the man expressed his annoyance at his not having an umbrella...}% or an overcoat...}

% \textbf{No Context.} \citet{sandri-etal-2023-dont} distinguish a type of disagreements, where a missing antecedent is preventing the annotators from uniformly interpreting an anaphoric expression. This category occurs often in DiscoGEM data, as many of its items contain unresolved anaphoric expressions in their first argument for reasons that we outline below. Resolving them is not always necessary, as \textit{arg2} can contain sufficient hints to infer the relation without placing the sentence pair into the larger context. However, when this is not the case, understanding the first argument may prove crucial in inferring the relation. Consider the following example:

% \textnormal{ARG1: \textit{'This directive is a contribution to this.'}, ARG2: \textit{'What we are doing today is essentially a nuisance.'} [de\_en\_batch\_02\_item\_02]} 

% An unambiguous reading of \textit{arg1} would clearly make it easier to infer the IDRR relation, as it would at least clarify if it is the speakers' adoption of the directive that amounts to "nuisance" or if "nuisance" refers to something else in the context. Examples like this align fully with the existing classification.

% Meanwhile, we also argue that the \textbf{No Context} category applied to DiscoGEM can be extended to cover a broader set of cases: namely, those where the context of \textit{arg1} does not include at least 1 preceding sentence. In our view, this missing context can complicate the identification of certain relations — primarily, \textsc{norel} — and causes them to be selected more rarely and by fewer crowd workers. Notably, \textsc{norel} is annotated infrequently in all of the corpus versions \cite{yung-demberg-2025-crowdsourcing}. 

% In our view, the missing context obstructs what seems to be a reliable \textsc{norel} heuristic: sharing of the topic and/or entities between \textit{arg2} and the preceding context of \textit{arg1}, but not between \textit{arg2} and \textit{arg1} itself. Although PDTB 2.0 does not explicitly posit this rule and prescribes \textsc{norel} only when no other relation is applicable, two of its \textsc{norel} illustrations (examples 96 and 97) clearly demonstrate this principle \cite{prasad2017penn}:

% \textit{The \underline{products} already available are cross-connect systems} [...]. ARG1: \textit{This cuts down greatly on labor, Mr. Buchner said.} ARG2: \textit{To be introduced later are \underline{a multiplexer} \textnormal{[...]}; \underline{a light-wave system} \textnormal{[...]}; and \underline{a network controller}.} (1064)

% Understandably, given DiscoGEM 1.5's task formulation, the crowd annotators are not expected to be aware of this heuristic and to make regular use of it. However, they also cannot apply it intuitively, something that we could expect if context sentences were available \footnote{Where it is available and this heuristic is observed, at least one annotator uses \textsc{norel} in 6 out of 7 cases.}. Moreover, missing context would also pose a problem if trained annotators or automated shallow discourse parsers were tasked with judging or reconciling the crowd annotations in DiscoGEM, as there would not be enough information to apply the rules.

% algorithm

% The algorithm adopted in DiscoGEM for sampling IDRR items from source texts \cite{scholman-etal-2022-discogem} may have brought forth many of the \textbf{No Context} occurrences. In particular, as per the algorithm, the first argument of a relation could include at least one sentence (\textit{arg1} itself) plus an arbitrary number of context sentences as long as the entire span did not exceed 50 words. Additionally, when the window overlapped with a conversational turn switch or a paragraph boundary, it would be cut to start at that mark. In practice, this resulted in a considerable number of instances where the first argument would only comprise one sentence, \textit{arg1}, leading to the issues mentioned above. In that respect, DiscoGEM differs from the existing PDTB or RST treebanks, as in those, the annotators are exposed to either the entire text or, at least, a sufficient number sentences to contextualize \textit{arg1}.

% % discard problematic items? Compute agreement afterwards

% The said issue is hard to address in the already collected data; at the same time, we can hypothesize a solution for future data collection. Specifically, prioritizing sentential heuristics over word count and sampling a fixed number of sentences (e.g., at least \textit{arg1} and one context sentence) may produce more informative contexts. The reason for that is that the \textit{arg1} sentence may be arbitrarily long and, at the same time, it may crucially depend on the preceding sentences with respect to its status in the discourse (as in the case of \textsc{norel}). 

% Another suggestion may be to use automated coreference resolution tools, so as to see if either argument shares any entities with the preceding context. When this is the case, the maximum length of 50 words may be ignored to include the relevant information. We hypothesize that these adjustments are likely to facilitate the crowd workers' overall understanding of the discourse. With that in mind, we also realize that tuning the algorithm in such a way is not guaranteed to elicit higher agreement scores, as larger contexts are likely to increase the annotators' cognitive effort and subsequently demotivate them \cite{scholman-etal-2022-discogem}. Practical data collection rounds are required to judge a the validity of this suggestion. 

Another source of disagreement arises from the \textbf{lack of contextual information}. Because DiscoGeM is designed to capture the organic, local interpretation of DRs, 
%unlike corpora annotated by trained experts such as PDTB,
workers are instructed to label relations given only one to two sentences of context preceding Arg1 and following Arg2. Consequently, the interpretation of a DR can vary depending on each worker’s inferred or assumed broader context.
In the narrative fragment of Example (3), the relationship between the characters and the nature of the events remain unclear without access to the wider context of the novel, leading to annotator disagreement. %X, Z: because. Y: no relation

\ex {Arg1: \textit{Pianelli was constantly looking for a scoop, but I’d often wondered what lay behind his fascination with this particular case.}
Arg2: From what I remembered, he and Vinca had never been close. They’d never spent time together...}

%
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
In addition, workers also exhibit \textbf{individual preferences} when labeling ambiguous DRs.  In Example (4), both Arg1 and Arg2 refer to the same entity pair (Fanny and “I”), yet the DR between them is not obvious. This type of relation corresponds to what the PDTB framework classifies as an \textit{entity relation} \cite{prasad2017penn}, which is typically implicit in natural text and often resists representation by an explicit connective. In this case, Annotator X labeled the relation as \textit{conjunction}, whereas Annotator Y chose \textit{level-of-detail}. Both interpretations are reasonable and highlight distinct annotation preferences toward particular relation types, preferences that are also reflected in the individual label distributions shown in Table~\ref{tab:top3}.
\ex {Arg1: \textit{...Fanny was the first girl I ’d ever dated, back when I was fourteen and in the ninth grade.}
Arg2: We went to see a Saturday matinee of Rain Man...}
% \textbf{EntRel-type relations.} PDTB 2.0 recognized the \textsc{entrel} relation as one of the cases where a connective cannot be meaningfully inserted \cite{prasad2017penn}. Since DiscoGEM annotators could not indicate an entity-based relation, only the connective-grounded relations or \textsc{norel}, they often struggled to agree on a connective in cases of this kind. Straightforward \textsc{entrel} candidates, where \textit{arg2} points at a physical or a legal entity introduced in \textit{arg1}, are not uncommon in DiscoGEM 1.5. Particularly, in the \textbf{Wiki} subset, where the genre is, relatively seen, the closest to the news articles that form the original PDTB data. 

% For instance, the following text elicits as many as 7 distinct level-2 annotations, including some incompatibles, such as \textsc{synchronous} and \textsc{asynchronous}; this points at a possible confusion among the annotators.

% ARG1: \textit{\underline{Nazi Germany}, \textnormal{[...]}, was the German state between 1933 and 1945, when Adolf Hitler and the Nazi Party controlled the country which they transformed into a dictatorship.} ARG2: \textit{Under Hitler's rule, \underline{Germany} quickly became a totalitarian state}... [wiki\_571]

% At the same time, DiscoGEM also includes a number of borderline cases, in which \textsc{entrel} can arguably be applied to discourse-deictic relations. In particular, the utterance pronounced in \textit{arg1} is referred to as an entity in \textit{arg2}; e.g., as "this question" or "this statement".

% ARG1: \textit{The overarching objectives of the structural funds \textnormal{[...]} are only mentioned in passing.} ARG2: \textit{\underline{This attitude} appears to me to be unjustified.} [de\_en\_batch\_02\_item\_05]

% Here, "this attitude" refers to a fact introduced in \textit{arg1}, making this sentence a possible \textsc{entrel} candidate. \textit{Per se}, this analysis may appear less than intuitive; however, the standard does not provide direct instructions for handling discourse-deictic expressions. At the same time, the PDTB corpus itself includes a number of examples where an utterance cited as direct speech is analyzed as an entity.

% ARG1: \textit{\underline{Some long-tenured employees} \underline{will receive additional benefits}, the company said.} ARG2: \textit{Hours after \underline{the announcement}, representatives of the Orange County Register were in a bar across the street recruiting.} [wsj\_0114]

% Whether or not similar examples from DiscoGEM should be analyzed as \textsc{entrel}, they occur in the low-agreement subset considerably frequently. Re-annotating such cases in accordance with PDTB guidelines would likely improve the overall agreement in the dataset.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

The types of ambiguity discussed above are also commonly observed in other annotation tasks, often leading to similar patterns of disagreement among annotators \cite{jiang-marneffe-2022-investigating,sandri-etal-2023-dont}. 
%
We additionally identified types of disagreement that emerge from the \textbf{cognitively demanding} nature of IDRR. 
%
%
First, some workers appear to be more sensitive to or linguistically skilled in generating discourse inferences based on subtle \textbf{cues}, while others may overlook these cues or rely less on them in their interpretations \citep[][]{graesser1994constructing,scholman2020individual,zwaan1998situation}. In Example (5), the specification of a year in the text prompted Annotators X and Y to select an \textit{asynchronous} relation. In contrast, Annotator Z focused on the event development, labeling the relation as \textit{cause}, which is also justifiable given the context.
\ex {Arg1: \textit{
%After what happened, I was never the same. 
...I lived in fear and in a constant state of mental anguish that led me to disastrously fail my final exams.}
Arg2: By the summer of '93, I had already left the Côte d’ Azur for Paris, where ... I enrolled in a second-rate business school.}
%
%\textbf{Lexicalized relations}
% The category covers those argument pairs where a coherence relation is in fact lexicalized in the data, which deviates from the abstract IDRR setting. It comprises two distinct subgroups: 1) items containing explicit connectives; 2) items containing alternative lexicalizations, by analogy with PDTB's AltLex relation type.

% Concretely, the first subgroup that we distinguish comprises those pairs where \textit{arg2} contains an explicit connective. These examples have made it past the rule-based filters implemented in DiscoGEM 1.0, which discarded all pairs where the first 5 words of \textit{arg2} included an explicit connective from a predefined list \cite{scholman-etal-2022-discogem}. Overall, this filtering strategy appears justified, as it helped avoid discarding data excessively. On the other hand, it could not ensure the complete removal of those instances where explicit connectives are present. This complicated the task of DiscoGEM annotators, as they had to reconcile their own responses with the existing connectives.

% ARG1: \textit{All the European Union institutions were bypassed and presented with a fait accompli.} ARG2: \textit{\underline{Hence} my question : what will happen in the future if we disregard the case of Austria now?} [de\_en\_batch\_31\_item\_06]

% In the example above, the annotators went for a variety of diverse labels that they found to be compatible with a causal connective: \textsc{condition}, \textsc{conjunction}, and \textsc{asynchronous}. While some of these analyses convey plausible discourse understanding, the aim of staying consistent with PDTB's annotation paradigm suggests that examples like this can be removed. The possible advantage of this course of action is that this would mean harmonizing DiscoGEM and PDTB data.

%The second subgroup can be illustrated by the following example. 

%ARG1: \textit{Of course, there can be no objection to any structural improvement of the ECB 's financial situation.} ARG2: \textit{\underline{The only question is} whether or not pouring more money into it will solve the problem once and for all}. [de\_en\_batch\_34\_item\_05]

%In this context, "the only question is" can be perceived an a lexicalization of the \textsc{arg2-as-excpt} relation. While lexicalizations such as this one do not exclude the use of implicit connectives, they make some of them redundant (e.g., "an exception is that" which should serve as the most plausible analysis), complicating the annotators' task.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%Sentence structure (intra-connectives)
Another cognitive challenge in IDRR arises from the difficulty of interpreting \textbf{complex sentence} structures, which obscure the intended semantics of the discourse arguments. In Example (6), Arg2 begins with the connective \textit{after}, which actually signals an intra-argument temporal relation within Arg2 itself. However, this connective appears to have prompted Annotator X to label the relation between Arg1 and Arg2 as \textit{asynchronous}. The DR should instead concern the contrast between Maxime and "I", as identified by Annotator~Y.
%
\ex {Arg1: \textit{...While it had devastated me and stopped me in my tracks, for Maxime, it had broken down barriers, released him from a straitjacket, and left him free to write his own story.}
Arg2: After what happened, I was never the same...}
%
%\textbf{Intrasentential connectives in arg2.}
%Another commonly occuring pattern in the disagreed on items includes those sentences that start with a subordinate clause containing a connective such as "if", "when" or "while". 
%
%ARG1: \textit{In Kosovo, it was for humanitarian reasons that we systematically destroyed all the civilian infrastructure in Serbia.} ARG2: \textit{When the US killed 2 000 people in Panama, it was not to impose their politics, it was to combat drug trafficking.} [fr\_en\_batch
%\_22\_item\_15]
%
%Unlike the previous category, it cannot be viewed as a flaw of the filtering algorithm, as instances of this kind were not meant to be filtered out -- the connective is meant for the relation within arg2, not the relation between arg1-arg2. However, such instances are associated with increased disagreement, suggesting that these items turned out harder to process. We see several reasons as to why this is the case. First, this class puts restrictions on what connectives are compatible: e.g., an intrasentential "if" makes it impossible to insert "if" as an implicit connective. Second, it makes the propositional main clause in \textit{arg2} less accessible, as the readers have to process a possibly lengthy dependent clause before they reach it. Finally, in case of conditional clauses, the epistemic status of the proposition in \textit{arg2} also becomes unclear, making causal relations harder to infer.
%
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
In our manual analysis, we also identified a portion of labels that we could not justify and therefore categorized as \textbf{annotation errors}. In Example (7), Annotator Y labeled the relation as \textit{equivalence},
%and Annotator Z as \textit{instantiation}, neither of 
which is not reasonable given the context. It is possible that they were uncertain about the correct relation and selected a label at random. Here, Annotator X assigned the label \textit{no-relation}, which is appropriate in this case. However, since this worker tends to select \textit{no-relation} in a range of cases, such as Example~1, it can also be argued that they may be using this label as a fallback strategy when unsure of the correct relation.
%
\ex {Arg1: \textit{From what I remembered, he and Vinca had never been close...
%They ’d never spent time together, and
they had almost nothing in common.} Arg2: Vinca's mother was Pauline Lambert, an Antibes-born actress with close-cropped red hair...}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%
% 
% \textbf{Questions.} The classification by \citet{sandri-etal-2023-dont} highlights rhetorical questions as a potential source of ambiguity in subjective tasks. Specifically, the degree of rhetoricity of a question—i.e., whether an answer is pragmatically expected—can be subjective. These concerns are particularly relevant to DiscoGEM, not least because the Europarl corpus, a major data source, contains a substantial amount of argumentative discourse, including rhetorical questions.

% ARG1: \textit{My main question is: to whom is it actually addressed?} ARG2: \textit{I am afraid that there will be a sense of unease within the targeted nations about this text.} [cs\_en\_batch\_07\_item\_07]

% Our analysis revealed that for IDRR, this category needs to be formulated more broadly to include almost all instances where either argument is a question. We identified 179 such items in the corpus. The reason for this decision is that the mean number of distinct labels assigned to these instances by the annotators is notably higher than the average: $6.7 \pm 1.9$ vs $5.6 \pm 1.7$.

% A possible explanation for this wider spread of responses is that annotators experience increased cognitive load when dealing with questions, as they encounter a changed word order and different punctuation. When joining the relation parts, DiscoGEM annotators always had to disregard the original punctuation, such as full stops at the end of \textit{arg1}. However, it may be much harder to do so with question marks, as these are motivated by the word order. Similarly, it may be harder to ignore sentence boundaries when \textit{arg2} is a question and starts with a question word, as a change in word order makes it harder to unite the two sentences.

% Moreover, such question words pose a further problem, as certain canonical connectives from DiscoGEM 1.5 are syntactically incompatible with most of them, e.g., "this illustrates that" for \textsc{arg1-as-instance}. For example, "this illustrates that how" may be perceived by workers as ungrammatical. At the same time, if we were to treat rhetorical questions as implicit assertions, i.e., to rewrite \textit{How is it possible not to be outraged?}[fr\_en\_batch\_03\_item\_14] as \textit{We are outraged.}, \textsc{arg1-as-instance} would be pragmatically fitting. In practice, this could result in noisier annotations when handling questions, as annotators may opt for the first possible option to avoid cognitive load.

% Interestingly, although the crowd workers were not aware of the PDTB annotation instructions, these instructions also do not provide exhaustive recommendations on how to handle questions in practice. In that regard, even trained annotators could run into issues if they had attempted to annotate DiscoGEM's interrogative arguments consistently. According to the instructions, PDTB 2.0 specified a category in which the speaker themselves answers their own question with a "yes" or a "no": in such cases, the question does not function as an argument, but rather as supplementary material, with the answer particle itself posing as \textit{arg1} \cite{prasad2017penn}. PDTB 3.0 introduced the \textsc{hypophora} category, which covers question-answer combinations uttered by the same speaker \cite{AB2/SUU9CB_2019}. However, the occurrences of questions in DiscoGEM appear to be much more varied: for example, a question can elaborate on a preceding question.

% ARG1: \textit{Is this not a manipulation of words?} ARG2: \textit{Is not the real issue of stationing the radar in the Czech Republic also about gaining advantage? }[cs\_en\_batch\_06\_item\_07]

% Assuming that pragmatically \textit{arg2} provides details, the task of adding a connective (DiscoGEM 1.5 uses "in more detail" for \textsc{arg2-as-detail}) introduces an additional layer of complexity for the reasons we outlined earlier: it needs to be integrable into the changed sentence structure, lexically appropriate, etc.

% Sparse guidelines regarding questions are not only characteristic of PDTB, as neither RST, nor CCR instruct the workers quite exhaustively on how to proceed with them \cite{mann1988rhetorical,sanders1992toward}. On the other hand, these frameworks introduce different categories that may potentially complement those of PDTB. We believe that summarizing and, possibly, reconciling the existing approaches to questions in discourse annotation could be a fruitful direction of future work.
% Concerning DiscoGEM itself, the increased cognitive complexity seems to have impacted the annotation quality. Given that the annotation also did not involve the existing question-related instructions from PDTB (cf. \textsc{hypophora}), it may be worthwhile to re-annotate the subset of DiscoGEM involving questions (179 instances) based on more detailed instructions; alternatively, one can simply discard it when training or testing parsers on DiscoGEM data, so as not to expose the models to potentially noisy data.

Finally, we observe that some of the disagreements in DiscoGeM may have arisen from the \textbf{task design}, specifically, from the choice of connectives to represent each DR type. In example (8),  \textit{condition} is a reasonable interpretation (i.e., If the report is on the table, we are determined to adopt it). However, the connective associated with this relation, \textit{in that case}, does not fit naturally in the context, discouraging workers from choosing it. In other words, the annotation outcome depends on the workers’ flexibility in performing the connective insertion task; some may prioritize selecting a connective that sounds more natural in the given context, even if it does not perfectly represent the underlying relation. % For instance, Annotator X chose in that case, Annotator Y for that purpose, and Annotator Z as a result.
%
\ex {Arg1: \textit{...The report has nevertheless been passed by the Committee for Foreign Affairs, it is on the table and it is essential to vote on the report... 
%within the lifetime of this Parliament.
} Arg2: Our group is fully determined to debate and adopt it ...}%the report....}

Overall, much of the observed variation stems from the cognitively demanding nature of IDRR, which leads to substantial disagreement that is difficult to model using perspectivist approaches due to inconsistencies in annotator behavior. When aggregated to the coarser Level-1 relation classes, there remains a higher likelihood that noisy labels overlap with consciously assigned labels. However, at the more fine-grained Level-2 classes, the noise appears to amplify disagreement, making individual perspectives harder to model consistently and resulting in markedly lower performance.