\section{Revised Guidelines and Re-annotation}
\label{sec:upgrade:annotation}

% Plan for this Section:

% \begin{itemize}
%     \item Changes to the annotation scheme: new account of genericity, first and second person 
%     \item Reannotation procedure
% \end{itemize}

\subsection{Revised annotation scheme and guidelines} 

The changes to the annotation scheme and guidelines between {\ARRAU} 2 and {\ARRAU} 3 can be summarized as follows:
(i) alternative schemes especially for the more complex aspects of the annotation (e.g., bridging reference, genericity, discourse deixis) were carefully analyzed and the annotation scheme and guidelines for these aspects were (partially) revised at the light of the solutions proposed in this work;
(ii) a more \emph{semantic} approach was adopted for the annotation of certain aspects that had been previously annotated following purely syntactic guidelines (e.g., predication, genericity);
(iii) for the dialogue sub-corpora, more attention was paid to aspects of reference in dialogue that previously had not been  sufficiently considered (e.g., 
%the use of 
deictic first and second person pronouns, 
or
the use of referring expressions for grounding purposes).

\paragraph{Predicative NPs}

The {\ARRAU} 2 guidelines for 
predicative {\NP}s 
%marking markables as predicative 
were not very explicit and essentially relied on syntactic information, marking as predicates object {\NP}s in copular clauses (\LINGEX{Antonio Conte was [an Italian prime minister]}) and clauses with verbs such as \LINGEX{become} (\LINGEX{Antonio Conte became [the Italian prime minister]} as well as appositions (\LINGEX{Antonio Conte, [the Italian prime minister], arrived in London for talks today}). 
% Cut to save space
% A few special cases were considered.

However, the decision whether an NP is  predicative 
%is not always easy, and 
cannot always be made on syntactic grounds alone \cite{zeldes:DND22}.
For instance, in 
\LINGEX{[The Italian prime minister, [Antonio Conte]], arrived in London for meetings today},
it is the NP in appositive position (\LINGEX{Antonio Conte}) that acts as term-denoting, whereas the outside {\NP} has a predicative function. 
In so-called \NEWTERM{specificational} copular clauses, it is the subject that is predicative, whereas the object is generally taken to be referential:

\begin{exe}
\ENEW{ex:specificational} [The director of Anatomy of a Murder] is Otto Preminger
\end{exe}
Whereas in so-called \NEWTERM{identificational} copular clauses, both the subject and object are generally taken to be referring:

\begin{exe}
\ENEW{ex:identificational} [That woman] is [Sylvia]
\end{exe}
Some of these cases were covered in the previous guidelines, but not systematically.
The annotation guidelines were therefore thoroughly revised, 
%also 
to make the decision about whether a clause is predicative depend %not only on syntactic criteria but 
more on semantic criteria.
%based on tests. 

% One test that you can use to decide whether an NP in one of the
% positions just discussed is predicative is to ask yourself whether the
% clause is talking about two objects or just one. 
% This test works reasonably well, except in the case of identificational clauses, whose function is to equate two terms previously considered distinct.
% We consider  both NPs in such clauses as referring, as well as the two NPs in so-called \term{equative} copular clauses which report the discovery that two entities hereto thought to be distinct are in fact the same:

% \begin{snippet}
% We discovered that \np{the killer} was  \np{Mr. Ray}\\
% It has been claimed that \np{the location of the Battle of Brunanburh} is in fact \np{Bromborough on the Wirral}.
% \end{snippet}

% \paragraph{Measure NPs} (Not much to say!)

\paragraph{Non-identity anaphora}
%\paragraph{Bridging references}
% Already said
%In {\ARRAU} 1, the type of relation underlying a bridging reference was not annotated - only whether it was a bridging reference or not. 
%As part of the {\ARRAU} 2 annotation, this information was added, but only for the {\RST} subdomain. 
The first objective of the revision of the bridging reference annotation for {\ARRAU} 3 was to add  information
about the semantic relation 
for all subcorpora.

Equally importantly, however, we intended to produce much more explicit guidance.
%for bridging reference.
One issue was highlighted by the {\CRAC} 2018 shared task \cite{poesio-etal-2018-anaphora}.
Following her participation to the shared task, in which she found that the approach proposed by Hou et al \cite{hou-etal:EMNLP14,hou-et-al:CL18} for the \ACRO{isnotes} corpus \cite{markert-et-al:ACL2012} achieved very poor results on {\ARRAU} \cite{roesiger:CRAC18}, Ina R{\"{o}}siger  et al carried out a detailed analysis of the difference between the annotation of bridging references in the two corpora \cite{roesiger-et-al:COLING18}, concluding that very different notions of 'bridging' were used.
In {\ISNOTES}, only what they called \NEWTERM{referential} bridging references were annotated, such as \LINGEX{the door} in \SREF{ex:bridging:referential}--cases where the anaphoric expression contains an implicit anaphoric argument (\LINGEX{the door [of the house]}). 
(We think the term 'referential' is misleading, so we will call these bridging references \NEWTERM{implicitly anaphoric}, or IA.) 
In {\ARRAU}, in addition to implicitly anaphoric bridging references, a second category of referring expressions was also annotated as bridging references, that R{\"{o}}siger et al called \NEWTERM{lexical} bridging references.
One example is \LINGEX{Dubrovnik} in \SREF{ex:bridging:lexical}: the {\NP} is not implicitly anaphoric, but it establishes entity coherence with its anchor \LINGEX{Croatia} through shared %encyclopedic 
knowledge. 
(We will call this category of bridging references \NEWTERM{coherence-establishing}, or CE.)
R{\"{o}}siger et al disagreed with this broader definition of bridging reference, but also pointed out that several examples of both IA and CE bridging references were not actually annotated in {\ARRAU} 2.

\begin{exe}
    \ENEW{ex:bridging:referential} John walked towards the house. [The door] was open. 
    \ENEW{ex:bridging:lexical} Croatis's tourism industry  has been booming. 
    % Cut to save space
    % since 2012. 
    The number of yearly visitors to [Dubrovnik]  grew to over 2 million by 2019.
\end{exe}
Following that discussion, 
the annotation guidelines for bridging %reference 
were expanded to provide more explicit information about these types of bridging references.
% and one cycle of  re-annotation was  devoted 
% to improving the recall of this %type of 
% annotation.
%
% A number of other revisions to the guidelines for bridging references were made as well. 
%to the guidelines for bridging reference. 
%as well. 
%First of all, 
Explicit instructions were also added to mark split-antecedent plurals not as bridging references, but using the separate multiple antecedent mechanism offered by {\MMAX}2. 
Furthermore, %it was decided to add 
explicit instructions about %marking 
identity of sense anaphora weree added.
% Cut to save space
% (now to be marked as \CODE{undersp-rel}) 
Further instructions were also added requiring  attributes to be marked as bridging (e.g., \LINGEX{income} in 
\LINGEX{Kellogg reported its financial results for the year yesterday.  [Income] grew to ....}).

\paragraph{Genericity}

Another issue observed while running the shared tasks was that the guidelines for genericity followed in {\ARRAU} 1 and 2 has resulted in an excessively syntactic interpretation of scope in general and genericity in particular. 
Consider for instance  the contrast between \SREF{ex:genericity:inscope} and \SREF{ex:genericity:notinscope}, from the {\TRAINS} corpus.
We consider instructions as introducing an implicit modal operator, and our guidelines  therefore required to annotate {\NP}s in such utterances as \CODE{operator-instruction}. 
This is appropriate for both \LINGEX{a boxcar from Elmira} and \LINGEX{oranges} in \SREF{ex:genericity:inscope}.
However, not all such {\NP}s are in fact in the scope of the implicit modal operator--for instance, \LINGEX{the boxcar from Elmira} refers deictically to an entity in the visual scence (the {\TRAINS} world map).
As a result, we changed the guidelines to only annotate {\NP}s in utterances containing implicit or explicit operators when they were actually in the \emph{semantic} scope of the operator. 

\todo{Cite perhaps work on scope annotation and mention either here on in a final section that when we started the {\ARRAU} annotation there wasn't much in terms of scope annotation guidelines - now in retrospect we should probably have this annotated separately}

\begin{exe}
    \ENEW{ex:genericity:inscope}     take [a boxcar from Elmira]$_i$ and load [it]$_i$ with [oranges]
    \ENEW{ex:genericity:notinscope}   take [the boxcar from Elmira]$_i$ and load [it]$_i$ with [oranges]
\end{exe}
%We also introduced guidelines for dealing with ambiguity - cases in which multiple operators were present ... 

\todo{Does this make our scheme more compatible with the schemes of Herbelot / Nedoluzhko / Friedrich?}

\paragraph{Reference in dialogue}

One issue with the previous guidelines that emerged in particular from the annotation for the {\CODICRAC} dataset was that many aspects of reference in dialogue were not covered, or covered only in part. 

The first such issue was the annotation of \NEWTERM{first and second person pronouns}. 
Such pronouns were not annotated in the {\TRAINS} sub-corpora in {\ARRAU} 1 and 2, based on the belief that they were all deictic and referring to one or the other speaker, such as the instance of \LINGEX{you} in \SREF{ex:you:deictic}. 

\begin{exe}
    \ENEW{ex:you:deictic} 
    \begin{tabular}{cl}
    S: & hello how can I help [you] 
    \end{tabular}
    % Cut to savee space
    %\\
    %(TRAINS93 subcorpus, D93\_26\_3)
\end{exe}
However, this belief proved %to be completely 
incorrect;
first and second person pronouns are used 
in a number of other ways.
%much more widely than that. 
E.g., %For instance, 
in \SREF{ex:you:generic} the  two instances of \LINGEX{you} in the first utterance are most likely interpreted \emph{generically}--U is asking about what is possible in the task. 
We revised the guidelines 
% cut to save space
% to mark deictic pronouns, 
providing 
directions for distinguishing between the uses. 

\begin{exe}
    \ENEW{ex:you:generic} 
    \begin{tabular}{cl}
      U:   &  an [you] do can [you] do things \\
           &  simultaneously here or do they have to \\
           &  be done like can I have the same time \\
           &  having it the engine
    \end{tabular} 
    % Cut to save space
    %\\
    %(TRAINS93 subcorpus, D93\_26\_3)
\end{exe}
% We therefore revised the guidelines, 
% % cut to save space
% % to mark deictic pronouns, 
% providing 
% directions for distinguishing between the uses. 

Another issue %of {\NP} 
that had not been sufficiently considered in previous releases was the relation between a wh-{\NP} like \LINGEX{how long} in \SREF{ex:wh-np} and the answer to the question, \LINGEX{eight hours}. 
Clearly, this is not a case of coreference. 
However, even though wh-{\NP} are annotated as quantifiers in {\ARRAU}, it's not a case of bound anaphora either (as in \LINGEX{[No student]$_i$ forgot [their]$_i$ passport}). 
In the end, we decided to mark such cases as cases of 
associative references 
%bridging anaphora 
of type \CODE{element}, given that it may be argued that the wh-{\NP} denotes a set (the set of possible answers) of which the answer is an element; but %we expect 
this decision may be reconsidered in the future. 

\begin{exe}
    \ENEW{ex:wh-np}
    \begin{tabular}{cl}
  U :  & so  [how long] will it take if I take \\
       & the two boxcars\\
       & \ldots\\
       % cut to save space 
       % &  ship from Elmira to Corning load up \\
       % & the oranges take  the two tankers to \\
       % & Elmira and then make orange juice \\
       % & there \\
  S    & {[eight hours]}\\ 
  U    & eight hours
\end{tabular}
\end{exe}

\todo{Leave discussion of grounding and repairs for future work}

\paragraph{A new manual}
%Following the {\CRAC} 2018 shared task, 
A revised version of the annotation guidelines was produced.\footnote{\url{https://github.com/arrauproject/data/blob/main/ARRAU_3_Annotation_Manual_1.0.pdf}}
These new guidelines were 
%used not only for the  {\ARRAU} 3 re-annotation, but 
also used for the annotation of  the documents included in the  {\CODICRAC} dataset used for the 2021 and 2022 shared tasks. 
%The new guidelines are discussed in detail in Section \ref{sec:upgrade:annotation}.

\subsection{Re-Annotation}

% {\ARRAU} 3 was checked and revised  using the same tool used for {\ARRAU} 1 and 2, 
% {\MMAX}2  \CITE{mueller&strube:06}. 
% The following caused much confusion!!
% The  {\MMAX} style 
% used for {\ARRAU} 1 and 2 
% was slightly revised to incorporate the changes to the annotation scheme. 
%
% Cut to save space
%The annotation was carried out at Queen Mary University and the University of Essex by Dr. Maris Camilleri and Dr. Paloma Carretero Garcia. 
The revision proceeded in two passes.
In the first pass we checked
%The first pass was devoted to checking 
the more settled aspects of the annotation: 
the attributes encoding morphosyntactic information, referentiality (non referring / referring), 
identity anaphora, and bridging references (including e.g., checking split antecedent anaphora).
%
The second pass was devoted to %checking 
the more complex forms of annotation, including in particular  genericity, ambiguity, and discourse deixis.  
In this second pass, we also reconsidered the annotation of the dialogue corpora at the light of the experience with the {\CODICRAC} annotation.
In both passes all documents were checked and possibly corrected; and each document was completely checked by each annotator.
%and then partially checked.
% Cut to save space
%by 
%Massimo Poesio.



