\section{Results and Analysis}
%Based on whether the introduced syntactic information is explicit, we constructed two sets of comparative experiments
In this section, we first conduct experiments on the dev data to 
verify the effectiveness of our proposed approaches from several aspects: 1) how to effectively use explicit syntactic information; 
2) usefulness of explicit vs. implicit syntax and their combination; 3) which MTL framework is most effective. 
%group is mainly about several MTL approaches, which demonstrates the alleviation of the error propagation problem; and finally, 
Then we present overall results on the test dataset, with and without BERT. 
Finally, we conduct detailed analysis to gain more insights. 

\iffalse
\begin{table}[t]
\begin{center}
\begin{tabular}{ l | c  c  c }
\hline  \textbf{Explicit Info.} &  \textbf{P} &  \textbf{R} &  \textbf{F1}  \\ \hline
\textsc{Baseline} & 59.08 & 55.15 &  57.02 \\ \hline
\textsc{DepHead} & 60.82 & 55.30 & 57.91 \\ \hline
%label emb & 61.068  \\ \hline
\textsc{TPF} & \textbf{62.20} & \textbf{58.66} & \textbf{60.35} \\ \hline
%SDP & 60.726 \\ \hline
%TPF + SDP & 61.698 \\ \hline
%\textsc{TreeLSTM} & 61.92 & 56.74 & 59.16 \\ \hline
\textsc{TreeLSTM} & xx & xx & xx \\ \hline
\textsc{DepGCN-Hard} & 61.10 & 56.16 &58.50 \\ \hline
\end{tabular}
\end{center}
%\caption{Experiments result on the dev data. We compare different hard incorporation methods. }
\caption{Comparison with baseline model and  syntax-aware model with explicit/implicit syntax on the dev dataset. We report the average results of the 5-fold experiments in exact precision (P), exact recall (R) and their F1 measure (F1)}
\label{hard_result}
\end{table}
\fi

\begin{table}[t]
%\addtolength{\tabcolsep}{-1.5mm}
\small
\begin{center}
\begin{tabular}{l |c c c  } \hline 
 & \textbf{P} & \textbf{R} & \textbf{F1 } \\ \hline
\textbf{w/o Syntax} & & & \\
\ \ \  \textsc{Baseline} & 59.08 & 55.15 &  57.02 \\ \hline
\textbf{w/ Explicit Info.} & & & \\
\ \ \ \textsc{DepHead} & 60.82 & 55.30 & 57.91 \\ 
%\ \ \textsc{TPF} & \textbf{62.20} & \textbf{58.66} & \textbf{60.35} \\ 
%SDP & 60.726 \\ \hline
%TPF + SDP & 61.698 \\ \hline
%\ \textsc{TreeLSTM} & {61.92} & {56.74} & {59.16} \\ 
\ \ \ \textsc{TreeLSTM} & {60.85} & {55.25} & {57.90} \\ 
\ \ \ \textsc{DepGCN-Hard} & 61.10 & 56.16 &58.50 \\ 
\ \ \ \textsc{DepGCN} & 61.53 & 57.26 & 59.28 \\ 
\hline
\textbf{w/ Implicit Info.} & & & \\
\ \ \ \textsc{DepHDN} & 63.42 & 59.61 &  61.45 \\ \hline
\textbf{Explicit \& Implicit} & & & \\
\ \ \ \small{\textsc{DepGCN+DepHDN}} & \textbf{63.80} & \textbf{61.43} &  \textbf{62.58} \\ \hline
\end{tabular}
\end{center}

\caption{Experiments with explicit and implicit syntactic information on the dev dataset.}
\label{rst_pipeline}
\end{table}


\subsection{Using Explicit Syntax}

In order to know the best way to use explicit information from the dependency parser,  we conduct comparative experiments  by integrating the information of the explicit 1-best trees or the explicit edge-weighted graphs. The second major row of Table \ref{rst_pipeline} shows the results of integrating such explicit syntactic information on the dev data. 

In particular, \textsc{Baseline} uses no syntactic information, known as the syntax-agnostic method; 
\textsc{DepHead} concatenates an extra embedding of the head word in the 1-best parse tree with the original input; %  (the head embedding approach)
%\textbf{\textsc{TPF}}: apply the TPF approach to integrate syntax, as \citet{xia2019syntax} have demonstrated that TPF is the best performing way of encoding the syntactic information into the SRL model. 
\textsc{TreeLSTM} applies the TreeLSTM to encode the 1-best tree structures; 
\textsc{DepGCN} applies GCN to encode the edge-weighted graphs. For \textsc{DepGCN-Hard}, the 1-best tree is converted to a binary adjacency and is encoded by \textsc{DepGCN}.  %We call it DepGCN-Hard, as the adjacency matrix contains only binary values.
%\textsc{DepHDN} exploits implicit information of parser hidden states. Please refer to Section \ref{sec:proposed} for details.

%\textsc{DepGCN} exploits structural knowledge of the edge-weighted graph, refer to Section \ref{sec:proposed} for details.

It is obvious that using explicit syntactic information is helpful for ORL. All the  syntax-aware models improve the performance by 0.88$\sim$ 2.26 F1 score. The \textsc{DepHead} approach is the most intuitive way to represent syntactic information by using  head word embeddings, which serves as a simple syntax-aware baseline method.  
The \textsc{TreeLSTM} approach encodes 1-best tree recursively in a much more complex way, but achieves nearly the same performance with the  \textsc{DepHead} method. We suspect the reason may be that the \textsc{TreeLSTM} method is prone to parsing errors. 

The \textsc{DepGCN-Hard} approach also encodes the 1-best tree, and achieves higher performance.  %while \textsc{TreeLSTM} only catches up with the \textsc{DepHead}.
% Compared with \textsc{TreeLSTM} and \textsc{DepGCN-Hard}  approaches that both encode the parse tree, \textsc{DepGCN-Hard} is higher, however \textsc{TreeLSTM} only catches up with the \textsc{DepHead}.
Compared with the \textsc{TreeLSTM} approach, the \textsc{DepGCN-Hard} approach is less sensitive to parsing errors, since a GCN layer only considers local adjacent structures and performs one-hop information propagation, whereas a TreeLSTM propagates information in either bottom-up or top-down order where earlier errors affect later computations a lot. 

% The \textsc{TreeLSTM} approach that encodes the whole tree only catch up with the \textsc{DepHead} as this approach is greatly sensitive to parsing errors. 
% Compared with the \textsc{DepHead} encoding less syntax information, the other approaches can achieve higher performance. % as such approaches encode more syntactic knowledge. 

The best result of exploiting explicit information is obtained by the \textsc{DepGCN}  method, which is able to integrate richer structural information from edge-weighted graphs. 
% than the \textsc{DepGCN-Hard}  that only encodes the  single 1-best   parse trees. 

\subsection{Explicit vs. Implicit Syntax, and Combination} 
% 1. 最基本加句法的实验 hard (dev)
% baseline（无句法），head，TPF，TreeLSTM，GCN hard
%TPF可以引用庆荣的文章，说这个是那篇说最好的方式

The bottom two major rows of Table \ref{rst_pipeline} show the results on the dev data.
\textsc{DepHDN} exploits implicit information of parser hidden states. 

We can see that the implicit \textsc{DepHDN} method outperforms the best explicit \textsc{DepGCN} method by 2.17 F1 score, indicating the effectiveness of the integration of parser hidden states, which is consistent with previous studies on the SRL task \cite{xia2019mtl}. The advantage of using implicit hidden states is being able to greatly alleviate the error propagation from explicit parsing results. 


%in exact precision (P), exact recall (R) and their F1 measure (F1).
% The results are shown in Table \ref{rst_pipeline}. 


% the implicit DepHDN approach achieves higher performance than the best explicit \textsc{DepGCN} approach, which indicates that implicit information have advantages over the explicit counterparts as it  is much less sensitive to parsing errors and oppose richer syntactic information. 
We further simultaneously integrate explicit and implicit syntactic information into one model, which achieves the best performance of 62.58 F1 score, and outperforms the syntax-agnostic baseline and the \textsc{DepHDN} method by 5.56 and 1.13  F1 scores, respectively. This demonstrates that ORL can benefit from both explicit and implicit syntactic information. 

In summary, we can conclude that encoding the edge-weighted graphs is more effective than the 1-best trees, and combining both explicit and implicit syntactic information brings higher performance than either. 
% In general, the edge-weighted graph represents richer syntactic structural information than the parse tree, and combination with implicit information can xxxx.

% tree encoding configurations like \textsc{TreeLSTM} and \textsc{DepGCN-Hard} perform between \textsc{TPF} and less structural \textsc{DepHead}. 
% The possible reason is 1) TreeLSTM/DepGCN-Hard approach encodes more syntactic knowledge than DepHead approach.

% We attribute this phenomenon to the trade-off between capacity and competence. Parsing accuracy at the dependency relation level is close to 95\%, while for the complete tree structure, more than half contains at least one error. Besides, the \textsc{TreeLSTM/DepGCN-Hard} approach produce unified syntactic representation without considering the opinion expression, whereas the \textsc{TPF} approach obtain expression-specific representations.

% We attribute this phenomenon to the trade-off between capacity and competence. Parsing accuracy at the dependency relation level is close to 95\%, while for the complete tree structure, more than half contains at least one error.

%\textsc{DepHead} encodes too less structural knowledge, the \textsc{TreeLSTM} and \textsc{DepGCN-Hard} suffer from the accuracy of the whole parse tree.

% \subsection{Implicit Syntactic Information}

% 2. soft的表格 (dev)
% baseline（无句法），SAR+weighted sum，GCN soft
%解释一下hard的其他方法不适用在这里
% To compensate the parsing errors, we loosen the condition of allowing only the single best dependency tree as our syntactic information source. In this way, we make use of more knowledge from the parser, i.e., the edge-weighted graph and parser hidden states of BiLSTM encoder. We compare the following configurations:
% while keeping the robustness.

% \textsc{GCN-Soft}: Being similar to the \textsc{GCN-Hard} encoding method, we allow real values for the dependency relation between nodes. In particular, each value represents the label (?) distribution instead of whether it exists.
% \textbf{\textsc{DepGCN}}: use structural knowledge of the edge-weighted graph, refer to  the subsection of graph dependency network for details.
% \textbf{\textsc{DepHDN}}: extract hidden states from parser BiLSTM encoder as extra input of ORL model, refer to the subsection of basic syntax-aware model for details.
% syntax-aware representation of the dependency parser. In particular, we extract the hidden states of the BiLSTM encoder of our well-trained parser and use the weighted sum operation to combine different layers.



\begin{table}[t]
\small
%\addtolength{\tabcolsep}{-0.8mm}
\begin{center}
\begin{tabular}{ l | c c c }
\hline \textbf{Multi-task learning} & \textbf{P} & \textbf{R} & \textbf{F1} \\ % &\multicolumn{2}{c}{\textbf{Dep}}  \\
\hline %& \textbf{UAS} &\textbf{ LAS} \\ \hline
\ \ \ \textsc{M-Baseline}  & 62.23 & 56.84 & 59.39 \\ \hline %& xx.xx & xx.xx \\ \hline %(+2.37)
%Hard GCN & 65.872 & 61.334 & 63.50 \\ \hline % & xx.xx & xx.xx \\ \hline
\ \ \ \textsc{M-DepGCN} & 65.59 & 61.61 & 63.52 \\  % & xx.xx & xx.xx \\ \hline (\textbf{+4.24})
\ \ \ \textsc{M-DepHDN} & 65.74 &	63.67 & 64.68\\ \hline % & xx.xx & xx.xx \\ \hline (+3.23) 
\ \ \ {\textsc{M-DepGCN+DepHDN}} & \textbf{65.94} & \textbf{64.15} & \textbf{65.03} \\ \hline % & xx.xx & xx.xx \\ \hline
\end{tabular}
\end{center}
\caption{Experimental results under the MTL framework on the dev dataset.}
\label{mtl_result}
\end{table}

\iffalse
\begin{table}
\begin{center}
\begin{tabular}{ l | c  c  c}
\hline \textbf{Implicit Info.} &  \textbf{P} &  \textbf{R}  &  \textbf{F1}  \\ \hline
\textsc{Baseline} & 59.08 & 55.15 & 57.02 \\ \hline
\textsc{DepGCN} & 61.53 & 57.26 & 59.28 \\ \hline
\textsc{DepHDN} & \textbf{63.42} & \textbf{59.61} &  \textbf{61.45} \\ \hline
\end{tabular}
\end{center}
\caption{Comparison with baseline model and  syntax-aware model with implicit syntax on the dev dataset. We report the average results of the 5-fold experiments in exact precision (P), exact recall (R) and their F1 measure (F1)}
\label{soft_result}
\end{table}
\fi

% Table \ref{soft_result} presents the results of our experiments on configurations using implicit information from the parser. If we compare \textsc{DepGCN} with \textsc{DepGCN-Hard} in the Table \ref{hard_result}, we notice that ``softer'' method achieves better result as it encodes richer information from the parser, making the model less sensitive to the parsing errors. The last row of \textsc{DepHDN} represents an even more ``softer'' way of encoding syntax, which achieves higher performance than the \textsc{DepGCN}. This result is consistent with previous studies on the SRL task as well \cite{xia2019mtl}. Thus, in general, .

% %as it takes the hidden states after the BiLSTM encoders

% Notice that we have not included all the ``hard'' approaches in the previous subsection, since it is not straightforward to transform many configurations into the implicit versions. %We leave these attempts for the future work. 
\begin{table*}[t]
\addtolength{\tabcolsep}{-0.8mm}
\begin{center}
\small
\begin{tabular}{l | c c c | c c c  | c c c }
\hline 
 &  \multicolumn{3}{|c|}{\textbf{Exact F1}}  & \multicolumn{3}{c|}{\textbf{Binary F1}} & \multicolumn{3}{c}{\textbf{Proportional F1}}   \\ %\cline{2-10} 
\multirow{2}{*}{} & \textbf{Holder} & \textbf{Target} & \textbf{Overall }& \textbf{Holder} & \textbf{Target} & \textbf{Overall } & \textbf{Holder} & \textbf{Target} & \textbf{Overall } \\ \cline{3-4}  \hline
\textbf{Basic Model} & & & & & & & & \\
\ \ \textsc{Baseline} & 73.05 & 44.21 & 58.79 & 81.21 & 69.50 & 75.43 & 79.33 & 62.53 & 71.03  \\ 
\ \ \textsc{Baseline+BERT} & 76.74 & 52.61 & 64.73 & 85.45 & 75.74 & 80.62 & 83.58 & 69.31 & 76.48  \\
\ \  \citet{zhang2019enhancing} & 73.07 & 42.70 & 58.30 & 81.57 & 68.34 & 75.15 & 79.35 & 61.22 & 70.55 \\
 \hline 
\textbf{w/ SRL} & & & & & & & & \\
\ \  {\citet{marasovic2017srl4orl}} & 75.58 & 46.40 & 61.51 & 83.80 & 72.06 & 77.87 & 81.67 & 65.18 & 73.61  \\ 
\ \  \citet{zhang2019enhancing} & 76.95 & 50.50 & 63.74 & 84.91 & 73.29 & 79.10 & 82.82 & 67.31 & 75.08 \\ \hline
\textbf{w/ Syntax}  & & & & & & & &  \\
%\  \ \ \textsc{TPF} & 74.47 & 47.49 & 61.09 & 82.47 & 70.42 & 76.49 & 80.67 & 64.29 & 72.55  \\ 
\ \ \textsc{DepGCN} & 73.82  & 45.97 & 60.12 & 81.11 & 68.54 & 74.93 & 79.15 & 61.96 & 70.70 \\  
\ \ \textsc{DepHDN} & 76.96 & 46.95 & 62.29 & 83.79 & 70.20 & 77.15 & 82.44 & 63.56 & 73.21 \\
\ \ \textsc{DepGCN + DepHDN} & 76.21 & 49.38 & 63.12 & 83.0 & 72.25    & 77.81  & 81.58   &  66.59    & 74.28 \\ \hline 
\textbf{w/ Syntax + MTL}  & & & & & & & &  \\
\ \  \textsc{M-DepGCN}  & 77.50 & 50.78 & 64.28 & 84.17 & 72.91 & 78.60 & 82.77 & 66.77 & 74.85 \\ 
\ \  \textsc{M-DepHDN} & 77.36 & 50.81 & 64.31 & 84.35 & 72.45 & 78.50 & 82.95 & 66.51 & 74.87 \\ 
\ \  \textsc{M-DepGCN+DepHDN}& 78.01 & 51.92 & 65.13 & 84.97 & 73.36 & 79.24 & 83.67 & 67.77 & 75.82 \\ 
\ \  {\textsc{M-DepGCN+DepHDN+BERT}}& \textbf{79.51} & \textbf{56.61} & \textbf{68.08} & \textbf{87.09} & \textbf{76.99} & \textbf{82.04} & \textbf{85.70} & \textbf{72.32} & \textbf{79.01} \\ 
\hline 
\end{tabular}
\end{center}
%\caption{Experimental results on the test data.}
\caption{Overall experimental results on the test dataset.
}
\label{exp_res}
\end{table*}

\subsection{Effects of Multi-Task Learning}

% 3. MTL (dev)
% baseline（共享编码层），SAR+weighted sum，GCN soft, GCN hard
%解释一下hard的其他方法不适用在这里，运行效率，或者结构要大改
%解释MTL为了减少parser错误和error propagation
% Based on the previous experimental results, we observe that explicit syntactic information from the edge-weighted graph is effective, and the combination with implicit parser hidden states further boosts the performance. 
In order to alleviate the error propagation problem and explore better integration of different approaches, we apply MTL frameworks to the above-mentioned pipeline architectures. %in the  explore the potential of syntactic parser, we employ the MTL framework, which can be viewed as an even more implicit way of encoding extra information.

Table \ref{mtl_result} shows the results of the MTL settings with previously better-performing configurations on the dev dataset, together with a commonly used hard-parameter-sharing MTL for parsing and ORL. 
\textsc{M-Baseline} serves as an MTL baseline, which shares the encoder for the two tasks (Figure 2-c). \textsc{M-DepGCN} and \textsc{M-DepHDN} respectively apply the \textsc{DepGCN} and \textsc{DepHDN} approaches under our MTL framework, and \textsc{M-DepGCN+DepHDN} combines them.
%\textsc{M-DepHDN} applies \textsc{DepHDN} approach under our MTL framework.
%\textsc{M-DepGCN+DepHDN} combines the \textsc{M-DepGCN} and \textsc{M-DepHDN} approaches.

% \textsc{M-baseline} gives us an impression of a simple joint model of the two tasks.
% Firstly,  Without any sophisticated integration, sharing the encoder of the parser and ORL already brings in more than 2 exact F1 score compared with the syntax-agnostic baseline (\textsc{Baseline} in Table\ref{rst_pipeline}). 
Firstly, although sharing the encoder of the parser and ORL already brings in more than 2  F1 score improvement compared with the syntax-agnostic baseline (\textsc{Baseline}), it is much inferior to other MTL approaches and the pipeline \textsc{DepHDN} method (comparing Table \ref{rst_pipeline}). This may be caused by the weakness of the encoder parameters for ORL, as discussed in Section \ref{sec:proposed} and \citet{xia2019mtl}. 

Secondly, compared with the corresponding approaches under the pipeline architecture, all approaches under our MTL framework improve the performance by 2.45$\sim$4.24 F1 scores, which indicates that MTL is highly effective in alleviating the error propagation problem. %Besides, the \textsc{M-DepGCN} approach gains the maximum improvement in MTL than it in pipeline, which indicates that the issues MTL can alleviate is a big limit of \textsc{DepGCN} approach. %TODO: the meaning of this sentence is unclear
%parser can deliver ORL-preferred edge-weighted graph under the MTL framework. 


Finally, the combination of the explicit edge-weighted graphs and the implicit parser hidden states is still the most effective model under the MTL framework, outperforming the \textsc{Baseline} in Table \ref{rst_pipeline} by 8.01 F1 score.


% although the \textsc{M-DepHDN} also integrate the neural feature from parser encoder, our MTL achieves higher performance than \textsc{M-baseline} by 5.25\% in exact F1, which indicates our MTL is superior to the hard parameters sharing MTL in our scenario. Besides, \textsc{M-DepGCN}  and \textsc{M-DepHDN} gain more than 4\% and 3\% in exact F1 respectively after employing MTL. Moreover, we combine these two syntax-encoding methods for integrating more syntactic information, and achieve a further improvement. MTL does show advantages over the pipeline architecture which suffers from the error propagation issue.



\subsection{Final Results}
% After the ablation tests we have experimented (Table \ref{rst_pipeline} and \ref{mtl_result}), in this section, we present the overall results of our approaches together with previous state-of-the-art methods on the test data.  
%After the ablation study we have experimented (Table \ref{rst_pipeline} and \ref{mtl_result}), 
In this section, we report the overall performance of our approaches compared with previous 
%state-of-the-art 
methods on the test data, as shown in  Table \ref{exp_res}.
% We report the overall as well as the ﬁne-grained performance in term of opinion arguments (i.e., holder and target

% Besides, to analysis the effect of syntax over the BERT, we list the results of baseline with BERT and our best model with BERT.

\input{content/analysis.tex}
In particular, we list our syntax-agnostic baseline (\textsc{Baseline} in Table \ref{rst_pipeline}), others' works (\citet{zhang2019enhancing} and \citet{marasovic2017srl4orl}, using SRL for ORL), best non-MTL approaches based on our results on the dev data (\textsc{DepGCN} for explicit syntactic information and \textsc{DepHDN} for implicit syntactic information), and finally the MTL-based models. 
The results of \textsc{Baseline} with BERT and our best model with BERT are also listed to demonstrate the contributions from the contextualized word representations.

%Besides, to analysis the effect of syntax over the BERT, we list the results of baseline with BERT and our best model with BERT.
% In particular, we include two best (single) MTL configurations (\textsc{DepGCN} and \textsc{DepHDN}) and the combined one. Besides, we also list the results of \textsc{Baseline+BERT} and \textsc{DepGCN+DepHDN+BERT}.%\textsc{DepHDN+GCN-Soft}.

%leverages the robustness to the parser errors and 
%As we can see, different configurations show similar trends on the test data: 
% 1) \textsc{DepGCN} the edge-weighted graph delivers richer structural syntactic knowledge  and is less sensitive to parsing errors than the parse tree.  
We can draw the following findings. 
\begin{itemize}
\item Combining explicit and implicit syntactic information improves the performance, indicating they are complementary to each other. \item Compared with the \textsc{DepGCN} and \textsc{DepHDN} approaches (i.e., explicit or implicit only), the \textsc{DepGCN+DepHDN} approach achieves better performance on both \emph{Holder} and \emph{Target} recognition. % opinion targets of the combination achieves much improvement, which indicates that extra structural information from the edge-weighted graph is very effective for identifying opinion targets;
\item All of the MTL configurations boost the performance compared with their pipeline counterparts, as ORL-oriented parsing models are learned, and the error propagation problem is less severe. 
\item Our best syntax-aware MTL model combined with BERT achieves the best performance, outperforming the baseline with BERT by more than 3 F1 score. 
\item Compared with the previous state-of-the-art methods, we obtain 4.34 and 1.39 improvement of F1 scores  with and without BERT, respectively. Overall, our best model achieves 9.29 higher F1 score over the syntax-agnostic baseline.
\end{itemize}

%by more than 3 points of F1 score. 
% 3)  Our ﬁnal results outperforms previous state-of-the-art  model \cite{zhang2019enhancing} by  and 4.24 F1 scores with BERT representations or not, respectively.
% 3) If we compare our best model without BERT and BERT-aware model with previous state-of-the-art approaches, we achieve more than 2\% and of exact f1-score improvement, respectively. Overall, compared with the syntax-agnostic baseline, our final syntax-aware MTL model achieves 14\% increasement of exact f1-score.
% If we compare our best model with previous state-of-the-art approaches, we achieve  6.8\% and 2.2\% of exact f1-score improvement with BERT representations or not, respectively.


% Notice that the results of \citet{zhang2019enhancing} can be directly comparable, as we use the same split of the data; while for \citet{marasovic2017srl4orl}, since the data split is different, the numbers displayed here are just for reference.

\subsection{Further Analysis}
% 错误分析 0.5 pages including examples
In this section, we conduct analysis to better understand the contributions from the syntactic information and BERT. In particular, we compute the exact F1  score according to different lengths of opinion arguments, as well as different distances between the arguments and their corresponding expressions.

\paragraph{Influence of Syntax.}
Figure \ref{fig:analysis_syn_bert}-(a-b) show the effects of syntax on predicting arguments of different span lengths and distances to their expressions, respectively. We observe that 
% 1) Implicit syntactic information is more effective for identifying longer-distance arguments, whereas structural syntactic knowledge is more effective for longer-distance arguments as the dependency structure narrows their surface distance. 
% Implicit syntactic information is more effective for identifying longer-distance arguments, whereas structural syntactic knowledge is more effective for longer-distance arguments as the dependency structure narrows their surface distance.
% 1) Implicit syntactic information is more effective for identifying longer-distance arguments than explicit syntactic information
1) the performance of combining explicit and implicit syntactic information is always higher than either of them, while the \textsc{DepGCN} and \textsc{DepHDN} approaches compensate each other at different argument span lengths; and 2) MTL performs better than the best pipeline model consistently, which indicates that the usage of syntax is further enhanced as the error propagation is less severe.
% To analyze the effect of different syntactic information on ORL, we compute the exact F1 according to the length of opinion arguments and the distance of arguments to their opinion expression for firmer conclusions, as shown by subfigure a, b in Figure \ref{fig:analysis_syn_bert}. Both of them show that:
% 1) Implicit syntactic information is more effective for longer-length arguments, whereas structural syntactic knowledge is more effective for longer-distance arguments as the dependency structure narrows their surface distance. 
% 2) performance of combining explicit and implicit syntactic information  is always higher than  either of the two, demonstrating the complementarity of them.
% 3) MTL framework performs better than the best pipeline model in all sets of distance and length, which indicates that the  potential of syntax can be further explored by MTL as its capacity of alleviating the errors propagation.

\paragraph{Influence of BERT.}
Figure \ref{fig:analysis_syn_bert}-(c-d) show the similar graphs of the best syntax-aware model and BERT. Firstly, both M-Comb and BERT bring substantial improvements over the syntax-agnostic baseline. Secondly, despite that the syntactic information and BERT are similar in the overall performance, the syntactic information is more effective for arguments with longer spans and farther distances to the expressions, as the syntax helps to capture long-distance dependencies between words. And lastly, the integration of syntax and BERT can further improve the performance, demonstrating that contributions from the two are complementary.

% Furthermore, we also analyze the effect of BERT representations and its integration with syntactic information on identifying longer-length and longer-distance arguments. From subfigure c, d in Figure \ref{fig:analysis_syn_bert}, we can see that: 
% 1) BERT representations brings substantial improvements over the syntax-agnostic baseline.
% 2) Syntactic information is more effective than  BERT for longer-distance and longer-length arguments,  despite that the both are similar in the overall performance improvement.
% 3) Integration of syntax based on richer word representation of BERT can further improve the performance, demonstrating Bert is compatible and complementary with syntactic information.


% \textbf{The Distance between Arguments.} 
% % As dependency tree can capture relationships between long-distance words, we analysis the effect of distance between expression and target/holder spans on the performance of ORL models as the Figure \ref{fig:error_analysis} shows.
% % From the results, we find that integrating syntactic information into ORL model can improve the performance while the distance between expression and target/holder spans is long.
% % BERT can help the ORL model with the target/holder spans in the context of expression.
% % Combining them together can achieve best performance.
% To analyze the effect of different syntactic information and BERT representation on predicting the long-distance arguments, we compute the exact F1 according to several sets of the distance between opinion expression and their arguments, as shown by subfigure a and b in Figure \ref{fig:error_analysis}. Overall, longer-distance arguments are more difficult to identify for all models. Compared with different syntactic information in subfigure a, it is clear that: 1) for longer-distance arguments, the effectiveness of explicit edge-weighted graph and implicit parser hidden states is almost the same. Moreover performance of the combination is always higher than  either of the two, demonstrating the complementarity of explicit and implicit syntactic information. 2) MTL framework performs better than the best pipeline model in all sets of distance, which indicates that the  potential of syntax can be further explored by MTL as its capacity of alleviating the errors propagation. 
% % From the subfigure b, we can see that syntactic information is more effective than  BERT when predicting long-distance arguments,  despite that the both are similar in the overall performance improvement.
% From the subfigure b, we can see that 1) BERT achieves better performance than syntactic information for shorter-distance opinion arguments, whereas structural information of syntax is more effective for longer-distance arguments as the dependency structure narrows their surface distance.
% 2) Integration of syntax based on richer word representation of BERT can further improve the performance, demonstrating Bert is compatible and complementary with syntactic information.
% % 3) Furthermore, simultaneously exploit syntactic information and BERT can better recognize the  long-distance arguments. In total, syntactic information is more effective to longer-distance arguments than BERT as the dependency structure narrows their surface distance.

% \textbf{The Length of the Argument Span}
% On the other hand, we also analyze the effect of syntactic/BERT information on the  different sets of span lengths of opinion arguments, as shown in subfigure c and d in Figure \ref{fig-new}. 
% % Similar with the Long-distance arguments, syntactic information is more 
% The overall trend on the long-length arguments is consistent with the trend of long-distance arguments. The difference is that: 1) the trend that structural knowledge is more effective than the BERT representation is more obvious. 2) The implicit hidden states still achieve higher performance than structural edge-weighted graph for longer-length arguments.

\input{content/case.tex}
\paragraph{Case Study.} 
% syn is highly effective for ORL.
% syn is also effective based on bert.
One case study is given in Figure \ref{fig:case_study}. In this example, the gold holder ``US and UK" is difficult to be identified by the baseline model. 
% The closest holder ``UK" can be identified after integrating BERT representations, but still not fully identified. 
Even with the help of BERT, which brings more contextual information, the model still only captures one of them, the closest holder ``UK".
Our syntax-aware model accurately predicts the holder due to the coordination structure being captured by the syntactic dependency information. 
% Our syntactic model can  accurately identify the holder as the farther hold ``US" and ``UK" are appositive in syntax. 
%The observation further confirms our motivation of this work.

\section{Conclusions}
%总的来说句法超过6.53%
In this paper, we present a syntax-aware opinion role labeling approach based on  dependency GCN and MTL. We compare different representations of syntactic dependency information and propose  dependency GCN to encode richer structural information from different processing levels of the parser. The MTL framework further boosts the performance, and together with BERT, our best model achieves a new state-of-the-art result on the widely-used ORL benchmark MPQA corpus. Overall, our syntax-aware model brings in about 
%15.8\%
9.29 %(TO BE CONSISTENT! using \% or not) 
improvement of exact F1 score compared with the syntax-agnostic baseline.

%用external resources like ELMo and BERT
%We admit that external pre-trained language models like ELMo and BERT are left out in our work, as we want to focus on the contribution (solely) from syntax and MTL in the context of ORL. However, this is definitely one direction needs to be further explored.

% For the future work, we would like to adapt our approaches to other tasks or languages. Moreover, further experiments are needed to better understand the different contributions from the syntax and contextualized word representations in various other NLP tasks.

%as the general architecture is language independent and able to handle tasks with different sizes of training data. In addition

% \section{Experimental setup}
% \subsection{Datasets}
% %For ORL we use the manually annotated MPQA 2.0. corpus (Wiebe et al., 2005; Wilson, 2008). It mostly contains news documents, but also travel guides, transcripts of spoken conversations, emails, fundraising letters, textbook chapters and translations of Arabic source texts.

% For ORL, we exploit the MPQA version 2.0 corpus (Wiebe et al., 2005; Wilson, 2008) to evaluate our models, which has been widely adopted as a benchmark dataset for opinion mining (YangandCardie, 2013; Katiyar and Cardie, 2016; Marasovi´c and Frank, 2018; meishanzhan2019). There are 482 documents in the dataset. Following meishanzhang, we set aside 132 documents as the development set,and the remaining 350 documents are used as the test set in our experiments. We conduct experiments using ﬁve fold cross-validation (CV) on the test set at the document level. Following Marasovi´c and Frank (2018) and meishanzhang, we focus on opinion holders and targets only. The gold standard opinion expressions, holders and targets correspond to the direct subjective annotations, agent annotations and target annotations, respectively. For a sentence with multiple opinion expressions, we treat them as multiple sentence with one opinion expressions. Tabel xxx shows the statistics of the data.

% %In recent years, neural network based dependency parsing has achieved significant progress. We adopt the state-of-the-art biaffine parser proposed by Dozat and Manning (2017) in this work.
% For Dependency parsing, we use the original phrase-structure Penn Treebank (PTB) data to produce syntactic information  for the ORL data, which include dependency tree, probabilistic dependency tree, and syntax-aware representations of tokens.
% Following standard practice in the dependency parsing community, the phrase-structure trees are converted into Stanford dependencies using the Stanford Parser v3.3.02. Since the biaffine parser needs part-of-speech (POS) tags as inputs, we use stanford pos tagger to produce automatic POS tags on all the data. 

% \subsection{Evaluation Metrics???}
% We use recall (R), precision (P) and their F1 measure value to measure our proposed models for holders, targets and overall, separately. The average values of the ﬁve-fold CV results are reported in this work. We exploit exact matching as the major metric. Following Marasovi´c and Frank (2018) and meiszhang, two kinds of soft evaluation methods are also adopted for evaluation, namely binary and proportionalover lapping. Binary overlap treats an entity as correct if it contains an overlapped region with the gold-standard entity, and the proportional overlap assigns a partial score proportional to the ratio of the overlapped region.

% \subsection{Setting}
% There are several hyper-parameters to deﬁne our neural network structures. We simply set their values according to previous work (meishanzhang, 2019; biaffineparser, 2017), without much tuning work. Concretely, For biaffine parser, please . For ORL, we set the dimension size of all embeddings to 100, the output hidden size of LSTMs to 200 and the layer number of BiLSTM to 3. For preotrained word embeddings, we use the pretrained 100-dimensional glove embeddings (Pennington et al., 2014).  

% We train on the entire training instances for 30 epochs with the batch size of 50, choosing the best-epoch model according to the performance on the development corpus. We use Adam (Kingma and Ba, 2014) with a learning rate 10−3 to update model parameters,and use gradient clipping by a max norm 1.0 and l2-regularization by a parameter 10−8. We apply dropout with a ratio of 0.33 over word representation and lstm hidden.(\footnote{biaffine dropout}

% For MTL approach, considering the gap of size between the ORL data and parsring data and the convergence of the model of the dependency paring and the ORL, the batch size of dependency parsing is setted as 750, which the preliminary results bring better performance.  

% \section{Results and Analysis}
% big Table 


% \section*{Acknowledgments}

% The acknowledgments should go immediately before the references. Do not number the acknowledgments section.
% Do not include this section when submitting your paper for review.