\section{Experiment Setup}
\label{sec:setting}
%\subsection{Datasets}
%For ORL we use the manually annotated MPQA 2.0. corpus (Wiebe et al., 2005; Wilson, 2008). It mostly contains news documents, but also travel guides, transcripts of spoken conversations, emails, fundraising letters, textbook chapters and translations of Arabic source texts.

\textbf{Dataset.} We conduct experiments on MPQA version 2.0 corpus \cite{wiebe2005annotating}, which has been widely adopted as a benchmark dataset for opinion mining \cite{katiyar2016investigating,marasovic2017srl4orl,zhang2019enhancing}. 
%It consists of 482 documents collected from news, travel guides, emails and other domains.
In this work, we adopt the same data split (132/350 documents as dev/test data) and the same ﬁve-fold cross-validation (CV) data split on the test data as \citet{zhang2019enhancing} for a fair comparison. 
%We only retain the direct subjective annotations, agent annotations and target annotations in the corpus for our evaluation metric, corresponding to gold standard opinion expressions, holders and targets, respectively. Just like SRL, we treat a sentence with multiple expressions as multiple sentences with one expression.

% For ORL, we exploit  to evaluate our models, which has been widely adopted as a benchmark dataset for opinion mining . There are 482 documents in the dataset. Following meishanzhang, we set aside 132 documents as the development set,and the remaining 350 documents are used as the test set in our experiments. We conduct experiments using ﬁve fold cross-validation (CV) on the test set at the document level. Following Marasovi´c and Frank (2018) and meishanzhang, we focus on opinion holders and targets only. The gold standard opinion expressions, holders and targets correspond to the direct subjective annotations, agent annotations and target annotations, respectively. For a sentence with multiple opinion expressions, we treat them as multiple sentence with one opinion expressions. Tabel xxx shows the statistics of the data.

\textbf{Evaluation Metrics.} Unless specified, we use recall (R), precision (P) and their F1 measure value of exact match to evaluate the ORL performance, and the results are the average of the ﬁve-fold CV experiments. Following \citet{marasovic2017srl4orl} and \citet{zhang2019enhancing}, we also include the \textit{binary} and \textit{proportional} overlap as additional evaluation metrics. 
% Binary overlap treats an entity as correct if it contains an overlapping region with the gold entity, and the proportional overlap assigns a partial score proportional according to the ratio of the overlapping region.  
%Unless otherwise specified

% \textbf{Evaluation Metrics.} We use recall (R), precision (P) and their F1 measure value for evaluating holders, targets and overall, separately. The experimental results are the average of the ﬁve-fold CV results. We adopt the exact match to evaluate the performance of the ORL model during training. Following \cite{marasovic2017srl4orl} and \cite{zhang2019enhancing}, we also include the \textit{binary} and \textit{proportional} overlap as additional evaluation metrics. Binary overlap treats an entity as correct if it contains an overlapping region with the gold entity, and the proportional overlap assigns a partial score proportional according to the ratio of the overlapping region.  

%In recent years, neural network based dependency parsing has achieved significant progress. We adopt the state-of-the-art biaffine parser proposed by Dozat and Manning (2017) in this work.
\textbf{Dependency Parser.} Following the standard practice in the dependency parsing community, the original phrase-structure Penn Treebank data are converted into the Stanford dependencies using the Stanford Parser v3.3.0. We use the converted dependency data to train our biaffine parser for obtaining the 1-best trees, the edge-weighted graphs, and the parser hidden states. In addition, we use the Stanford POS tagger to obtain POS tags for the biaffine parser. For other settings, we follow the work of \citet{dozat2016deep}.
%Because the biaffine parser needs part-of-speech (POS) tags as inputs, we use stanford pos tagger to produce automatic POS tags on all the data.

\textbf{BERT.} We use BERT (Bidirectional Encoder Representations from Transformers) \cite{devlin2018bert} to obtain deep contextualized word representations as our extra inputs. In particular, we use BERT-base (uncased) model and extract representations from the top-1 hidden layer. Our experiments show that using the top-1 layer representations performs better than the more common use of aggregating top-4 hidden layers.\footnote{In fact, we also investigate another typical use of BERT, i.e., the fine-tuning method. However, the ORL performance is much lower than   the feature extraction method described above. Besides,  considering the training speed and flexibility in our proposed syntax-aware model, it is more flexible to adopt the feature extraction method, i.e., extracting BERT outputs as extra word representations (frozen during training).}

\textbf{Parameters.} We follow the previous works of \citet{zhang2019enhancing} and \citet{marasovic2017srl4orl} without much parameter tuning. Specifically, we use the pretrained 100-dimensional glove embeddings \cite{pennington2014glove}. The BiLSTM layer number is set to 3, and the hidden output size is 200. We apply 0.33 dropout to word representation and the hidden states of the BiLSTM. We choose Adam \cite{kingma2014adam} to optimize model parameters with a learning rate $10^{-3}$. 
The entire training instances are trained for 30 epochs with the batch size of 50, and the best-epoch model at the peak performance on the dev corpus is chosen.
For the MTL, We train the batches of ORL and parsing in turn since this interleaving training can obtain better performance in our experiments. Besides,  we use the corpus weighting trick to balance the gap in data sizes between the two tasks. 

% For the MTL approaches, there exists a gap in data sizes between the ORL model and the parser.  we combine the training datasets and randomly shuffle them to generate batches where each batch consists of both parser and ORL data. 

% Specifically, we set the dimension size of all embeddings to 100, the output hidden size of LSTM to 200 and the layer number of BiLSTM is 3. For pretrained word embeddings, we use the pretrained 100-dimensional glove embeddings \cite{pennington2014glove}. We apply dropout with a ratio of 0.33 over word representation and the hidden states of the BiLSTMs. We use Adam \cite{kingma2014adam} with a learning rate 10-3 to update model parameters,and use gradient clipping by a max norm 1.0 and l2-regularization by a parameter 10-8. For MTL approach, there exists the gap of data size between the ORL and parser,  So we combine the train datasets and randomly shuffle them for generating batches where each batch consists of parser and ORL data. Besides, we train on the entire training instances for 30 epochs with the batch size of 50, and choose the best-epoch model at the peak performance on the development corpus. 