\section{Experimental Results}\label{results}
%In this section, we conduct quantitative analysis of our experimental results in Sec.~\ref{Quantitative}, perform ablation test to illustrate the effectiveness of our designed modules in Sec.~\ref{Ablation}, report qualitative analysis to verify that our model can simulate human's reading for extractive summarization in Sec.~\ref{human} and conduct human evaluation in Sec.~\ref{human_evaluation}.
\begin{table}[!t]\small
\centering
%The result of our proposed HER is based on the probability termination mechanism.}
%\centering
\begin{tabular}{|l|l|l|l|}
\hline
\multirow{2}{*}{\textbf{Model}}         & \multicolumn{3}{c|}{\textbf{ROUGE}}                    \\ \cline{2-4}
             & R1            & R2            & RL            \\ \hline
% Lead-$3^1$  & 39.6          & 17.7          & 36.2          \\
Lead-$3$  & 40.0          & 17.5          & 36.2          \\
SummaRuNNer  & 39.6          & 16.2          & 35.3          \\
DQN          & 39.4          & 16.1          & 35.6          \\
Refresh      & 40.0          & 18.2          & 36.6          \\
RNES         & 41.3          & \textbf{18.9} & 37.6          \\
BANDITSUM    & 41.5          & 18.7          & 37.6          \\ \hline
\textbf{HER} & \textbf{42.3} & \textbf{18.9} & \textbf{37.9} \\ \hline
\end{tabular}
\caption{Results on the combined CNN/DailyMail test set. We report F1 scores of ROUGE-1 (R1), ROUGE-2 (R2), and ROUGE-L (RL). The result of Lead-$3$ is taken from~\citet{dong2018banditsum}.}%\vspace{-3mm}
% \caption{Results on the combined CNN/DailyMail test sets. We report F1 scores of ROUGE-1 (R1), ROUGE-2 (R2), and ROUGE-L (RL). The result of Lead-$3^1$ is taken from~\citet{narayan2018ranking} and Lead-$3^2$ is from ~\citet{dong2018banditsum}, the results are various due to different pre-prosessings and numbers of selected sentences.}
\label{exp1}
\end{table}

\subsection{Quantitative Analysis}\label{Quantitative}
We first report the ROUGE metrics on the combined CNN/DailyMail test sets in Table~\ref{exp1} and the separate results in Table~\ref{exp2}.
We can get several observations from these two tables.

Firstly, our model generally performs the best and even surpasses $42$ on ROUGE-1 score on the combined CNN/DailyMail dataset.
It also shows better results on the separate datasets.
We argue that global and local features from rough reading can help extract summaries by capturing deep contextual relations, and the designed structure in careful reading makes it more flexible in selecting sentence sets.
Hence a two-stage framework based on the human's reading cognition is more appropriate for extractive summarization.
% Notice that results on CNN dataset are no better than BANDITSUM on ROUGE-2. We assume that CNN dataset is not large enough to learn a better framework for simulating people's reading cognitive process.
%\redfont{First, our model performs best since we ... Second, for baselines... To sum up, ...}

Secondly, directly optimizing the evaluation metric by combining cross-entropy loss with rewards may improve the extractive results. RL-based methods, Refresh~\cite{narayan2018ranking} and RNES~\cite{wu2018learning}, perform better than the sequence labeling methods like SummaRuNNer~\cite{nallapati2017summarunner}.
BANDITSUM~\cite{dong2018banditsum} generally performs better than the other baselines, and it reports that framing the extractive summarization based on contextual bandit is more suitable than sequential labeling setting and also has more search space than other RL-based methods~\cite{narayan2018ranking,yao2018deep,wu2018learning}.
% \redfont{So we consider extractive summarization as a contextual bandit to construct a better framework where two stages rough reading and careful reading are devised.}
% Notice that our lead-$3$ model has the same results shown as Lead-$3^2$ in Figure~\ref{exp1},~\ref{exp2} as our pre-prosessings is similar to \citet{dong2018banditsum}.


\begin{table}[!t]\scriptsize
\centering
%\centering
\begin{tabular}{|l|l|l|l|l|l|l|}
\hline
\multirow{2}{*}{\textbf{Model}}        & \multicolumn{3}{c|}{CNN} & \multicolumn{3}{c|}{DailyMail} \\ \cline{2-7}
             & R1     & R2     & RL     & R1       & R2       & RL       \\ \hline
% Lead-$3^{1}$ & 29.1   & 11.1   & 25.9   & 40.7     & 18.3     & 37.2     \\
Lead-$3$ & 28.8   & 11.0   & 25.5   & 41.2     & 18.2     & 37.3     \\
NN-SE        & 28.4   & 10.0   & 25.0   & 36.2     & 15.2     & 32.9     \\
Refresh      & 30.4   & 11.7   & 26.9   & 41.0     & 18.8     & 37.7     \\
BANDITSUM    & \textbf{30.7}   & \textbf{11.6}   & 27.4   & 42.1     & 18.9     & 38.3     \\ \hline
\textbf{HER}    & \textbf{30.7}    & 11.5   & \textbf{27.5}   & \textbf{42.7}       & \textbf{19.0}      & \textbf{38.5}        \\ \hline
\end{tabular}
\caption{Results of the test sets on the CNN and DailyMail datasets separately.}
% \vspace{-3mm}
\label{exp2}
\end{table}




%We will discuss the effectiveness in later sections.



\subsection{Ablation Test}\label{Ablation}

Next, we conduct ablation test by removing the modules of the proposed HER step by step.
Firstly, we replace the automatic termination mechanism with a fixed extracting strategy that always selects three sentences for every document and we present the model as HER-$3$.
Based on HER-$3$, we also remove bandit policy, local net, general net gradually, and denote them as HER-$3$ w/o policy, HER-$3$ w/o policy \& local net and HER-$3$ w/o policy \& rough reading individually.
% where L and F are local net and rough reading for short individually.
The results are reported in Table~\ref{Ablation_table} and it proves the effectiveness of each proposed module.
Firstly, HER constructed with an automatic termination mechanism is more flexible and reliable in extracting various numbers of sentences varying different documents.
Secondly, HER use $\epsilon$-greedy to select sentences in order to raise the exploration chances on discovering important but easily ignored information.
Thirdly, general cognition from rough reading process is useful in extractive summarizarion.



\subsection{A Closer Look}\label{human}
%%%%%% 对应CNN
%Based on the reading cognitive process, human beings can focus on gist of paragraphs to form a rough cognition at first, and keep skimming to extract summary. 
%%%%%% 对应e-policy
%They also can explore detailed but easily ignored information through re-reading.
%%%%%%% 对应总体来说可以取得不同数量的句子
%Intuitively, they can eventually extract various numbers of sentences for different documents more flexibly.
To verify whether our proposed HER can simulate human beings' reading cognitive process, and whether such simulation are inherently helpful on extractive summarization,
we conduct extensive evaluations and try to answer the following three questions.

\begin{table}[!tb]\small
\centering
%\centering
\begin{tabular}{|l|l|l|l|}
\hline
\multirow{2}{*}{\textbf{Model}}        & \multicolumn{3}{l|}{\textbf{ROUGE}}                    \\ \cline{2-4}
             & R1            & R2            & RL            \\ \hline

\textbf{HER} & \textbf{42.3} & \textbf{18.9} & \textbf{37.9} \\
HER-$3$                & 42.0          & 18.5          & 37.6          \\
HER-$3$ w/o policy   & 41.7          & 18.3          & 37.1          \\
HER-$3$ w/o policy\&L & 41.2          & 18.4          & 37.0          \\ 
HER-$3$ w/o policy\&F & 40.6          & 18.2          & 36.9          \\  \hline
\end{tabular}
\caption{The results of ablation test on the test split of the combined CNN/DailyMail dataset. L and F are short for local net and rough reading.}
\label{Ablation_table}
\end{table}

\begin{figure}[!t]
    \centering
    \includegraphics[width=1\linewidth]{figure/scatter.eps}
    % \vspace{-4mm}
    \caption{The statistics of model HER, BANDITSUM~\cite{dong2018banditsum}, HER w/o Local Net on the selected sentences' indexes varying different document lengths. This is reported on the documents the lengths of which are all less than $80$ on the test split.}
\label{fig:scatter}
\end{figure}

%%%%%%%%%%%%%%%% CNN 的有效性%%%%%%%%%%%%
\noindent \textbf{(1) Can CNN-based network extract local features of different paragraphs?}

\noindent In Figure~\ref{fig:scatter}, we report the distribution of selected sentences' positions on our proposed model HER, BANDITSUM and HER w/o Local Net. 
We test each model at $10$k, $50$k, $100$k training steps.
It shows that all the three models can focus on different parts of the context to form summary at first and BANDITSUM performs the best after training $10$k steps.
However, with training steps growing, BANDITSUM and HER w/o Local begin to prefer earlier sentences.
HER, on the other hand, can focus on various paragraphs and extract information from different parts of the texts with constant training.
The contextual bandit~(CB) based frameworks seems to be able to attend on various parts of the contexts to some degree in the beginning.
However, with constant training, both BANDITSUM and HER w/o Local start to focus on earlier sentences due to the nature that sentences similar to the main idea usually lie on the head of the text.
As our proposed HER is equipped with a variant CNN to extract local features, our model can focus on gist of paragraphs rather than only the first several sentences.
It also encourages the exploration on extracting information from various positions more flexibly.

% Figure~(b) is the distribution after removing local network from HER, and Figure~(c) is the result only taking Eq.~\ref{Dec_1} as sentence affinity. 
% It shows that both of them mainly form summary with earlier sentences of the context with high probability.
% Figure~(d) reports BANDITSUM can extract later sentences as well because it has more freedom to explore the search space.
% Figure~(a) shows that HER is able to discover local features from gist of paragraphs and focus on different parts of the context.

% Therefore, HER can form summary more flexibly and outperform HER w/o Local~(shown in Table~\ref{Ablation_table}) and BANDITSUM~(shown in Table~\ref{exp1}).





\noindent \textbf{(2) Can the proposed bandit policy discover low-score but easily ignored information?}
%%% 需要统计一下多少的句子有效！！！！


%To verify the effectiveness of the bandit policy, we count how many sentence selected by greedy policy with a probability of $0.1$ are effective and the result is $68.5\%$.
\noindent To answer this question, we demonstrate a detailed case on sentence selection in Figure~\ref{fig:case_policy}.
We observe that although the fourth sentence has a high affinity, it should not be included in the summary since its meaning is close to the third sentence which has already been extracted.
Instead, the $13$th sentence is supposed to be selected though it has low affinity. Since our HER adopts the $\epsilon$-greedy policy, it can explore such sentence and extract it out correctly.




\begin{figure}[!t]
    \centering
    \includegraphics[width=1\linewidth]{figure/case.eps}
    \caption{A case on sentence selection of HER and HER w/o policy. The article is from CNN dataset. The highlighted indices indicate the corresponding sentences which should be extracted as summary.}
\label{fig:case_policy}
\end{figure}




\noindent \textbf{(3) Can HER extract varied but proper numbers of sentences?}

\noindent We answer this question by drawing the frequency distribution of extracted sentence numbers by our model on the test set of combined CNN/DailyMail, and Figure~\ref{fig:statistics} exhibits the results. We observe that the frequency distribution of extracted sentence number is basically similar to that of the gold-standard summary. Compared with BANDITSUM which extracts fixed number of sentences, our model shows more flexibility and extensibility on extractive summarization.



\begin{figure}[t]%[!thpb]
    \centering
    \includegraphics[width=1\linewidth, trim=0 0 0 10]{figure/frequency.eps}
    \caption{The statistics on extracted sentence number of our model. Frequency is the number of documents.}
\label{fig:statistics}

\end{figure}



\subsection{Human Evaluation}\label{human_evaluation}


%as we design a Sentence Decoder for more coverage, most salient and less redundancy, how to show it.
Lastly, we conduct a qualitative evaluation.
Following~\citet{wu2018learning}, we randomly sample $50$ documents from the test set on the combined CNN/DailyMail dataset and ask three volunteers to evaluate the summaries extracted by HER w/o $\mbox{Dec}_2$, HER w/o Local Net, BANDITSUM and HER, respectively.
HER w/o $\mbox{Dec}_2$ only uses Eq.~(\ref{Dec_1}) to compute sentence affinities without inhibiting redundant sentences.
For each document-summary pair, they are asked to rank the output of each system on three aspects, namely overall quality, coverage and non-redundancy.
Notice that the best one will be marked rank $1$ and so on,
and two system would be ranked the same if their extracted summaries are identical.
We report the average results in Table~\ref{human_eval} and it shows that our HER is leading than BANDITSUM on overall quality and coverage. 
%\redfont{We conjecture it is because the human-inspired framework of extractive summarization that improve the overall performance.\reminder{do not kidding me!}}
Additionally, HER w/o $\mbox{Dec}_2$ performs the worst on non-redundancy as it does not specialize these unselected sentences which are similar to already extracted ones. % should have low affinities.
Furthermore, HER w/o Local Net takes on bad performance on coverage because the local features can focus on paragraphical messages and help to refine thorough information.

%\subsection{Learning Performance}\label{learning}
