\documentclass[11pt,a4paper]{article}
\usepackage[hyperref]{acl2020}
\usepackage{times}
\usepackage{latexsym}
\usepackage{url}
\usepackage{amsmath}
\usepackage[linesnumbered,lined,ruled,commentsnumbered]{algorithm2e}
\usepackage{adjustbox}
\usepackage{natbib}
\usepackage{CJKutf8}
\usepackage{multirow}

\usepackage{xcolor}
\newcommand\SJ[1]{\textcolor{red}{\{SJ: #1 \}}}
\newcommand\ZW[1]{\textcolor{orange}{\{ZW: #1 \}}}
% \newcommand \newcite[1]{{\citeauthor{#1}~(\citeyear{#1}})}

% \pdfinfo{
% /Title (A Reinforced Generation of adversarial examples for Neural Machine Translation)
% /Author ()
% /Keywords (adversarial; robustness; machine translation)
% }

\aclfinalcopy % Uncomment this line for the final submission
%\def\aclpaperid{***} %  Enter the acl Paper ID here
\setlength\titlebox{5cm}
% You can expand the titlebox if you need extra space
% to show all the authors. Please do not make the titlebox
% smaller than 5cm (the original size); we will check this
% in the camera-ready version and ask you to change it back.

\newcommand\BibTeX{B\textsc{ib}\TeX}

\title{A Reinforced Generation of Adversarial Examples for Neural Machine Translation}

\author{Wei Zou$^{1}$\quad Shujian Huang$^{1}$\quad Jun Xie$^{2}$\quad Xinyu Dai$^{1}$\quad Jiajun Chen$^{1}$ \\
$^{1}$National Key Laboratory for Novel Software Technology, Nanjing University, China\\
$^{2}$Tencent Technology Co, China \\
\texttt{zouw@smail.nju.edu.cn},\ \texttt{\{huangsj,daixinyu,chenjj\}@nju.edu.cn}\ \\
\texttt{stiffxie@tencent.com} \\
}

\date{}


\begin{document}
% \linenumbers   % show line numbers
\maketitle

\begin{abstract}
Neural machine translation systems tend to fail on less decent inputs despite its significant efficacy, which may significantly harm the credibility of these systems—fathoming how and when neural-based systems fail in such cases is critical for industrial maintenance. Instead of collecting and analyzing bad cases using limited handcrafted error features, here we investigate this issue by generating adversarial examples via a new paradigm based on reinforcement learning. Our paradigm could expose pitfalls for a given performance metric, e.g., BLEU, and could target any given neural machine translation architecture. We conduct experiments of adversarial attacks on two mainstream neural machine translation architectures, RNN-search, and Transformer. The results show that our method efficiently produces stable attacks with meaning-preserving adversarial examples. We also present a qualitative and quantitative analysis for the preference pattern of the attack, demonstrating its capability of pitfall exposure.
\end{abstract}

\input{introduction.tex}

\input{background.tex}

\input{approach.tex}

\input{experiments.tex}

\input{analysis.tex}

\input{related_work.tex}

\section{Conclusion}
We propose a new paradigm to generate adversarial examples for neural machine translation, which is capable of exposing translation pitfalls without handcrafted error features. 
Experiments show that our method achieves stable degradation with meaning preserving adversarial examples over different victim models. 

It is noticeable that our method can generate adversarial examples efficiently from monolingual data. 
As a result, the mass production of adversarial examples for the victim model's analysis and further improvement of robustness become convenient. 
Furthermore, we notice some exceptional cases which we call as ``reinforced samples'', which we leave as the future work.

\section*{Acknowledgement}
We would like to thank the anonymous reviewers for their insightful comments. Shujian Huang is the corresponding author. This work is supported by National Science Foundation of China (No. 61672277, 61772261),  the Jiangsu Provincial Research Foundation for Basic Research (No. BK20170074).

\bibliographystyle{acl_natbib}
\bibliography{reference}

\input{appendix.tex}

\end{document}


