% This must be in the first 5 lines to tell arXiv to use pdfLaTeX, which is strongly recommended.
\pdfoutput=1
% In particular, the hyperref package requires pdfLaTeX in order to break URLs across lines.

\documentclass[11pt]{article}

% Change "review" to "final" to generate the final (sometimes called camera-ready) version.
% Change to "preprint" to generate a non-anonymous version with page numbers.
\usepackage[final]{acl}

% Standard package includes
\usepackage{times}
\usepackage{latexsym}
\usepackage{float}%提供float浮动环境
\usepackage{booktabs}%提供命令\toprule、\midrule、\bottomrule
\usepackage{makecell}
\usepackage{graphicx}
\usepackage{colortbl}  %彩色表格需要加载的宏包
%\usepackage{xcolor}
%\usepackage[dvipsnames]{xcolor}
\usepackage{array}   %对表列和表格线的设置需要用到array宏包
% For proper rendering and hyphenation of words containing Latin characters (including in bib files)
\usepackage[T1]{fontenc}
% For Vietnamese characters
% \usepackage[T5]{fontenc}
% See https://www.latex-project.org/help/documentation/encguide.pdf for other character sets

% This assumes your files are encoded as UTF8
\usepackage[utf8]{inputenc}

% This is not strictly necessary, and may be commented out,
% but it will improve the layout of the manuscript,
% and will typically save some space.
\usepackage{microtype}

% This is also not strictly necessary, and may be commented out.
% However, it will improve the aesthetics of text in
% the typewriter font.
\usepackage{inconsolata}

% If the title and author information does not fit in the area allocated, uncomment the following
%
%\setlength\titlebox{<dim>}
%
% and set <dim> to something 5cm or larger.

\title{Puer at SemEval-2024 Task 4: Fine-tuning Pre-trained Language Models for Meme Persuasion Technique Detection}

% Author information can be set in various styles:
% For several authors from the same institution:
\author{Jiaxu Dao, Zhuoying Li, Youbang Su, Wensheng Gong\\
	School of Technology\\
	Pu'er University \\
	\texttt{\{daojiaxu, lizhuoying, suyoubang, gongwensheng\}@peu.edu.cn} \\}
% if the names do not fit well on one line use
%         Author 1 \\ {\bf Author 2} \\ ... \\ {\bf Author n} \\
% For authors from different institutions:
% \author{Author 1 \\ Address line \\  ... \\ Address line
%         \And  ... \And
%         Author n \\ Address line \\ ... \\ Address line}
% To start a separate ``row'' of authors use \AND, as in
% \author{Author 1 \\ Address line \\  ... \\ Address line
%         \AND
%         Author 2 \\ Address line \\ ... \\ Address line \And
%         Author 3 \\ Address line \\ ... \\ Address line}



\begin{document}
\maketitle
\begin{abstract}

The paper summarizes our research on multilingual detection of persuasion techniques in memes for the SemEval-2024 Task 4. Our work focused on English-Subtask 1, implemented based on a roberta-large pre-trained model provided by the transforms tool that was fine-tuned into a corpus of social media posts. Our method significantly outperforms the officially released baseline method, and ranked 7th in English-Subtask 1 for the test set. This paper also compares the performances of different deep learning model architectures, such as BERT, ALBERT, and XLM-RoBERTa, on multilingual detection of persuasion techniques in memes. The experimental source code covered in the paper will later be sourced from Github.
\end{abstract}

\section{Introduction}

Memes has been steadily increasing as human behavior as social media platforms have become more prevalent. This type of content is known for its rapid spread, achieved through the manipulation of audience psychology and the blurring of logical relationships. 

Memes are generally made up of stacked images and text. The essence of its expression in order to generate an emotional effect is actually the skillful role of three persuasive strategies \citep{davison2012language} in rhetorical portions: 
%These three techniques are as follow:

\begin{itemize}
  \item [1)] Ethos: This involves the strategic employment of statements from individuals endowed with authority or credibility, thereby persuading the audience of the veracity of the content and augmenting its perceived legitimacy.

  \item [2)] Pathos: By sharing personal anecdotes or experiences, memes forge a connection with the audience, evoking emotional resonance and deepening the affective engagement with the content.

  \item [3)] Logos: The application of logical arguments and reasoning enhances the structural integrity and coherence of the message, fortifying its persuasiveness.
\end{itemize}

If we further split these three categories of persuasion strategies into twenty-two, scientists are able to obtain textual and visual features from memes for analysis. For instance, it is feasible to efficiently decrease or prevent the spread of hate speech, racial discrimination, and deceptive information by analysing memes, then simultaneously preserving the peace and stability of social media. 

Memes can assist merchants in quickly capturing market trends, allowing them to carry out advertising and marketing operations more effectively and raise brand influence. Memes helps media workers in understanding the concerns of their audiences. Memes in politics have the potential to help voters demonstrate their policy views. The goal of the task is to classify corpora of text in memes and assign them to relevant persuasive strategies. Our work in SemEval-2024 Task 4 focuses on subtask 1,and this is a multi label classification task.

Our contrbutions can be highlighted as follows:
\begin{itemize}
	\item [1)] We explored new possibilities by screening models for news texts and multilingual corpus models. Fine-tuning using the social media posts corpus on the roberta-large model, and the experiment obtained hierarchical F1 of 0.647 on the English - Subtask 1 the dev set.
	
	\item [2)] In SemEval-2024 Task 4, our model has an hierarchical F1 result of 0.66 in the English - Subtask 1 the test set, and our model ranks 7th on the leaderboard.
\end{itemize}

\begin{figure*}[t]
	\centering
	% 调整图像以适应表格的高度
	\begin{minipage}[c]{0.48\textwidth}
		\centering
		% 假设表格的视觉高度较小，减少图像的高度，这里假设调整为0.5\textwidth
		\includegraphics[width=\linewidth,height=0.5\textwidth,keepaspectratio]{train_class_graph.png}
		\caption{Number of Samples}
		\label{fig:number-of-samples}
	\end{minipage}\hfill
	% 表格部分
	\begin{minipage}[c]{0.48\textwidth}
		\centering
		\scalebox{0.68}{ % 可能需要根据实际情况调整缩放比例
			\begin{tabular}{ | c | c | c |}
				\hline
				ID & text & labels \\
				\hline
				67641 & \makecell{WHEN YOU'RE THE FBI,\\ THEY LET YOU DO IT.} & Thought-terminating cliché \\
				\hline
				66402 & \makecell{PUTIN'S SECRET \\ CAMOUFLAGE ARMY} & none \\
				\hline
				71251 & \makecell{Heaven has a Wall and strict \\ immigration policies. \\ Hell has open borders. \\ President Donald J. Trump} & \makecell{Appeal to authority, \\ Exaggeration/Minimisation} \\
				\hline
				65282 & \makecell{ME VOTING ANTI-TRUMP \\IN 2016\\ ME VOTING ANTI-TRUMP\\ IN 2020} & Repetition \\
				\hline
			\end{tabular}
		}
		\captionof{table}{{Data Sample}}
		\label{table:data-sample}
	\end{minipage}
\end{figure*}



\section{Related Work}


Since the introduction of BERT in 2018\citep{devlin2018bert}, its impact on the landscape of natural language processing and multimodal analysis has been profound\citep{khare2021mmbert}. BERT and its advanced derivatives, such as RoBERTa\citep{liu2019roberta}, XLM-RoBERTa\citep{xie2021pali}, and ALBERT\citep{lan2019albert}, have demonstrated their robust capabilities in a broad spectrum of applications, ranging from sentiment analysis to complex multimodal tasks that combine textual and visual data. Notably, RoBERTa has been recognized for its superior performance in accurately classifying sentiment\citep{liao2021improved}, emotion\citep{kamath2022enhanced}, and offensive content\citep{xu2023rb_bg_mha}, highlighting the model's efficiency as a sophisticated text encoder.

The advent of these models has revolutionized the approach to analyzing diverse datasets and tasks, enabling nuanced understanding and processing of complex language patterns. This has been particularly evident in the domain of multimodal research, where BERT-based models have been instrumental in advancing the study of visual and textual data integration\citep{khan2021exploiting,he2021mf,lee2021multimodal}.

The success of these models in such a unique and culturally rich context exemplifies their broad applicability and the expanding frontiers of computational linguistics and content analysis. In conclusion, the inclusion of BERT and its variants in the analysis of persuasion techniques in memes marks a significant milestone in the field\citep{avvaru2020bert,kougia2021multimodal,khedkar2022hateful}. It underscores the models' unparalleled flexibility and their emerging role in understanding the complexities of human communication in the digital age. As these models continue to evolve, their contribution to bridging the gap between textual and visual data analysis will undoubtedly pave the way for groundbreaking research and applications across diverse disciplines.



\section{System Overview}
\subsection{Datasets}
Our experiment employed four distinct datasets: the training set, validation set, development (dev) set, and test set, all formatted in JSON. The datasets feature a minimum sentence length of one. The training set comprises 7,000 entries, categorized into 20 distinct classes, showcasing an average sentence length of 19.94 and a maximum of 253. The validation set includes 500 entries, with an average sentence length of 18.85 and a maximum reaching 333. The development set, containing 1,000 samples, presents an average sentence length of 18.73 and a peak length of 145. Lastly, the test set encompasses 1,500 instances, with the sentences averaging 18 words in length.

Table \ref{table:data-sample} presents the sample dataset, illustrating the structured data used in our analysis.


Figure \ref{fig:number-of-samples} sorts the distribution of categories in the training set in descending order of frequency, highlighting the frequency of each category. The term "None" denotes instances lacking specific classification. According to the depicted statistics, the category "Smear" constitutes the most significant portion of the dataset. In contrast, categories such as "Obfuscation", "Intentional vagueness" and "Confusion" represent the smallest proportions. 



%\includegraphics[width=1\linewidth]{train_class_graph.png}
	
%	\caption{The architecture of model construction.}
	

\begin{figure*}[t]
	\centering
	\includegraphics[width=0.8\textwidth]{net.png} % \textwidth
	\caption{The architecture of model construction}
	\label{The architecture of model construction}
\end{figure*}

\subsection{Pre-trained Model}
%As the data originates from the annotation of articles in the news,  
The research team tends to choose from models related to news, tweets, and comments. The research team tested a number of models and deciding that Jochen Hartmann's sentiment-roberta-large-english-3-classes model (As shown in Table \ref{table:pre-trained model}) while it received the best ratings. A comparison of outcomes from multiple models will be presented in the results section.


%The PIUBA team's betonews-tweetcontext model was finally chosen for the experiment. From an interdisciplinary standpoint, the team is devoted to applying artificial intelligence technologies to detect and classify hate speech on social media platforms. The team's corpus consists of various languages, and the annotation team includes both computer scientists and social scientists \citep{cotik2020study}. The team included contextual information to digital newspapers and media comments on tweets, which allows binary classification and multi label prediction. The experiment's results revealed that the Macro F1 score increased by 4.2 and 5.5 points, respectively. They released the research model, Betonews-Tweetcontext, accessible for the public \citep{perez2023assessing}. Bert serves as the foundation for this model. 

The sentiment-roberta-large-english-3-classes model \citep{hartmann2021power} is trained based on tweets on social media platforms such as Twitter and Instagram, and includes text that is expected to include captions from the sender in the tweet image and comments from other observers. RoBERTa is used to construct the model. Achieving a hold out accuracy of 86.1 \% , this model is used to evaluate user comments on posts and identify if the user is willing to buy a certain product. It demonstrates that the model has high robustness and a strong capacity to extract complicated text features.

\begin{table}[t]
	
	\scalebox{0.97}{
		\begin{tabular}{ | c | c | }
			\hline
			ID & Model \\
			%	\thead{A Head} & \thead{A Second \\ Head} & \thead{A Third \\ Head} \\
			\hline
			1& bert-base-uncased \\
			\hline
			2& bert-base-multilingual-cased \\
			\hline
			3& albert-base \\
			\hline
			4& roberta-base \\
			\hline
			5& xlm-roberta-base \\
			\hline
			6& roberta-large \\
			\hline
%			\rowcolor{lime} 
			7& roberta-large(social media posts fine-tuned) \\
			\hline
		\end{tabular}
	}
\caption{{Pre-trained Model}}%标题
\label{table:pre-trained model}
\end{table}
%经典三线表
%\begin{table}[H]
%	\caption{\textbf{Candidate Pre-trained Model}}%标题
%	\centering%把表居中
%	\begin{tabular}{cc}%四个c代表该表一共四列，内容全部居中
%		\toprule%第一道横线
%		ID & Model \\
%		\midrule%第二道横线 
%		1& bert-base-uncased \\
%		2& bert-base-multilingual-cased \\
%		3& albert-base \\
%		4& roberta-base \\
%		5& xlm-roberta-base \\
%		6& roberta-large \\
%		7& roberta-large(social media posts fine-tuned) \\
%		\bottomrule%第三道横线
%	\end{tabular}
%\end{table}

\subsection{Model Construction}
In English-Subtask 1, to commence our experiment, we utilize the officially provided Train.json and Validation.json files as the training and validation datasets for supervised learning. Additionally, we assess subsequent results using the officially available dev dataset.

Secondly, we'll perform data preprocessing. The training and validation sets are fed into the Tokenizer, and the pre-trained model roberta-large(social media posts fine-tuned) is used for word segmentation and vectorization processing.

Following that, regarding model structure:

\begin{itemize}
\item [1)] Input processing: Feed the pre-trained model with the processed token.
\item [2)] Dropout processing: Enter the dropout layer after model processing and set the inactivation probability to 0.1.
\item [3)] Linear fully connected layer: 1024 features are carried into the linear fully connected layer.
\item [4)] Loss function: For multi label classification jobs, Binary Cross Entropy With Logits Loss \citep{wang2022intelligent}serves as the loss function throughout the backpropagation gradient calculation procedure. BCEWithLogitsLoss comes with a sigmoid function that can convert predicted result values into probabilities, and can automatically handle numerical instability while preventing the sigmoid function from overflowing upwards or downwards \citep{yue2023iesrgan}.

\end{itemize}

Finally, the output layer is made up of 21 neurons, 20 of which are classified and one of which is none. The architecture of model construction is shown in Figure \ref{The architecture of model construction}.



%\begin{figure*}[htbp]
%	
%	\begin{center}
%		
%		\includegraphics[width=0.9\linewidth]{net.png}
%		
%	\end{center}
%	
%	\caption{The architecture of model construction.}
%	
%	\label{net}
%	
%\end{figure*}


\begin{figure*}[t]
	\centering
	% 图1
	\begin{minipage}{0.48\textwidth}
		\centering
		\includegraphics[width=\linewidth]{threshold_selection.png}
		\caption{Impact of threshold on dev set}
		\label{fig:threshold-selection}
	\end{minipage}\hfill % 在两个minipage之间添加一些空间
	% 图2
	\begin{minipage}{0.48\textwidth}
		\centering
		\includegraphics[width=\linewidth]{epoch_selection.png}
		\caption{Impact of epoch on dev set}
		\label{fig:epoch-selection}
	\end{minipage}
\end{figure*}
\section{Experiment Setup}

\subsection{Evaluation Metrics}


For English-Subtask 1,  the participating systems are evaluated using standard evaluation metrics, including precision, recall, and hierarchical F1 scores. These metrics are calculated as follows:
\begin{equation}
	\small
	{Precision} = \frac{{True Positives}}{{True Positives} + {False Positives}}
\end{equation}

\begin{equation}
	\small
	{Recall} = \frac{{True Positives}}{{True Positives} + {False Negatives}} 
\end{equation}

\begin{equation}
	\small
	F1 = \frac{2 \cdot {Precision} \cdot {Recall}}{{Precision} + {Recall}} 
\end{equation}


The organizers provided baseline models for each subtask. For English - Subtask 1, the Hierarchical F1 scores for the baseline model were 0.358 on the development set and 0.369 on the test set.



\subsection{Threshold Selection}

The experimental results in training tasks will depend on the threshold selection. We select the most optimal hierarchical F1 value for determining the threshold, assuming that recall and precision are of identical significance. With a 0.01 interval, the experiment increased the threshold from 0 to 1. 

The red dots on the hierarchical F1 value curve in Figure \ref{fig:threshold-selection} represent the experimental results, which show that the most suitable threshold value for hierarchical F1 value is approximately 0.08. In our threshold parameter experimentation, we attained a recall rate of 0.69 and a precision of 0.60. Owing to the threshold being established at 0.08, Figure \ref{fig:threshold-selection} incorporates merely a fraction of the experimental data. The hierarchical F1 scores start to decline as the threshold surpasses 0.4.



%\includegraphics[width=1\linewidth]{threshold_selection.png}

\subsection{Epoch Selection}


The epoch was raised in the experiment from 1 to 20 at intervals of 1. The Figure \ref{fig:epoch-selection} illustrates that the Precision is low and unstable and the Recall value is high but swings continuously when the epoch is under seven. As a result of the Precision and hierarchical F1 values' continued continuous increase, the experimental model's instability will grow. The Recall steadily stabilizes as the epoch gets closer to 20, while the hierarchical F1 value also tends to stabilize.


%\includegraphics[width=1\linewidth]{epoch_selection.png}

In addition to the above parameters, other training parameters are set in Table \ref{table:Training Arguments} below.


\begin{table}[H]

%	\scalebox{1}{
		\centering%把表居中
		\begin{tabular}{cc}%四个c代表该表一共四列，内容全部居中
			\toprule%第一道横线
			Params & Value \\
			\midrule%第二道横线 
			num\_train\_epochs& 20 \\
			per\_device\_train\_batch\_size& 4 \\
			per\_device\_eval\_batch\_size& 8 \\
			warmup\_steps& 500 \\
			weight\_decay& 0.01 \\
			logging\_steps& 100 \\
			save\_strategy& epoch \\
			evaluation\_strategy& epoch \\
			learning\_rate& $1.5 e^{-5}$ \\
			threshold & 0.08 \\
			\bottomrule%第三道横线
		\end{tabular}
%	}
\caption{{Training Arguments}}%标题
\label{table:Training Arguments}
\end{table}

\section{Results}

As Table \ref{table:Dev Set Results} shown, the model's performance on the development set revealed an hierarchical F1 score of 0.64, a precision of 0.63, and a recall of 0.65. The results indicate that our model achieves better results than other models. The performance of English-Subtask 1 on the test set yielded an hierarchical F1 score of 0.66, a precision of 0.65, and a recall of 0.67, ultimately securing the 7th position in the ranking.

\begin{table}[t]
	
	\centering % 添加居中对齐，以确保表格在页面中间
	\scalebox{0.9}{ % 将缩放比例调整为0.9
		\begin{tabular}{ | c | c | c | c |}
			\hline
			Model &  F1 & Precision & Recall \\
			\hline
			bert-base-uncased & 0.59335 & 0.60017 & 0.58668 \\
			\hline
			\makecell{bert-base-\\multilingual-cased} & 0.58840 & 0.58235 & 0.59459 \\
			\hline
			albert-base & 0.59484 & 0.58081 & 0.60957 \\
			\hline
			roberta-base & 0.62268 & 0.60781 & 0.63829 \\
			\hline
			xlm-roberta-base & 0.58612 & 0.57927 & 0.59313 \\
			\hline
			roberta-large & 0.63679 & 0.61831 & 0.65640 \\
			\hline
			\makecell{roberta-large \\(social media \\ posts fine-tuned)} & 0.64708 & 0.63666 & 0.65786 \\
			\hline
		\end{tabular}
	}
\caption{{Dev Set Results}}%标题
\label{table:Dev Set Results}
\end{table}

\section{Conclusion}
In our participation in SemEval-2024 Task 4, specifically English-Subtask 1, we focused on addressing the challenge of multi-label text classification. Our study investigated the impact of various pre-trained models on experimental outcomes and the influence of different hyperparameters on model performance. Ultimately, the adoption of the roberta-large model fine-tuned on social media posts led to outstanding performance, achieving a hierarchical F1 score of 0.66 on the test set and securing a commendable 7th position among English-Subtask 1 participants.

In our experimentation, we did not pursue a finer-grained classification within the multi-label task. Moving forward, our future research direction will pivot towards fine-grained multi-label classification. This would entail optimizing the loss function or implementing multi-level classification techniques to enhance the model's generalization capabilities.

\section*{Acknowledgements}

This work was supported by the 2024 Science and Technology special project of Pu 'er University(PYKJZX202401 Research on medical relationship extraction task based on pre-trained language model). The authors would like to thank the anonymous reviewers for their constructive comments.

\bibliography{custom}

%The first line of the file must be
%\begin{quote}
%\begin{verbatim}
%\documentclass[11pt]{article}
%\end{verbatim}
%\end{quote}
%
%To load the style file in the review version:
%\begin{quote}
%\begin{verbatim}
%\usepackage[review]{acl}
%\end{verbatim}
%\end{quote}
%For the final version, omit the \verb|review| option:
%\begin{quote}
%\begin{verbatim}
%\usepackage{acl}
%\end{verbatim}
%\end{quote}
%
%To use Times Roman, put the following in the preamble:
%\begin{quote}
%\begin{verbatim}
%\usepackage{times}
%\end{verbatim}
%\end{quote}
%(Alternatives like txfonts or newtx are also acceptable.)
%
%Please see the \LaTeX{} source of this document for comments on other packages that may be useful.
%
%Set the title and author using \verb|\title| and \verb|\author|. Within the author list, format multiple authors using \verb|\and| and \verb|\And| and \verb|\AND|; please see the \LaTeX{} source for examples.
%
%By default, the box containing the title and author names is set to the minimum of 5 cm. If you need more space, include the following in the preamble:
%\begin{quote}
%\begin{verbatim}
%\setlength\titlebox{<dim>}
%\end{verbatim}
%\end{quote}
%where \verb|<dim>| is replaced with a length. Do not set this length smaller than 5 cm.
%
%\section{Document Body}
%
%\subsection{Footnotes}
%
%Footnotes are inserted with the \verb|\footnote| command.\footnote{This is a footnote.}
%
%\subsection{Tables and figures}
%
%See Table~\ref{tab:accents} for an example of a table and its caption.
%\textbf{Do not override the default caption sizes.}
%
%\begin{table}
%\centering
%\begin{tabular}{lc}
%\hline
%\textbf{Command} & \textbf{Output}\\
%\hline
%\verb|{\"a}| & {\"a} \\
%\verb|{\^e}| & {\^e} \\
%\verb|{\`i}| & {\`i} \\ 
%\verb|{\.I}| & {\.I} \\ 
%\verb|{\o}| & {\o} \\
%\verb|{\'u}| & {\'u}  \\ 
%\verb|{\aa}| & {\aa}  \\\hline
%\end{tabular}
%\begin{tabular}{lc}
%\hline
%\textbf{Command} & \textbf{Output}\\
%\hline
%\verb|{\c c}| & {\c c} \\ 
%\verb|{\u g}| & {\u g} \\ 
%\verb|{\l}| & {\l} \\ 
%\verb|{\~n}| & {\~n} \\ 
%\verb|{\H o}| & {\H o} \\ 
%\verb|{\v r}| & {\v r} \\ 
%\verb|{\ss}| & {\ss} \\
%\hline
%\end{tabular}
%\caption{Example commands for accented characters, to be used in, \emph{e.g.}, Bib\TeX{} entries.}
%\label{tab:accents}
%\end{table}
%
%\subsection{Hyperlinks}
%
%Users of older versions of \LaTeX{} may encounter the following error during compilation: 
%\begin{quote}
%\tt\verb|\pdfendlink| ended up in different nesting level than \verb|\pdfstartlink|.
%\end{quote}
%This happens when pdf\LaTeX{} is used and a citation splits across a page boundary. The best way to fix this is to upgrade \LaTeX{} to 2018-12-01 or later.
%
%\subsection{Citations}
%
%\begin{table*}
%\centering
%\begin{tabular}{lll}
%\hline
%\textbf{Output} & \textbf{natbib command} & \textbf{ACL only command}\\
%\hline
%\citep{Gusfield:97} & \verb|\citep| &  \\
%\citealp{Gusfield:97} & \verb|\citealp| & \\
%\citet{Gusfield:97} & \verb|\citet| &  \\
%  \citeyearpar{Gusfield:97} & \verb|\citeyearpar| &  \\
%  \citeposs{Gusfield:97}	&	& \verb|\citeposs|\\
%\hline
%\end{tabular}
%\caption{\label{citation-guide}
%Citation commands supported by the style file.
%The style is based on the natbib package and supports all natbib citation commands.
%It also supports commands defined in previous ACL style files for compatibility.
%}
%\end{table*}
%
%Table~\ref{citation-guide} shows the syntax supported by the style files.
%We encourage you to use the natbib styles.
%You can use the command \verb|\citet| (cite in text) to get ``author (year)'' citations, like this citation to a paper by \citet{Gusfield:97}.
%You can use the command \verb|\citep| (cite in parentheses) to get ``(author, year)'' citations \citep{Gusfield:97}.
%You can use the command \verb|\citealp| (alternative cite without parentheses) to get ``author, year'' citations, which is useful for using citations within parentheses (e.g. \citealp{Gusfield:97}).
%
%A possessive citation can be made with the command \verb|\citeposs|.
%This is not a standard natbib command, so it is generally not compatible
%with other style files.
%
%\subsection{References}
%
%\nocite{Ando2005,andrew2007scalable,rasooli-tetrault-2015}
%
%The \LaTeX{} and Bib\TeX{} style files provided roughly follow the American Psychological Association format.
%If your own bib file is named \texttt{custom.bib}, then placing the following before any appendices in your \LaTeX{} file will generate the references section for you:
%\begin{quote}
%\begin{verbatim}
%\bibliography{custom}
%\end{verbatim}
%\end{quote}
%
%You can obtain the complete ACL Anthology as a Bib\TeX{} file from \url{https://aclweb.org/anthology/anthology.bib.gz}.
%To include both the Anthology and your own .bib file, use the following instead of the above.
%\begin{quote}
%\begin{verbatim}
%\bibliography{anthology,custom}
%\end{verbatim}
%\end{quote}
%
%Please see Section~\ref{sec:bibtex} for information on preparing Bib\TeX{} files.
%
%\subsection{Appendices}
%
%Use \verb|\appendix| before any appendix section to switch the section numbering over to letters. See Appendix~\ref{sec:appendix} for an example.
%
%\section{Bib\TeX{} Files}
%\label{sec:bibtex}
%
%Unicode cannot be used in Bib\TeX{} entries, and some ways of typing special characters can disrupt Bib\TeX's alphabetization. The recommended way of typing special characters is shown in Table~\ref{tab:accents}.
%
%Please ensure that Bib\TeX{} records contain DOIs or URLs when possible, and for all the ACL materials that you reference.
%Use the \verb|doi| field for DOIs and the \verb|url| field for URLs.
%If a Bib\TeX{} entry has a URL or DOI field, the paper title in the references section will appear as a hyperlink to the paper, using the hyperref \LaTeX{} package.
%
%\section*{Acknowledgements}
%
%This document has been adapted
%by Steven Bethard, Ryan Cotterell and Rui Yan
%from the instructions for earlier ACL and NAACL proceedings, including those for 
%ACL 2019 by Douwe Kiela and Ivan Vuli\'{c},
%NAACL 2019 by Stephanie Lukin and Alla Roskovskaya, 
%ACL 2018 by Shay Cohen, Kevin Gimpel, and Wei Lu, 
%NAACL 2018 by Margaret Mitchell and Stephanie Lukin,
%Bib\TeX{} suggestions for (NA)ACL 2017/2018 from Jason Eisner,
%ACL 2017 by Dan Gildea and Min-Yen Kan, 
%NAACL 2017 by Margaret Mitchell, 
%ACL 2012 by Maggie Li and Michael White, 
%ACL 2010 by Jing-Shin Chang and Philipp Koehn, 
%ACL 2008 by Johanna D. Moore, Simone Teufel, James Allan, and Sadaoki Furui, 
%ACL 2005 by Hwee Tou Ng and Kemal Oflazer, 
%ACL 2002 by Eugene Charniak and Dekang Lin, 
%and earlier ACL and EACL formats written by several people, including
%John Chen, Henry S. Thompson and Donald Walker.
%Additional elements were taken from the formatting instructions of the \emph{International Joint Conference on Artificial Intelligence} and the \emph{Conference on Computer Vision and Pattern Recognition}.
%
%% Bibliography entries for the entire Anthology, followed by custom entries
%%\bibliography{anthology,custom}
%% Custom bibliography entries only
%\bibliography{custom}
%
%\appendix
%
%\section{Example Appendix}
%\label{sec:appendix}
%
%This is an appendix.
%
%This document is a supplement to the general instructions for *ACL authors. It contains instructions for using the \LaTeX{} style files for ACL conferences. 
%The document itself conforms to its own specifications, and is therefore an example of what your manuscript should look like.
%These instructions should be used both for papers submitted for review and for final versions of accepted papers.
%
%These instructions are for authors submitting papers to *ACL conferences using \LaTeX. They are not self-contained. All authors must follow the general instructions for *ACL proceedings,\footnote{\url{http://acl-org.github.io/ACLPUB/formatting.html}} and this document contains additional instructions for the \LaTeX{} style files.
%
%The templates include the \LaTeX{} source of this document (\texttt{acl.tex}),
%the \LaTeX{} style file used to format it (\texttt{acl.sty}),
%an ACL bibliography style (\texttt{acl\_natbib.bst}),
%an example bibliography (\texttt{custom.bib}),
%and the bibliography for the ACL Anthology (\texttt{anthology.bib}).
%
%To produce a PDF file, pdf\LaTeX{} is strongly recommended (over original \LaTeX{} plus dvips+ps2pdf or dvipdf). Xe\LaTeX{} also produces PDF files, and is especially suitable for text in non-Latin scripts.
\end{document}
