% This must be in the first 5 lines to tell arXiv to use pdfLaTeX, which is strongly recommended.
\pdfoutput=1
% In particular, the hyperref package requires pdfLaTeX in order to break URLs across lines.

\documentclass[11pt]{article}

% Change "review" to "final" to generate the final (sometimes called camera-ready) version.
% Change to "preprint" to generate a non-anonymous version with page numbers.
\usepackage[preprint]{acl}

% Standard package includes
\usepackage{times}
\usepackage{latexsym}
\usepackage{graphicx}
% For proper rendering and hyphenation of words containing Latin characters (including in bib files)
\usepackage[T1]{fontenc}

% For Vietnamese characters
% \usepackage[T5]{fontenc}
% See https://www.latex-project.org/help/documentation/encguide.pdf for other character sets

% This assumes your files are encoded as UTF8
\usepackage[utf8]{inputenc}

% This is not strictly necessary, and may be commented out,
% but it will improve the layout of the manuscript,
% and will typically save some space.
\usepackage{microtype}

% This is also not strictly necessary, and may be commented out.
% However, it will improve the aesthetics of text in
% the typewriter font.
\usepackage{inconsolata}
\usepackage{amsmath}
\usepackage{graphicx}
% If the title and author information does not fit in the area allocated, uncomment the following
%
%\setlength\titlebox{<dim>}
%
% and set <dim> to something 5cm or larger.

% \usepackage[marginal]{footmisc}   % 脚注，加上marginal 可以让脚注无缩进，顶格

% % 设置脚注不编号
% \renewcommand{\thefootnote}{*}

\title{DUTIR938 at SemEval-2024 Task 4: Semi-Supervised Learning and Model Ensemble for Persuasion Techniques Detection in Memes}

% Author information can be set in various styles:
% For several authors from the same institution:
% \author{Author 1 \and ... \and Author n \\
%         Address line \\ ... \\ Address line}
% if the names do not fit well on one line use
%         Author 1 \\ {\bf Author 2} \\ ... \\ {\bf Author n} \\
% For authors from different institutions:
% \author{Author 1 \\ Address line \\  ... \\ Address line
%         \And  ... \And
%         Author n \\ Address line \\ ... \\ Address line}
% To start a separate ``row'' of authors use \AND, as in
% \author{Author 1 \\ Address line \\  ... \\ Address line
%         \AND
%         Author 2 \\ Address line \\ ... \\ Address line \And
%         Author 3 \\ Address line \\ ... \\ Address line}

% \author{Erchen Yu \\
%   School of Computer Science and Technology, Dalian University of Technology / Address line 1 \\
%   Affiliation / Address line 2 \\
%   Affiliation / Address line 3 \\
%   \texttt{email@domain} \\\And
%   Second Author \\
%   Affiliation / Address line 1 \\
%   Affiliation / Address line 2 \\
%   Affiliation / Address line 3 \\
%   \texttt{email@domain} \\}


\author{
\parbox{\textwidth}{
\centering
    Erchen Yu\textsuperscript{1}, Junlong Wang\textsuperscript{2}, Xuening Qiao\textsuperscript{1}, Jiewei Qi\textsuperscript{1}, Zhaoqing Li\textsuperscript{1}, Hongfei Lin\textsuperscript{1}, Linlin Zong\textsuperscript{2}, Bo Xu\textsuperscript{1*} }\\ 
    \textsuperscript{1} School of Computer Science and Technology, Dalian University of Technology, China \\
    \textsuperscript{2} School of Software, Dalian University of Technology, China \\
    \texttt{\{yuerchen0809, jlwang, qiao, 1329027682, lizhaoqing\}@mail.dlut.edu.cn} \\
    \texttt{\{hflin, llzong, xubo\}@dlut.edu.cn}
}

\begin{document}
\maketitle
\renewcommand{\thefootnote}{*}
\footnotetext{Corresponding Author}
\renewcommand{\thefootnote}{\arabic{footnote}}
\begin{abstract}
The development of social platforms has facilitated the proliferation of disinformation, with memes becoming one of the most popular types of propaganda for disseminating disinformation on the internet. Effectively detecting the persuasion techniques hidden within memes is helpful in understanding user-generated content and further promoting the detection of disinformation on the internet. This paper demonstrates the approach proposed by Team DUTIR938 in Subtask 2b of SemEval-2024 Task 4. We propose a dual-channel model based on semi-supervised learning and model ensemble. We utilize CLIP to extract image features, and employ various pretrained language models under task-adaptive pretraining for text feature extraction. To enhance the detection and generalization capabilities of the model, we implement sample data augmentation using semi-supervised pseudo-labeling methods, introduce adversarial training strategies, and design a two-stage global model ensemble strategy. Our proposed method surpasses the provided baseline method, with Macro/Micro F1 values of 0.80910/0.83667 in the English leaderboard. Our submission ranks 3rd/19 in terms of Macro F1 and 1st/19 in terms of Micro F1.

\end{abstract}

\section{Introduction}

Social networks play a significant role in our society. The development of social platforms has facilitated the dissemination of information, but it has also fueled the proliferation of disinformation \citep{da2020semeval, dimitrov2021semeval}. The dissemination mechanism of disinformation involves the use of propaganda techniques. "Propaganda" is defined as a dissemination pattern referring to stakeholders influencing public opinion to support specific agendas and ideas by adopting persuasion techniques, such as disseminating one-sided, biased, or even fake news. Research on detecting propaganda techniques contributes to combating network disinformation \citep{da2021survey}. 

Among all types of content on social networks, memes play a significant role. Memes typically exist in the form of images, possibly with overlapping text, and convey information in the form of jokes, irony, etc. In the current era of social media, they spread rapidly and can influence many people without awareness. Memes are one of the most popular content types in online disinformation propaganda activities and serve as a powerful medium for promoting ideological and cognitive persuasion techniques \citep{moody2019analysis}. Therefore, research on automatically detecting persuasion techniques hidden in memes is of significant importance, which contributes to understanding user-generated content and further aids in detecting network disinformation. 

Subtask 2b of Semeval-2024 Task 4 aims to promote research on computational methods to detect persuasion techniques in memes \citep{semeval2024task4}, which is modeled as a binary classification problem. Due to the complexity and subjectivity inherent in persuasive language, single multimodal model may struggle to capture all relevant features effectively. To address this issue, we propose a dual-channel model based on semi-supervised learning and model ensemble in this paper. Within the image channel, we use CLIP \citep{radford2021learning} to extract image features from memes.  Concurrently, in the text channel, we employ diverse pretrained language models and conduct task-adaptive pretraining utilizing certain corpora provided by the task. We execute feature extraction from the pretrained language model, employing a variety of methodologies to capture sentence features, followed by a concatenation and fusion process of the extracted features, subsequently fed into the classification layer. We implement a semi-supervised pseudo-labeling approach, wherein pseudo-labels are assigned to the test set, thereby augmenting the training data to achieve data augmentation. Furthermore, to bolster each model's robustness, We employ Fast Gradient Method(FGM) as our adversarial training strategy, introducing perturbations to the embedding layer of the model. Lastly, We design a two-stage soft-voting ensemble strategy to amalgamate the predictions of multiple models, augmenting the model's generalization capacity and performance. 

We applied our proposed method to the English dataset. Our proposed method ranked 3rd/19 in terms of Macro F1 and 1st/19 in terms of Micro F1 in the English leaderboard for Subtask 2b.

\section{Related Work}

Transformer \citep{vaswani2017attention} is a deep learning model based on the self-attention mechanism, which is known for its ability to effectively capture long-range dependencies in sequential data. Based on Transformer, pretrained language models such as BERT \citep{kenton2019bert} have been proposed, which capture the contextual information of each token in the text through self-attention. Subsequent pretrained language models have mainly been modified on pretraining tasks, such as RoBERTa \citep{liu2019roberta}, DeBERTa \citep{he2020deberta}, and so on. These text models improve upon BERT by enhancing semantic feature representation in text feature extraction.

Existing image feature extraction methods mostly rely on multiple convolutional neural networks \citep{li2021survey}, such as VGG \citep{simonyan2015very} and ResNet \citep{he2016deep}. Vision Transformer (ViT) \citep{dosovitskiy2020image} is an image processing model based on the traditional Transformer, dividing the input image into image patches and using multi-head self-attention mechanisms to capture global relationships between images. ViT represents a significant advancement of Transformer in the field of computer vision, bringing new ideas and methods to image processing tasks.  CLIP \citep{radford2021learning} is a multimodal model which is trained through contrastive learning on 400 million pairs of images and text. CLIP achieves high-performance cross-modal semantic feature extraction for images and text.

\section{System Overview}
In this section, we will introduce the overall structure of our proposed system. Our system is divided into two stages. (1) Training of our Text-Image Multi-modal Classification Model. During the training process, we introduce several training strategies, including task-adaptive pretraining, pseudo-labeling and adversarial training. (2) Model Ensemble. We use k-fold cross-validation for model training and design a two-stage soft-voting strategy to globally integrate the models obtained in the first stage.

\begin{figure*}[htbp]
  \begin{center}
    \includegraphics[width=0.8\textwidth]{latex/11.png}
    \caption{The architecture of our Text-Image Multi-modal Classification Model.}
    \label{fig:example}
  \end{center}
\end{figure*}

\subsection{Model Architecture}
The architecture of our model is shown in Figure \ref{fig:example}. Our proposed text-image multi-modal classification model can be divided into two modules: feature extraction module and cross-modal fusion module. In the feature extraction module, we employ a parallel architecture to perform feature extraction separately for image and text channels using pretrained models.

For image inputs, we utilize the pretrained CLIP model. Before inputting images into the CLIP image encoder, they are resized and normalized, resulting in one-dimensional features of dimensionality 512. For text inputs, we experiment with various pretrained language models and their variants, including BERT, RoBERTa, DeBERTa, XLM \citep{conneau2020unsupervised}, DistilBERT \citep{sanh2019distilbert}, etc. Based on their performance on the validation and dev sets, we ultimately select two general domain models and two models pretrained using political domain related corpora as text encoder models: CLIP text encoder, Deberta-v3-large \citep{he2022debertav3}, politicalBiasBERT \citep{baly2020we}, and xlm-twitter-politics-sentiment \citep{antypas2023negativity}. We select the latter two models because the vast majority of persuasion techniques are reflected in political domain.

The output of the CLIP text encoder is a one-dimensional vector of 512 dimensions, while for three BERT-like models above, the encoder's output is a two-dimensional vector that needs to be converted into a one-dimensional vector through pooling operation. Common pooling operations include cls, pooler, last layer average and first-last layer average.The optimal pooling method is selected for each BERT-like model as the model's pooling strategy. After the pooling layer, the text features are ultimately obtained as one-dimensional features of 512 dimensions.

% \textbf{cls}: Extracting the feature corresponding to the beginning token of the sequence as the representation of the entire sequence.

% \textbf{pooler}: Compressing the semantics of the entire sequence into a single vector representation through a specially trained fully connected layer.

% \textbf{last layer average}: Averaging the vector representations of all tokens in the last layer.

% \textbf{first-last layer average}: Averaging the vector representations of all tokens in the first and last layers.
The cross-modal fusion module aims to integrate features from two modalities. We concatenate the output image features with the text features and employ a three-layer fully connected network as the classifier to map the final representation obtained from the fusion layer to a scalar, which is then bounded between 0 and 1 through a sigmoid function. The formula of cross-modal fusion module is defined as:
\[	\hat{y}_{c}=\operatorname{sigmoid}\left(W\left[h^{I}, h^{T}\right]+b\right)
\]
Where \(h^{I}\) is image feature and \(h^{T}\) is text feature. Finally, the model would be fit with the binary cross-entropy loss function.
\subsection{Training strategies}
To further improve the performance of our model, we adopt three training strategies:

\textbf{Task-adaptive Pretraining(TAPT)}: Certain research has proved that further pretraining models on the unlabeled task data itself or task related data, called TAPT, can improve the performance of model in downstream tasks \citep{gururangan2020don}. In this task, we adopt TAPT for three BERT-like text models. The purpose of this task is to detect persuasion techniques in memes. Therefore, we perform TAPT on text models using the provided dataset. Specifically, we collect and utilize all texts from task1, task2a, and task2b for pretraining our models with masked language model(MLM). By conducting MLM pretraining, these models can not only better fit the distribution in the task, but also learn rich knowledge and semantic information, thereby performing better on the task.

\textbf{Pseudo-labeling}: Pseudo-labeling is a semi-supervised method aimed at predicting labels for unlabeled data using a model trained on labeled data and adding the labeled data to the training set to achieve data augmentation. Considering the diversity of samples in the test sets, we adopt a semi-supervised pseudo-labeling method for data augmentation. We train four models on the original training set and then ensemble the four trained models using soft-voting for inference on unlabeled samples, i.e., test set samples. Subsequently, we consider samples with output probabilities greater than or equal to 0.8. Finally, we obtain 393 pseudo-labeled test set samples, which are re-incorporated into the training set for training.

\textbf{Adversarial Training}: Adversarial training is a common method to improve the robustness of neural networks. By adding perturbations in the embedding layer, we can obtain more stable embedding representations and more universal models, improving the performance of models on unseen data. In this task, we introduce FGM \citep{miyato2016adversarial} to enhance the model's robustness. The adversarial perturbation \(\delta\) on s is
defined as:
\[
	\delta=\epsilon \cdot g /\|g\|_{2} 
\quad where \quad
g=\nabla_{s} L(s, y)
\]
where \(\epsilon\) is a constant that controls the degree of perturbation suppression. The idea of FGM is to increase the perturbation direction along the gradient, where increasing along the gradient means maximizing the loss.

\subsection{Ensemble Learning}
To integrate the learning abilities of each model and improve the generalization ability of the final system, we design a two-stage soft-voting strategy to ensemble the four models saved from k-fold cross-validation. In this task, due to the small size of the dataset, we use k-fold cross-validation to train models, ensuring that all data participate in training and validation to effectively avoid over-fitting. In the first stage, we average output probability values generated by the four models in each fold of the k-fold cross-validation, resulting in the probability values for each fold. In the second stage, we aggregate the probability values from each fold by averaging, yielding the final output. We determine the optimal binary classification threshold by testing on the validation set for various thresholds. 

% \begin{figure*}[htbp]
%   \begin{center}
%     \includegraphics[width=\textwidth]{latex/2.png}
%     \caption{这是一个示例图片}
%     \label{fig:example}
%   \end{center}
% \end{figure*}

\section{Experimental setup}

\subsection{Data description and Evaluation}

The dataset is provided by Subtask 2b of Semeval-2024 Task 4. In the original dataset partition, there are 1200 samples in the training set, 150 samples in the validation set, 300 samples in the dev set, and 600 samples in the test set. After pseudo-labeling, 101 samples in the test set are labeled as non-propagandistic and 292 samples are labeled as propagandistic. We aggregate the training set, validation set, dev set, and the test set augmented with pseudo-labels into a new dataset for k-fold cross-validation. The label distribution of this dataset is shown in Table~\ref{tab:accents1}.

The official evaluation metrics for this task are Macro F1 and Micro F1, with a focus primarily on the performance of Macro F1 in our experiments. Both Macro F1 and Micro F1 performances are presented in the final test set ranking.

\begin{table}
\centering
\begin{tabular}{ccc}
\hline
\textbf{Dataset}&\textbf{Negative}&\textbf{Positive}\\
\hline
Train&400& 800\\
Validation & 50 & 100\\
Dev & 100 & 200 \\
Test(Pseudo-labeled) & 101 & 292\\
All & 651 & 1392\\
\hline
\end{tabular}
\caption{The label distribution of the dataset in task2b. Negative means non-propagandistic and positive means propagandistic.}
\label{tab:accents1}
\end{table}

\subsection{Implementation}

During validation, we conduct 8-fold cross-validation on the dataset and consistently use the average measure from the first fold as the new validation set to evaluate the performance of our model. We preserve the model parameters to achieve optimal performance. During the testing phase, we train each model separately and predict the test set through the two-stage soft-voting strategy we purposed, resulting in the final prediction by averaging the probabilities from all trained models in 8 folds. After comparing the overall performance under different thresholds, we set the final threshold as 0.5, which yields the optimal average performance on the validation set under 8-fold cross-validation.

\begin{table}
\centering
\begin{tabular}{cc}
\hline
\textbf{Setting}&\textbf{Value}\\
\hline
Epochs & 20 \\
    Max Sequence Length & 128 \\
    Batch Size & 16 \\
    Optimizer & Adam \\
    Learning Rate & 5e-5 \\
    Dropout & 0.5 \\
    Weight Decay & 0.001 \\
\hline
\end{tabular}
\caption{Hyper-parameter settings of the experiment.}
\label{tab:accents2}
\end{table}

% % table 3
\begin{table*}
\centering
\resizebox{\linewidth}{!}{ 
\begin{tabular}{ccccc}
\hline
\textbf{Setting}&\textbf{CLIP\textsubscript{img}+CLIP\textsubscript{text}}&\textbf{CLIP\textsubscript{img}+DeBERTa\textsubscript{text}}&\textbf{CLIP\textsubscript{img}+XLM\textsubscript{text}}&\textbf{CLIP\textsubscript{img}+BERT\textsubscript{text}}\\
\hline
Base & 0.8524 & 0.8375 & 0.8505 & 0.8385 \\
    +FGM & \textbf{0.8534} & 0.8555 & 0.8435 & 0.8455 \\
    +TAPT & / & \textbf{0.8574} & 0.8465 & 0.8505 \\
    +FGM+TAPT & / & 0.8515 & \textbf{0.8515} & \textbf{0.8586} \\
\hline
\end{tabular}
}
\caption{Macro F1 for four models on the new validation set. "Base" indicates no training strategy added, "+FGM" indicates the addition of FGM, "+TAPT" indicates the addition of TAPT.}
\label{tab:accents3}
\end{table*}

We implement our model using the transformer package\footnote{https://huggingface.co/}. We select the following four models as text encoder models: CLIP text encoder, Deberta-v3-large, xlm-twitter-politics-sentiment and politicalBiasBERT. And we select CLIP ViT-L/14@336px as the image encoder model. We sequentially combine each text encoder with the image encoder, and four final models are sequentially as follows: CLIP\textsubscript{img}+CLIP\textsubscript{text}, CLIP\textsubscript{img}+DeBERTa\textsubscript{text}, CLIP\textsubscript{img}+XLM\textsubscript{text} and CLIP\textsubscript{img}+BERT\textsubscript{text}. As for the pooling method, we test various options and selected the optimal one for each model based on performance on the validation set. Specifically, CLIP\textsubscript{img}+DeBERTa\textsubscript{text} and CLIP\textsubscript{img}+BERT\textsubscript{text} performed optimally with first-last layer average pooling, while CLIP\textsubscript{img}+XLM\textsubscript{text} performs optimally with pooler. We employ early stopping to retain the model parameters that exhibited the best performance on the validation set. Details of the hyper-parameter settings are provided in Table~\ref{tab:accents2}. We used a weighted binary cross-entropy loss using the class distribution. By default, We
set \(\epsilon\) to 1.0 in FGM. All experiments are conducted on an RTX 4090 with 24GB of memory.


\section{Results}

The overview statistics of four different models on the new validation set are shown in Table~\ref{tab:accents3}. Among all base models, CLIP\textsubscript{img}+CLIP\textsubscript{text} performs best, and CLIP\textsubscript{img}+XLM\textsubscript{text} likewise exhibits impressive performance. Additionally, CLIP\textsubscript{img}+BERT\textsubscript{text}+FGM+TAPT attains the highest Macro F1 score of 0.8586. The politicalBiasBERT model with FGM and TAPT strategies showcased its robust reasoning capability in detecting persuasion techniques in memes.

% , particularly with CLIP\textsubscript{img}+DeBERTa\textsubscript{text}, which experienced a 0.018 increase in Macro F1 score.
% particularly with CLIP\textsubscript{img}+DeBERTa\textsubscript{text}, which experienced a 0.02 increase in Macro F1 score.
Experimental results highlight the effectiveness of TAPT and FGM strategies. Introducing FGM to the CLIP\textsubscript{img}+CLIP\textsubscript{text}, CLIP\textsubscript{img}+DeBERTa\textsubscript{text} and CLIP\textsubscript{img}+BERT\textsubscript{text} lead to significant performance enhancements. Both the CLIP\textsubscript{img}+DeBERTa\textsubscript{text} and CLIP\textsubscript{img}+BERT\textsubscript{text} models demonstrate performance improvements after introducing TAPT, and  CLIP\textsubscript{img}+DeBERTa\textsubscript{text}with TAPT achieves the second-best overall performance. When both TAPT and FGM are employed simultaneously, all three models with BERT-like text encoder demonstrate substantial performance improvements compared with base model, while CLIP\textsubscript{img}+XLM\textsubscript{text} and CLIP\textsubscript{img}+BERT\textsubscript{text} achieve their respective optimal performance levels, showcasing the effectiveness of adding FGM and TAPT to base model.

% \begin{table}[htbp]
%   \centering
%   \caption{这是一个示例表格}
%   \label{tab:example}
%   \begin{tabular}{|c|c|c|}
%     \hline
%     Method & Macro F1 & Micro F1 \\
%     \hline
%     CLIP\textsubscript{img}+CLIP\textsubscript{text} & 0.8534 & 0.8711 \\
%     CLIP\textsubscript{img}+DeBERTa\textsubscript{text} & 0.8574 & 0.8750 \\
%     CLIP\textsubscript{img}+XLM\textsubscript{text} & 0.8515 & 0.8711 \\
%     CLIP\textsubscript{img}+BERT\textsubscript{text} & 0.8545 & 0.8789 \\
%     Our Model & \textbf{0.8654} & \textbf{0.8789} \\
%     \hline
%   \end{tabular}
% \end{table}

Regarding model ensemble, our analysis, illustrated in Table~\ref{tab:accents4}, demonstrates a significant enhancement in performance with the adoption of our two-stage global integration approach. Moreover, the scalability and robustness of our integration method are validated by the observed performance scalability relative to the number of integrated models. Results shows that all model ensemble performed best, which demonstrates the effectiveness of our model ensemble strategy.

\begin{table}
\centering
\resizebox{\linewidth}{!}{ 
\begin{tabular}{ccc}
\hline
\textbf{Method}&\textbf{Macro F1}&\textbf{Micro F1}\\
\hline
CLIP\textsubscript{img}+XLM\textsubscript{text} & 0.8515 & 0.8711 \\
    Two-Model Ensemble & 0.8515 & 0.8711 \\
    Three-Model Ensemble & 0.8596 & 0.8789 \\
    All-Model Ensemble & \textbf{0.8654} & \textbf{0.8789} \\
\hline
\end{tabular}
}
\caption{Results for model ensemble on the new validation set. Two-model ensemble refers to integrating CLIP\textsubscript{img}+XLM\textsubscript{text} and CLIP\textsubscript{img}+BERT\textsubscript{text}. Three-model ensemble adds CLIP\textsubscript{img}+DeBERTa\textsubscript{text}. For all-model ensemble, all four models are integrated for ensemble. }
\label{tab:accents4}
\end{table}

We employed all-model ensemble as the final model of our system and submitted the result file of the test set predicted by the final model. The official rankings are shown in Table~\ref{tab:accents5}. We ranked 3rd in terms of Macro F1 and 1st in terms of Micro F1. Results show that our system demonstrates outstanding performance in detecting persuasion techniques in memes, and the integration of TAPT, FGM and model ensemble techniques further enhances the detection capability of our system.

\begin{table}
\centering
\resizebox{\linewidth}{!}{ 
\begin{tabular}{cccc}
\hline
\textbf{Rank}&\textbf{Team}&\textbf{Macro F1}&\textbf{Micro F1}\\
\hline
1 & LMEME & 0.81030 & 0.82500 \\
    2 & SuteAlbastre & 0.80964 & 0.83500\\
    3 & \textbf{DUTIR938} & \textbf{0.80910} & \textbf{0.83667}\\
    4 & BCAmirs & 0.80337 & 0.82500\\
    5 & Snarci & 0.79860 & 0.82667\\
\hline
\end{tabular}
}
\caption{Results of top 5 teams for subtask2b English leaderboard on the test set.}
\label{tab:accents5}
\end{table}

\section{Conclusion}

The paper presents our system designed for Subtask 2b of Semeval-2024 Task 4. We propose a dual-channel model based on semi-supervised learning and model ensemble. Our framework leverages multiple pretrained models served as feature extractors for images and texts. We integrate a semi-supervised pseudo-labeling approach for data augmentation, and introduce TAPT and FGM adversarial training to significantly enhance the model's performance and robustness. Finally, to enhance the generalization capability of our system, we design a two-stage soft-voting model ensemble strategy. Our system achieves excellent performance in detecting persuasion techniques in memes, and we ranked 3rd in terms of Macro F1 and 1st in terms of Micro F1 in the test set for Subtask 2b. Our future research will be directed towards exploring the cross-modal fusion mechanisms within the model.

% Our future research will be directed towards exploring the fusion mechanisms within the model, and investigating the effects of class imbalance on model performance.

\section{Acknowledgments}

We thank our anonymous reviewers for their helpful comments. This work was supported by grant from the Natural Science Foundation of China(N0.62006034), the Ministry of Education Humanities and Social Science Project (No.22YJC740110), the Fundamental Research Funds for the Central Universities (No.DUT23YG136).
% Bibliography entries for the entire Anthology, followed by custom entries
%\bibliography{anthology,custom}
% Custom bibliography entries only
\bibliography{custom}


\end{document}
