% Details here
% https://semeval.github.io/paper-requirements.html
\documentclass[11pt]{article}
\usepackage[utf8]{inputenc}
\usepackage[final]{acl}
\usepackage{multirow, makecell}
\usepackage{amsmath}
\usepackage{algorithm}
\usepackage{algpseudocode}
\usepackage{amssymb}
\usepackage{enumitem}

% Standard package includes
\usepackage{times}
\usepackage{latexsym}
\usepackage{graphicx}
\newcommand\todo[1]{\textcolor{red}{#1}}
\usepackage{xcolor,colortbl}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{tcolorbox}
\tcbuselibrary{breakable}
\usepackage{mwe}
\usepackage{tabu}

\usepackage{microtype}

\input{cmds}

\title{Using BERT in Multilingual Real Valence-Arousal Scores Predictions for Aspects Mentioned in Texts}
\author{
  Harshal Dharpure   \\
  Indian Institute of Technology, Patna \\
  Patna, India \\
  \texttt{harshal\_2511ai30@iitp.ac.in} \\
  \\\And
  Nicolay Rusnachenko \\
  Newcastle Upon Tyne, England \\
  {\texttt{rusnicolay@gmail.com}}
}


\date{January 2024}

\begin{document}

\maketitle

%Elly: Question: do you pretrain the Flan-T5 model? Figure 2 is for pre-training stage. If yes, what kind of dataset you use for pretraining? If not, Figure 2 is confusing.
% YES, we tune Flan-T5. Clarifing this in the abstract and updating figure by adding the related datasets and resources on diagram.
%Thus, counting on these factors in an attempt to determine the character state is important \textcolor{red}{[don't understand this sentence]}.
% Nicolay: This sentence emphasizes that we need this information to COUNT IT in the methodology. Since abstract is quite long already, it was decided to hide it.
%\textcolor{red}{[introduce it after "in this work"?]}
%\textcolor{red}{[approach] [mention LLMs somewhere?]}. 
\begin{abstract}
Dimensional aspect-based sentiment analysis extends traditional sentiment classification by predicting continuous Valence-Arousal (VA) scores, offering fine-grained emotional understanding of user opinions. However, existing approaches struggle with arousal prediction, which exhibits significantly lower correlation than valence across multiple languages. We address this challenge through a novel arousal-focused architecture that employs separate prediction pathways for valence and arousal, enhanced by multi-head self-attention and convolutional intensity features. Our approach combines multilingual pretraining on diverse languages with domain-specific fine-tuning, and introduces a weighted loss function that emphasizes arousal prediction. Experiments on SemEval-2026 Task 3 demonstrate substantial improvements: our approach achieves 36.2\% average RMSE reduction across English, Russian, and Ukrainian, with Russian showing a remarkable 47.9\% improvement. Arousal prediction shows dramatic improvement, with RMSE reductions of 24--48\% while maintaining strong valence performance. Our findings reveal that architectural specialization and loss weighting are crucial for balanced VA prediction, particularly in multilingual settings where arousal patterns vary significantly across languages.
\end{abstract}

\section{Introduction}
\label{sec:introduction}

Sentiment analysis has evolved from simple polarity classification to sophisticated dimensional models that capture the nuanced emotional landscape of human expression. The Valence-Arousal (VA) framework, rooted in psychological research~\cite{russell1980}, represents emotions in a two-dimensional space where valence captures pleasantness (positive to negative) and arousal captures activation (calm to excited). This dimensional approach provides richer insights than categorical sentiment labels, enabling applications in recommendation systems, customer service, and social media analysis.

Aspect-based sentiment analysis (ABSA) further refines this by associating emotional scores with specific aspects mentioned in text. For instance, in the review ``The food was excellent but the service was terrible,'' traditional ABSA might identify positive sentiment for ``food'' and negative for ``service.'' Dimensional ABSA (DimABSA) extends this by predicting continuous VA scores, allowing for more precise emotional characterization---perhaps food receives high valence and moderate arousal, while service receives low valence and high arousal.

Despite these advantages, dimensional aspect-based sentiment analysis faces significant challenges. Most notably, existing models exhibit a performance asymmetry: valence prediction consistently outperforms arousal prediction across multiple languages and domains. This asymmetry stems from several factors. Valence often correlates more directly with explicit sentiment markers---words like ``excellent'' or ``terrible'' provide clear valence signals. Arousal, in contrast, requires understanding subtle intensity cues that may be expressed implicitly or through linguistic patterns that vary across languages. Consider how different languages express emotional intensity: English might use intensifiers (``absolutely amazing''), while Russian might rely on morphological variations or word order. Additionally, arousal is inherently more subjective and context-dependent, making it harder to learn from limited training data.

The multilingual nature of modern sentiment analysis further complicates this problem. Cross-lingual transfer learning has shown promise for valence prediction, but arousal patterns exhibit greater cross-lingual variation. For example, emotional intensity expressions differ substantially between languages---what constitutes ``high arousal'' in English may map differently to Russian or Ukrainian expressions. This suggests that effective multilingual dimensional sentiment analysis requires both shared representations and language-specific adaptations.

In this work, we propose an arousal-focused architecture that addresses these challenges through several key innovations. We design separate prediction pathways for valence and arousal, recognizing that these dimensions require different feature representations. The arousal pathway incorporates multi-head self-attention to capture emotional intensity patterns and convolutional layers to extract intensity features from the sequence. We introduce a weighted loss function that emphasizes arousal prediction, directly addressing the performance asymmetry. Finally, we employ a two-stage training strategy: multilingual pretraining on diverse languages (English, Japanese, Russian, Ukrainian) followed by domain-specific fine-tuning, enabling effective knowledge transfer while maintaining domain adaptability.

Our approach is evaluated on SemEval-2026 Task 3 (DimABSA), which involves predicting VA scores for aspects across multiple languages and domains. We demonstrate substantial improvements over baseline approaches: average RMSE reduction of 36.2\% across English, Russian, and Ukrainian, with Russian showing a remarkable 47.9\% improvement. Arousal prediction RMSE improves by 24--48\% across languages, while maintaining strong valence performance. These results validate our hypothesis that architectural specialization and loss weighting are crucial for balanced VA prediction.

The contributions of this work are threefold: (1) we introduce an arousal-focused architecture with separate pathways and enhanced feature extraction that significantly improves arousal prediction; (2) we demonstrate the effectiveness of weighted loss functions and two-stage multilingual training for dimensional sentiment analysis; and (3) we provide comprehensive empirical analysis across multiple languages and domains, revealing insights about cross-lingual transfer in dimensional sentiment tasks.

The remainder of this paper is organized as follows. Section~\ref{sec:methodology} describes our methodology, including the arousal-focused architecture and training strategy. Section~\ref{sec:dataset} presents the datasets and experimental setup. Section~\ref{sec:experiments} presents experimental results and analysis. Section~\ref{sec:conclusion} concludes with future directions.

\section{Task Overview}


% Using context. Context as history.
Extracting potential causes that lead to emotion expressions in text is the crucial aim of Emotion Cause Extraction (ECE) domain~\cite{xia2019emotion}.
In particular, the
SemEval-2024~Task~3~\cite{wang-EtAl:2024:SemEval20244} is aimed at
emotion-cause pair analysis in conversations from 
the sitcom \textit{Friends}. 
The conversations are organized into Emotion-Cause-in-Friends dataset~\cite{wang2023multimodal} and includes the JSON-formatted 
training (\trainData{}) and evaluation (\testData{}) parts.
% labels definition
The authors propose $6$ emotion classes to annotate: 
(i)~speaker emotion states, and 
(ii)~emotion caused by one utterance to the other.
These classes are: $\labelsMath{}=\{\text{\textsc{\small surprise, sadness, joy, disgust, fear, anger}}\}$, and {\small \textsc{neutral}} for absence of emotion.
We denote $\labelsAllMath{}=\labelsMath{}~\cup~\{\text{\small \textsc{neutral}}\}$ as a complete set.


Among the several subtasks of ECAC-2024, in this paper we focused on 
\textit{Subtask~1}: textual emotion-cause pair extraction in conversations.
In this subtask,
each conversation represents a list of utterances.
% Conversation parameters.
Every utterance ($u$) yields the following:
utterance text ($u_{text}$), 
speaker name ($u_{speaker}$), 
emotion state ($u_{state}\in\labelsAllMath{}$), 
and \textsc{id} ($u_{id}$).
The annotation of the emotion cause pairs represents a list $P=[ p_1 \ldots p_{|P|}]$, 
in which each pair $p\in P$
is a labeled source-target\footnote{Spans-prediction is beyond the scope of our methodology.} 
tuple
$p=\left<u^{src}, u^{tgt}, e_{c}\right>$,
where
$e_{c} \in \labelsMath{}$.

%We start our methodology with the selection of emotion-cause pairs .~\todo{[How we can start methodology? Start creation or other aspect that actually starts? Maybe \textit{rely on} instead?]}
We initiate our studies by analyzing the training data (\trainData{}) for the subject of annotated emotion-cause pairs $\left<u^{src}, u^{tgt}\right>$ in it, and report:
\begin{enumerate}
    \item Quantitative statistics 
    of the mentioned emotion-cause pairs (Table~\ref{tab:train_analysis_s});
    \item Distance statistics (in utterances) between 
    $u^{src}$ and $u^{tgt}$ (Table~\ref{tab:train_analysis_dist});
    \item Distribution statistics between speaker state ($u_{state}$) and emotion \textit{speaker causes} ($e^{u\to*}$)
    (Table~\ref{tab:train_analysis_se}).
\end{enumerate}

\input{tables/train_analysis}

According to the Table~\ref{tab:train_analysis_dist},
most emotion was found to be caused by such utterances $u^{src}$ that are
the same as or mentioned before $u^{tgt}$ ($\delta \geq 0$).  
% Context definition
Therefore, 
given $\left<u^{src}, u^{tgt}\right>$
we denote its context
$X = \{u^1 \ldots u^k\}$
as a \textit{history} of the past $k-1$
utterances of $u^{tgt}$, where $u^{tgt}=u^k \in X$, $u^{src} \in X$.
% Task definition: context and pairs.
%We define the emotion cause pair prediction task as follows.
\\
\textbf{Task definition:} Given an emotion-causing utterance pair within context
$\left< u^{src}, u^{tgt}, X\right>$ answer the emotion $e_{c}~\in~\labelsAllMath{}$ caused by $u^{src}$ towards $u^{tgt}$.


% Methodology core.
\section{Methodology}
\label{sec:methodology}

% Concept of the training.
We propose a two-stage training mechanism
for performing
instruction-tuning
on
large language models (LLMs),
aimed at accurately inferring of the task answers.
Given triplet $\left<u^{src}, u^{tgt}, X\right>$
of emotion-cause pair 
$\left<u^{src}, u^{tgt}\right>$
in context 
$X$, the proposed mechanism aims at LLM instruction-tuning, in order to answer $e \in E'$ that refers to: 
\begin{enumerate}[label={\textsc{stage \arabic*}:},leftmargin=2.1cm]
    \item emotion state $u^{tgt}_{state}$;
    \item emotion cause by $u^{src}$ to $u^{tgt}$. 
\end{enumerate}
% Answer for the task
Therefore, for emotion-cause pairs extraction we use the \textsc{stage 2} towards the model tuned in \textsc{stage 1} to infer $e_{c}~\in~\labelsAllMath{}$ caused by $u^{src}$ towards $u^{tgt}$.

Instead of directly asking LLM the final result at each stage, we exploit the Chain-of-Thought (CoT) concept in the form of the Three-hop Reasoning (\textsc{THoR}) framework~\cite{FeiAcl23THOR}. 
We believe that LLM can infer the span that conveys emotion and opinion about it before answering $e\in \labelsAllMath{}$.
Figure~\ref{fig:three-hop-methodology-adaptation} illustrates the proposed training methodology, empowered by the CoT prompting.
We refer to the instruction-tuning mechanisms of the
\textsc{stage~1} and 
\textsc{stage~2}
as
\thorState{} and
\thorCause{}
respectively.

\begin{figure*}[!t]
 \centering
  \includegraphics[width=\textwidth]{pics/pipeline.png}
  % Span -> Opinion -> Emotion.
  \caption{Two-stage LLM tuning methodology for inferring emotion caused by $u^{src}$ towards $u^{tgt}$ in context $X$ by adapting \thor{}~\cite{FeiAcl23THOR} 
 to reason and answer: (i) $u^{tgt}_{state}$ 
  (\thorState{}), and (ii) emotion caused by $u^{src}$ towards $u^{tgt}$ (\thorCause{}), optionally enhanced by Reasoning-Revision and by predicting $u^{src}_{state}$ (\thorCauseRR{}).}
  \label{fig:three-hop-methodology-adaptation}
\end{figure*}


\subsection{Chain-of-Thought Prompting}
\label{sec:cot-prompting}

%\textcolor{red}{[use "use" or "adopt", don't use consider. Consider is when you design your approach, you consider various factors or situations. If you used something, it's not consider.]} 
We adopt the \thor{} framework~\cite{FeiAcl23THOR} in LLM fine-tuning with the prompt templates adapted for 
emotion-cause pair analysis in conversations.
We define the intermediate \textit{span} ($s$) and latent \textit{opinion} expression ($o$). 
With $C_{i}, i\in \overline{1..3}$ we denote the prompts that wrap the content in the input context.
The construction of stages is as follows.

% FIRST STAGE
\textbf{\thorState{}}
This is a \textsc{stage 1} of the proposed training methodology, aimed at 
preliminary LLM instruction-tuning. 
Given $\left<u^{tgt}, X\right>$, we apply the following three steps to infer $u^{tgt}_{state} = e'_{1} \in \labelsAllMath{}$: 
\begin{tcolorbox}[breakable,colframe=gray,boxrule=2pt,arc=3.4pt,boxsep=-1mm]
    \textbf{Step~1}:%
    {
    $s'_1$ =
    [$C_1(X)$, which text spans are possibly causes emotion on $u^{tgt}_{text}$?]
    }  
\end{tcolorbox}
\begin{tcolorbox}[breakable,colframe=gray,boxrule=2pt,arc=3.4pt,boxsep=-1mm]
    \textbf{Step~2}: $o'_{1}$ = 
    [$C_2(C_1, s'_{1})$. 
    Based on the common sense, what is the implicit opinion towards the mentioned text spans that causes emotion on $u^{tgt}_{text}$, and why?]
\end{tcolorbox}
\begin{tcolorbox}[breakable,colframe=gray,boxrule=2pt,arc=3.4pt,boxsep=-1mm]
    \textbf{Step~3}: $e'_{1}=$
    [$C_3(C_2, o'_{1})$. Based on such opinion, what is the emotion state of $u^{tgt}_{text}$?]
\end{tcolorbox}
where $s'_{1}$ could be interpret as 
$s'_{1} = argmax~p(s_{1}|X,u^{tgt}_{text})$,
latent opinion $o'_{1}$ as $o'_{1} = argmax~p(o_{1}|X,u^{tgt}_{text},s'_{1})$, and
the final answer $e'_{1}$ noted as: $e'_{1}= argmax~p(e_{1}|X,u^{tgt}_{text},s'_{1},o'_{1})$.

% SECOND STAGE
\textbf{\thorCause{}} 
This is a \textsc{stage 2} of the proposed methodology, based on emotions-cause pairs.
We use this stage for (i)~\textit{fine-tuning} and (ii)~task result \textit{inferring} purposes.
%Due to the task definition (Section~\ref{sec:methodology}), 
Given context $\left<u^{src}, u^{tgt}, X\right>$ % and pair $\left<u_i, u_k\right>$,
we omit\footnote{To reduce the problem statement to the one for which \thor{} was originally designed~\cite{pontiki2016semeval}} $u^{tgt}\in X$ from the input parameters by referring to it as <<\textit{end of the conversation}>>.
We apply the following steps to infer 
$e_{2}' \in \labelsAllMath{}$ 
caused by $u^{src}$ to $u^{tgt}$:
\begin{tcolorbox}[breakable,colframe=gray,boxrule=2pt,arc=3.4pt,boxsep=-1mm]
    \textbf{Step 1}:~$s'_{2}$ = 
    [$C_1(X)$, which specific text span of $u^{src}_{text}$ is possibly causes emotion?]
\end{tcolorbox}
\begin{tcolorbox}[breakable,colframe=gray,boxrule=2pt,arc=3.4pt,boxsep=-1mm]
    \textbf{Step 2}:~$o'_{2}$ = 
    [$C_2(C_1, s_{2}')$. Based on the common sense, what is the implicit opinion towards the cause of mentioned text span of $u^{src}_{text}$, and why?]
\end{tcolorbox}
\begin{tcolorbox}[breakable,colframe=gray,boxrule=2pt,arc=3.4pt,boxsep=-1mm]
    \textbf{Step 3}:~$e'_{2}$ = 
    [$C_3(C_2, o'_{2})$. Based on such opinion, what is the emotion caused by {source} towards the last conversation utterance?]
\end{tcolorbox}
where $s'_{2}$ could be interpret as 
$s'_{2} = argmax~p(s'_{2}|X,u^{src}_{text})$,
opinion $o'$ could be interpret as 
$o'_{2} = argmax~p(o_{2}|X,u^{src}_{text},s'_{2})$, 
and the  final answer $e'_{2}$ noted as: 
$e'_{2}~=~argmax~p(e_{2}|X,u^{src}_{text},s'_{2},o'_{2})$.


\subsection{Reasoning Revision with Supervision}
\label{sec:reasoning-revision}

During the LLM instruction-tuning process with the \thor{}, it is possible to devise a reasoning path.
Technically, at each step of the chain we have all the necessary information to query our model with the final answer. 
With the following approach, we believe in a better model alignment on state-cause dependency (Table~\ref{tab:train_analysis_se}):
speakers are likely to cause an emotion, similar to their states\footnote{Except \textsc{neutral} speaker state (Table~\ref{tab:train_analysis_se})}.
%Due to the relatively reach annotation, in the cause of cause emotion prediction, 
%we are aware of the speaker state.
To revise this knowledge, in this paper, we impute the following prompt to support our opinion $O$, obtained at the end of the \thorCause{} step~2 (Fig.~\ref{fig:three-hop-methodology-adaptation}):
\begin{tcolorbox}[breakable,colframe=gray,boxrule=2pt,arc=3.4pt,boxsep=-1mm]
\textbf{Step 3.1}: $u'^{src}_{state} = $[$C_3$($C_2$, $o'_{2}$), Based on such opinion, what is the emotion state of $u^{src}_{text}$?]
\end{tcolorbox}
Due to the definition of the task, 
we believe in the correctness of this knowledge within the emotion cause task. 
%Finally, we complement the response by asking to predict the result emotion state $e'_{state}$ .
Once step 3.1 is embedded,
the result answer
$e'_{2} \in E'$ 
in
\thorCause{}
from the step~3
could be reinterpret as
$e'_{2}~=~argmax~p(e_{2}|X, u^{src}_{text}, s'_{2}, o'_{2}, u'^{src}_{state})$.
We refer to this setup as
\thorCauseRR{}.

\section{Datasets and Experiential Setup}
\label{sec:dataset}

\textcolor{red}{[Add description for 80 / 20 \% split of the dataset]}


\input{tables/datasets}


\section{Experiments}
\label{sec:experiments}

\textcolor{red}{[Compose table for experiments (similar to Table 5.)]}

% Providing analysis details
\input{pics/analysis}

\textbf{Stage 1}.
Figure~\ref{fig:analysis-emotion-state} illustrates the analysis of the 
$F1$ on \texttt{dev} part during the preliminary tuning
of Flan-T5\textsubscript{base}
on \dState{}.\footnote{We left the comparison with other pre-training techniques listed in~\ref{sec:dataset} out of scope of this paper due to alignment with the CoT concept in \textsc{stage 2}.}
We investigate the overfitting after $2$ epochs of training.
The best state, obtained at the end of the epoch~\#2 
with the \fdev{}$=47.81$ on the \dState{}-\texttt{dev} part,
has been selected. 
In further, we refer to this model as
Flan-T5\textsubscript{base}\textdagger{}.
\\
\textbf{Stage 2}
Figure~\ref{fig:analysis-emotion-cause} provides a comparative analysis of different fine-tuning techniques.
As at the pre-training stage, we investigate the ability to learn task emotion states 2-3 training epoch, followed by overfitting.
Switching from \textsc{prompt} to \thorCauseRR{} technique,
we investigate the improvement by $2.5$\% percent by \fdev{} on the \texttt{dev} part
of the \dState{} dataset. 
We refer to the best fine-tuned versions as Flan-T5\textsubscript{base}\textdaggerdbl{},
separately per each fine-tuning technique in Table~\ref{tab:results} (\texttt{dev} column).

The official evaluation includes the following $F1$ measures: 
(i)~weighted averaged $F1^{w}_{*}$ / non-weighted ($F1_{*}$), 
and 
(ii)~strict ($F1_{s}$) / not-strict ($F1_{p}$) towards predicted spans. 
%Since our task task definition does not involve spans, 
To form the submissions for official evaluation, 
the following span corrections approaches were used:
(i)~punctuation terms\footnote{We use \texttt{string.punctuation}  preset in Python}
exclusion from utterance prefixes and suffixes (by default), and (ii)~algorithm-based (Section~\ref{sec:spans-correction}).
Table~\ref{tab:results} (\texttt{test} columns) illustrate the available results of 
T5\textsubscript{base}\textdaggerdbl{} in official evaluation.
\\
% Final submission.
\textbf{Final submission} represents the results of Flan-T5\textsubscript{base}\textdaggerdbl{} 
(\textsc{THOR\textsubscript{cause-rr}} technique),
and application of algorithm-based spans correction.

\input{tables/results}%
\input{tables/final_submission_analysis}%


% Prefix and suffix vocabulary.
\subsection{Algorithm-based Spans Correction}
\label{sec:spans-correction}
Our methodology (Section~\ref{sec:methodology}) is limited on utterance level emotion cause prediction.\footnote{
Technically it is possible to obtain spans (Section~\ref{sec:methodology}), 
however we could not investigate the practical valuty of the 
\thorCause{}-based Flan-T5\textsubscript{base}\textdaggerdbl{} 
responses from step~\#1.}
%For all the submissions we follow the organizers guidance on correction of exclusion of punctuation signs from the spans.  
We believe it is reflected in the relatively low results of $F1_{s}$ 
on the \texttt{test} dataset (see Table~\ref{tab:results}).
Therefore, we analyze \trainData{}
and adopt a placeholder solution, 
aimed at enhancing the results by $F1_{s}$.
\input{tables/span_correction_algo}

We apply a \textit{rule-based approach} 
based on differences between the original utterance texts and their span annotations in the training data.
Using
\trainData{}, we compose 
\textit{prefix-} ($V_p$) 
and 
\textit{suffix-} ($V_s$) 
vocabularies. 
For vocabulary entries, we select those that satisfy all of the following criteria:
(i)~the length of entry does not exceed $5$ words,
(ii)~entry starts 
(in the case of $V_s$), or ends
(in the case of $V_p$)
with the punctuation sign\textsuperscript{7}.

For each utterance text ($u_{text}$) that causes emotion, 
we compose an updated $u_{text}'$ by applying:
(1)~correction of $u_{text}$ prefixes with $V_p$, followed by 
(2)~correction of suffixes from $V_s$ for the results from (1).
We alter $u_{text}'$ in the case of
$u_{text}'=\emptyset$.
The algorithm~\ref{alg:rule-based-method}
illustrates an implementation for the prefixes correction with~$V_p$.\footnote{
Implementation is publicly available in 
\url{https://github.com/nicolay-r/SemEval2024-Task3}}


\section{Conclusion}
\label{sec:conclusion}

In this paper, we present a Chain-of-Thought (CoT) methodology aimed at
fine-tuning LLM for emotion state and cause extraction. 
We consider the problem of 
\textit{emotion cause analysis in conversations}
as a context-based problem with the mentioned utterance that causes emotion towards the last utterance in context.
We devise our CoT for emotion causes and propose a reasoning revision methodology aimed at imputing the speaker emotion to support the decision on caused emotion.
Our CoT represent a Three-hop Reasoning approach priory known as \thor{}.
We apply this approach to fine-tune LLM and predict: 
(i)~emotion state of the mentioned utterance, and (ii)~emotion caused by mentioned utterance towards the last utterance in context. 
We experiment with the Flan-T5\textsubscript{base} (250M) model fine-tuning 
using resources provided by task organizers.
The application of CoT with reasoning revision allows us
to improve the results by $2.5$\% (F1-measure) compared to prompt-based tuning. 
In further work, we expect to contribute with the:
(i)~analysis of larger models, and 
(ii)~enhanced reasoning revision techniques, mentioned in the
final submission analysis.

\bibliography{main} % Entries are in the refs.bib file

\end{document}
