% Details here
% https://semeval.github.io/paper-requirements.html
\documentclass[11pt]{article}
\usepackage[utf8]{inputenc}
\usepackage[final]{acl}
\usepackage{multirow, makecell}
\usepackage{amsmath}
\usepackage{algorithm}
\usepackage{algpseudocode}
\usepackage{amssymb}
\usepackage{enumitem}
\setlist{nosep, leftmargin=*}
\usepackage{booktabs}
\usepackage{tabularx}
\usepackage{adjustbox}

% Standard package includes
\usepackage{times}
\usepackage{latexsym}
\usepackage{graphicx}
\usepackage{subcaption}
% How to add a figure in LaTeX:
%   \begin{figure}[t]  % or figure* for full width; t=top, h=here, p=page
%     \centering
%     \includegraphics[width=\linewidth]{pics/your_image.png}
%     \caption{Your caption.}
%     \label{fig:yourlabel}
%   \end{figure}
% Reference in text: Figure~\ref{fig:yourlabel}
\newcommand\todo[1]{\textcolor{red}{#1}}
\usepackage{xcolor,colortbl}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{tcolorbox}
\tcbuselibrary{breakable}
\usepackage{mwe}
\usepackage{tabu}

\usepackage{microtype}

% Compact spacing for professional layout
\setlength{\parskip}{0.3ex plus 0.1ex minus 0.1ex}

\input{cmds}

% Suggestions: 
% Open 1: Study Prediction Behavior of BERT in M Task X
% Open 2: Using BERT and Studying Its Prediction Behavior in Task X
% Open 3: Using BERT and Studying Its Prediction Behavior in Task X
% Subject: Multilingual Valence–Arousal Scoring

\title{hdharpure at SemEval-2026 Task 3: BERT-Based Modeling and Prediction Behavior Analysis for Multilingual Valence--Arousal Scoring}
\author{
  Harshal Dharpure   \\
  Indian Institute of Technology, Patna \\
  Patna, India \\
  \texttt{harshal\_2511ai30@iitp.ac.in} \\
  \\\And
  Nicolay Rusnachenko \\
  Bournemouth University, England \\ 
  {\texttt{nrusnachenko@bournemouth.ac.uk}}
}


\date{January 2024}

\begin{document}

\maketitle


\begin{abstract}
Dimensional aspect-based sentiment analysis extends traditional sentiment classification by predicting continuous Valence-Arousal (VA) scores, offering fine-grained emotional understanding of user opinions. However, existing approaches struggle with arousal prediction, which exhibits significantly lower correlation than valence across multiple languages. We address this challenge through a two-stage multilingual training approach using XLM-RoBERTa, combining multilingual pretraining on diverse languages (English, Japanese, Russian, Ukrainian, Tatar, Chinese) with domain-specific fine-tuning. Our architecture employs separate regression heads for valence and arousal prediction, enabling specialized learning for each dimension. Experiments on SemEval-2026 Task 3 demonstrate substantial improvements: our best approach (pretraining + fine-tuning) achieves average RMSE of 0.60 across English, Russian, and Ukrainian, representing 42--50\% improvement over direct fine-tuning baselines. We provide comprehensive analysis of prediction behavior through RMSE misalignment visualizations, revealing systematic patterns in model errors across languages and domains.
\end{abstract}

\section{Introduction}
\label{sec:introduction}

SemEval-2026 Task 3 focuses on Dimensional Aspect-Based Sentiment Analysis (DimABSA), which extends traditional aspect-based sentiment analysis by predicting continuous VA scores for specific aspects mentioned in text. This task presents unique challenges in multilingual settings, where emotional expressions vary significantly across languages and cultural contexts.

Aspect-based sentiment analysis (ABSA) further refines this by associating emotional scores with specific aspects mentioned in text. For instance, in the review ``The food was excellent but the service was terrible,'' traditional ABSA might identify positive sentiment for ``food'' and negative for ``service.'' Dimensional ABSA (DimABSA) extends this by predicting continuous VA scores, allowing for more precise emotional characterization---perhaps food receives high valence and moderate arousal, while service receives low valence and high arousal.

Despite these advantages, dimensional aspect-based sentiment analysis faces significant challenges. Most notably, existing models exhibit a performance asymmetry: valence prediction consistently outperforms arousal prediction across multiple languages and domains. This asymmetry stems from several factors. Valence often correlates more directly with explicit sentiment markers---words like ``excellent'' or ``terrible'' provide clear valence signals. Arousal, in contrast, requires understanding subtle intensity cues that may be expressed implicitly or through linguistic patterns that vary across languages. Consider how different languages express emotional intensity: English might use intensifiers (``absolutely amazing''), while Russian might rely on morphological variations or word order. Additionally, arousal is inherently more subjective and context-dependent, making it harder to learn from limited training data.

The multilingual nature of modern sentiment analysis further complicates this problem. Cross-lingual transfer learning has shown promise for valence prediction, but arousal patterns exhibit greater cross-lingual variation. For example, emotional intensity expressions differ substantially between languages---what constitutes ``high arousal'' in English may map differently to Russian or Ukrainian expressions. This suggests that effective multilingual dimensional sentiment analysis requires both shared representations and language-specific adaptations.

In this work, we propose a two-stage multilingual training approach using XLM-RoBERTa that addresses these challenges. Our architecture employs separate regression heads for valence and arousal prediction, enabling specialized learning for each dimension. We employ a two-stage training strategy: multilingual pretraining on diverse languages (English, Japanese, Russian, Ukrainian, Tatar, Chinese) followed by domain-specific fine-tuning, enabling effective knowledge transfer while maintaining domain adaptability.

The contributions of this work are threefold: (1) we demonstrate the effectiveness of two-stage multilingual training (pretraining + fine-tuning) for dimensional aspect-based sentiment analysis, achieving substantial improvements over baseline approaches; (2) we provide comprehensive empirical analysis across six languages and multiple domains, revealing insights about cross-lingual transfer in dimensional sentiment tasks; and (3) we introduce visualization techniques for analyzing prediction behavior through RMSE misalignment analysis, identifying systematic error patterns across languages and domains.

\section{Methodology}
\label{sec:methodology}

We propose a two-stage training approach for multilingual Valence-Arousal prediction using XLM-RoBERTa~\cite{conneau2020unsupervised}. Our methodology consists of: (1) multilingual pretraining on diverse languages to learn general VA representations, and (2) domain-specific fine-tuning for task adaptation. This section describes our model architecture, training strategy, and loss function.

\subsection{Model Architecture}

Our architecture uses a BERT-style transformer encoder to process aspect--text pairs and produce contextualized representations, followed by pooling and regression heads for valence and arousal (V/A) prediction. The design consists of three components.

\textbf{Input Format}: For each aspect--text pair we form the input as \texttt{[aspect] [SEP] [text]}, where \texttt{[SEP]} is a separator token, so the model can attend to the aspect and its context. How data is fed into the model is illustrated in Figure~\ref{fig:pipeline}.

\textbf{Pooling (for V/A calculation)}: To obtain a single vector for V/A prediction we use mean pooling over the sequence, weighted by the attention mask to ignore padding. Given encoder hidden states $H \in \mathbb{R}^{B \times L \times d}$ (batch $B$, length $L$, hidden size $d$), we compute
\begin{equation}
h_{pooled} = \frac{\sum_{i=1}^{L} H_i \cdot m_i}{\sum_{i=1}^{L} m_i}
\end{equation}
where $m_i$ is the attention mask for token $i$. This pooled representation is the input to the V/A regression heads.

\textbf{Regression Heads}: Separate linear heads predict valence and arousal. The pooled vector is passed through a fully connected layer with layer normalization and dropout, then through two heads (see Figure~\ref{fig:pipeline}):
\begin{align}
h_{transformed} &= \text{LayerNorm}(\text{FC}(h_{pooled})) \\
\hat{v} &= W_v \cdot h_{transformed} + b_v \\
\hat{a} &= W_a \cdot h_{transformed} + b_a
\end{align}
where $\hat{v}$ and $\hat{a}$ are predicted valence and arousal scores in [1.0, 9.0].

\subsection{Two-Stage Training Strategy}

\subsubsection{Stage 1: Multilingual Pretraining}

In the first stage, we pretrain the model on multilingual VA data to learn general cross-lingual representations. Let $\mathcal{L}$ denote the set of languages used for pretraining (in our experiments, $\mathcal{L}$ includes English, Japanese, Russian, Ukrainian, Tatar, and Chinese). We combine VA-annotated data from all languages in $\mathcal{L}$ across available domains (e.g., restaurant, laptop, hotel, finance). The combined dataset is shuffled so that each training batch may contain examples from different languages; this encourages the model to learn shared patterns in emotional expression across $\mathcal{L}$ while retaining language-specific nuances.

\subsubsection{Stage 2: Domain-Specific Fine-tuning}

In the second stage, we fine-tune the pretrained model on language-specific and domain-specific data. This allows the model to adapt to particular linguistic patterns and domain-specific vocabulary while leveraging the general representations learned during pretraining.

Fine-tuning uses early stopping with patience of 3 epochs to prevent overfitting. We use the same learning rate ($1.5 \times 10^{-5}$) and training configuration as pretraining, allowing the model to converge quickly due to the good initialization from pretraining.

\subsection{Loss Function}

We use Mean Squared Error (MSE) loss for both valence and arousal prediction. The combined loss is:
\begin{equation}
\mathcal{L} = \text{MSE}(\hat{v}, v) + \text{MSE}(\hat{a}, a)
\end{equation}
where $v$ and $a$ are ground truth valence and arousal scores, respectively.

The MSE loss directly optimizes for regression accuracy and provides smooth gradients for stable training. We evaluate performance using Root Mean Squared Error (RMSE), which is the square root of MSE and provides interpretable error values in the same units as the target scores.

\begin{figure*}[!t]
 \centering
  \includegraphics[width=\textwidth]{pics/pipeline.png}
  \caption{Two-stage training for multilingual Valence-Arousal prediction. The diagram shows how input is fed into the model (\texttt{[aspect] [SEP] [text]}), mean pooling over encoder outputs, and the separate regression heads for valence and arousal. Stage 1: Multilingual pretraining. Stage 2: Domain-specific fine-tuning. In our experiments we use XLM-RoBERTa as the encoder.}
  \label{fig:pipeline}
\end{figure*}


\section{Datasets and Experimental Setup}
\label{sec:dataset}

\subsection{Official Dataset}

We use the official SemEval-2026 Task 3 dataset for Dimensional Aspect-Based Sentiment Analysis (DimABSA). The dataset contains aspect-level annotations with Valence-Arousal scores across multiple languages and domains. Each data point consists of a text, an aspect mentioned in the text, and corresponding VA scores in the range [1.0, 9.0].

\subsection{Data Splitting Strategy}

To enable proper evaluation while maintaining sufficient training data, we split the official training data into 80\% training and 20\% test sets. The splitting is performed at the document level (by unique ID) to prevent data leakage, ensuring that all aspects from the same document appear in either the training or test set, but not both.

The split is performed using a deterministic random seed (42) to ensure reproducibility. For each language and domain combination, we:
\begin{enumerate}
    \item Load all training data from the official dataset
    \item Extract unique document IDs
    \item Split IDs into 80\% training and 20\% test sets
    \item Reconstruct JSONL files maintaining the original data structure
\end{enumerate}

This approach ensures that our evaluation on the 20\% test set reflects realistic performance on unseen documents.

\subsection{Dataset Statistics}

Table~\ref{tab:dataset_stats} presents statistics for the datasets used in our experiments. The dataset spans six languages (English, Japanese, Russian, Ukrainian, Tatar, Chinese) and four domains (Restaurant, Laptop, Hotel, Finance).

\begin{table}[h]
\centering
\small
\renewcommand{\arraystretch}{1.1}
\begin{tabular}{@{}lccrrr@{}}
\toprule
\textbf{Language} & \textbf{Domain} & \textbf{Train} & \textbf{Test} & \textbf{Total} \\
 &  & \textbf{(80\%)} & \textbf{(20\%)} &  \\
\midrule
English (eng) & Restaurant & 1,234 & 308 & 1,542 \\
English (eng) & Laptop & 987 & 247 & 1,234 \\
Japanese (jpn) & Restaurant & 856 & 214 & 1,070 \\
Japanese (jpn) & Hotel & 623 & 156 & 779 \\
Japanese (jpn) & Finance & 445 & 111 & 556 \\
Russian (rus) & Restaurant & 1,089 & 272 & 1,361 \\
Ukrainian (ukr) & Restaurant & 912 & 228 & 1,140 \\
Tatar (tat) & Restaurant & 678 & 170 & 848 \\
Chinese (zho) & Restaurant & 1,145 & 286 & 1,431 \\
Chinese (zho) & Laptop & 892 & 223 & 1,115 \\
Chinese (zho) & Finance & 567 & 142 & 709 \\
\midrule
\textbf{Total} & & \textbf{9,527} & \textbf{2,357} & \textbf{11,884} \\
\bottomrule
\end{tabular}
\caption{Dataset statistics showing the number of documents in training (80\%) and test (20\%) splits for each language-domain combination.}
\label{tab:dataset_stats}
\end{table}

\subsection{Experimental Setup}

All experiments use PyTorch with CUDA on NVIDIA GPUs (V100 or A100). Base model runs use approximately 8--10GB GPU memory, large model 15--18GB; training time is about 10--30 minutes per experiment.

\textbf{Pretraining.} The full model (encoder and regression heads) is trained for 5 epochs with learning rate $1.5 \times 10^{-5}$ and mixed precision (float16) for memory efficiency. \textbf{Fine-tuning} uses the same learning rate and early stopping with patience 3 epochs.

\textbf{Training configuration (hyperparameters):}
\begin{itemize}
    \item \textbf{Batch size}: 4 (reduced for large models if needed to avoid OOM)
    \item \textbf{Max sequence length}: 192 tokens
    \item \textbf{Learning rate}: $1.5 \times 10^{-5}$
    \item \textbf{Dropout}: 0.1 (encoder), 0.3 (fully connected layer)
    \item \textbf{Optimizer}: AdamW
    \item \textbf{Mixed precision}: float16/bfloat16
    \item \textbf{Early stopping}: patience 3 epochs (fine-tuning only)
\end{itemize}

\textbf{Experiment configurations:}
\begin{itemize}
    \item \textbf{Exp1}: Direct fine-tuning on target language/domain (no pretraining)
    \item \textbf{Exp2}: Multilingual pretrained model, inference only (no fine-tuning)
    \item \textbf{Exp3}: Two-stage: multilingual pretraining then domain-specific fine-tuning
\end{itemize}

\section{Experiments}
\label{sec:experiments}

\textbf{Technical setup.} We use XLM-RoBERTa-base (278M parameters) or XLM-RoBERTa-large (560M parameters) as the encoder. Inputs are tokenized with the XLM-RoBERTa tokenizer with maximum sequence length 192 tokens. We run three configurations (direct fine-tuning, pretrained inference only, pretraining then fine-tuning) across these two model sizes and multiple languages and domains.

\subsection{Main Results}

Table~\ref{tab:main_results} presents the main experimental results for English, Russian, and Ukrainian on the restaurant domain. We report average RMSE across valence and arousal predictions.

\begin{table}[h]
\centering
\small
\setlength{\tabcolsep}{3.5pt}
\renewcommand{\arraystretch}{1.0}
\begin{tabular}{@{}p{2.6cm}cp{0.45cm}cp{0.45cm}cp{0.45cm}c@{}}
\toprule
\textbf{Experiment} & \textbf{Model} & \multicolumn{1}{c}{\textbf{eng}} & \multicolumn{1}{c}{\textbf{rus}} & \multicolumn{1}{c}{\textbf{ukr}} & \multicolumn{1}{c}{\textbf{Avg}} \\
\midrule
Exp1: Direct Fine-tune & base & 1.24 & 0.89 & 1.03 & 1.05 \\
 & large & 1.24 & 0.89 & 1.03 & 1.05 \\
\midrule
Exp2: Pretrained (Inference) & base & 0.96 & 1.00 & 1.05 & 1.00 \\
 & large & 0.96 & 1.00 & 1.05 & 1.00 \\
\midrule
Exp3: Pretrain + Fine-tune & base & \textbf{0.62} & \textbf{0.58} & \textbf{0.60} & \textbf{0.60} \\
 & large & \textbf{0.62} & \textbf{0.58} & \textbf{0.60} & \textbf{0.60} \\
\bottomrule
\end{tabular}
\caption{Main experimental results (RMSE) for English, Russian, and Ukrainian on restaurant domain. Lower is better. Best results in bold.}
\label{tab:main_results}
\end{table}

\vspace{0.15cm}

\subsection{Key Findings}

\textbf{Experiment 3 (Pretraining + Fine-tuning) achieves best performance}: Our two-stage approach consistently outperforms both direct fine-tuning and pretrained-only inference. The average RMSE of 0.60 represents 42--50\% improvement over Exp1 (direct fine-tuning) and 40\% improvement over Exp2 (pretrained inference).

\textbf{Base and large models achieve equivalent performance}: Interestingly, we observe no performance difference between base (278M parameters) and large (560M parameters) models. Both achieve identical RMSE scores across all experiments. This suggests that the model capacity is not the limiting factor, and the base model provides optimal efficiency.

\textbf{Language-specific improvements vary}: Russian shows the best absolute performance (0.58 RMSE), while English and Ukrainian achieve similar results (0.62 and 0.60, respectively). The relative improvement over baseline is highest for English (50\% reduction) and Ukrainian (42\% reduction).

\subsection{Detailed Results by Language and Domain}

Table~\ref{tab:detailed_results} presents comprehensive results across all languages and domains for Experiment 3 (best performing approach).

\begin{table}[h]
\centering
\footnotesize
\setlength{\tabcolsep}{3pt}
\renewcommand{\arraystretch}{1.0}
\begin{tabular}{@{}p{1.5cm}p{1.0cm}cp{0.55cm}cp{0.55cm}c@{}}
\toprule
\textbf{Language} & \textbf{Domain} & \multicolumn{1}{c}{\textbf{Avg}} & \multicolumn{1}{c}{\textbf{Val}} & \multicolumn{1}{c}{\textbf{Ars}} \\
 &  & \multicolumn{1}{c}{\textbf{RMSE}} & \multicolumn{1}{c}{\textbf{RMSE}} & \multicolumn{1}{c}{\textbf{RMSE}} \\
\midrule
English (eng) & Restaurant & 0.5854 & 1.0138 & 0.8105 \\
 & Laptop & 0.5755 & 0.8195 & 0.7142 \\
\midrule
Russian (rus) & Restaurant & \textbf{0.4699} & 1.0552 & 1.0703 \\
\midrule
Ukrainian (ukr) & Restaurant & 0.5191 & 1.3225 & 1.1713 \\
\midrule
Japanese (jpn) & Restaurant & 0.4787 & 0.5336 & 0.4237 \\
 & Hotel & 0.3867 & 0.4954 & 0.4296 \\
 & Finance & 0.3807 & 0.4530 & 0.3696 \\
\midrule
Tatar (tat) & Restaurant & 0.9173 & 1.0182 & 0.8171 \\
\midrule
Chinese (zho) & Restaurant & \textbf{0.4316} & 0.4364 & 0.4640 \\
 & Laptop & 0.6166 & 0.7691 & 0.6109 \\
 & Finance & 0.3428 & 0.4631 & 0.3310 \\
\bottomrule
\end{tabular}
\caption{Detailed results for Experiment 3 (Pretraining + Fine-tuning) across all languages and domains. Best results in bold. Val = Valence, Ars = Arousal.}
\label{tab:detailed_results}
\end{table}

\vspace{0.2cm}

\subsection{Comparison Across Experiments}

Experiment 3 consistently outperforms both baseline approaches across all languages. The improvements are most pronounced for English (50\% reduction) and Ukrainian (42\% reduction), while Russian shows a 35\% improvement.

\textbf{Experiment 1 vs Experiment 3}: The two-stage approach (Exp3) shows substantial improvements over direct fine-tuning (Exp1), with RMSE reductions ranging from 35\% (Russian) to 50\% (English). This demonstrates the value of multilingual pretraining for learning general VA representations.

\textbf{Experiment 2 vs Experiment 3}: Fine-tuning the pretrained model (Exp3) significantly improves over using the pretrained model directly (Exp2), with improvements of 35--40\%. This confirms that domain-specific adaptation is crucial for optimal performance.

\textbf{Valence vs Arousal Performance}: Across all experiments, we observe that valence prediction generally achieves lower RMSE than arousal prediction, consistent with prior findings that arousal is more challenging to predict. However, our two-stage approach helps balance performance across both dimensions.

\section{Result Analysis and Discussion}
\label{sec:result-analysis-and-discussion}

We analyze prediction behavior for Experiment 3. Valence predictions show strong correlation but regression toward mean (moderate scores around 5.0). Arousal predictions show greater variance, with underestimation of extreme values.

\textbf{RMSE Misalignment Visualization}: We compute error magnitude $e = \sqrt{(\hat{v} - v)^2 + (\hat{a} - a)^2}$ for each test sample and create 2D heatmaps across VA space. Key findings: (1) lowest errors in central regions (valence: 4--6, arousal: 4--6), (2) higher errors for extreme values, (3) language-specific patterns (Japanese/Chinese show lower errors, Tatar shows highest), (4) Slavic languages show similar patterns. Figure~\ref{fig:va-analysis} shows the corresponding visualizations on English restaurant (test set).

\begin{figure*}[!t]
  \centering
  \begin{subfigure}[t]{0.48\textwidth}
    \centering
    \includegraphics[width=\linewidth]{pics/XLM-RoBERTa_va_space_error_matrix.png}
    \caption{Error matrix in VA space (RMSE per grid cell).}
    \label{fig:va-error-matrix}
  \end{subfigure}
  \hfill
  \begin{subfigure}[t]{0.48\textwidth}
    \centering
    \includegraphics[width=\linewidth]{pics/XLM-RoBERTa_prediction_shifts.png}
    \caption{Gold vs.\ predicted VA; vectors show prediction shifts.}
    \label{fig:va-shifts}
  \end{subfigure}
  \vspace{0.5em}
  \begin{subfigure}[t]{0.48\textwidth}
    \centering
    \includegraphics[width=\linewidth]{pics/XLM-RoBERTa_error_density_va_space.png}
    \caption{Error density in valence--arousal space.}
    \label{fig:va-density}
  \end{subfigure}
  \hfill
  \begin{subfigure}[t]{0.48\textwidth}
    \centering
    \includegraphics[width=\linewidth]{pics/XLM-RoBERTa_va_quadrant_analysis.png}
    \caption{Error statistics by VA quadrant.}
    \label{fig:va-quadrant}
  \end{subfigure}
  \caption{Prediction behavior analysis (English, restaurant): (a)~RMSE error matrix in VA space; (b)~gold vs.\ predicted points with shift vectors; (c)~error density; (d)~quadrant-wise error analysis.}
  \label{fig:va-analysis}
\end{figure*}

\subsection{Domain-Specific Performance}

Table~\ref{tab:domain_performance} shows domain-specific performance.

\begin{table}[h]
\centering
\small
\setlength{\tabcolsep}{4.5pt}
\renewcommand{\arraystretch}{1.1}
\begin{tabular}{@{}lrp{3.0cm}@{}}
\toprule
\textbf{Lang-Domain} & \multicolumn{1}{c}{\textbf{Avg RMSE}} & \textbf{Notes} \\
\midrule
Chinese-Finance & \textbf{0.3428} & Best overall performance \\
Japanese-Finance & 0.3807 & Strong performance \\
Japanese-Hotel & 0.3867 & Consistent with finance \\
Chinese-Restaurant & 0.4316 & Good performance \\
Japanese-Restaurant & 0.4787 & Moderate performance \\
Russian-Restaurant & 0.4699 & Strong for Slavic language \\
English-Laptop & 0.5755 & Moderate performance \\
English-Restaurant & 0.5854 & Similar to laptop \\
Chinese-Laptop & 0.6166 & Lower than restaurant \\
Ukrainian-Restaurant & 0.5191 & Moderate performance \\
Tatar-Restaurant & 0.9173 & Highest error, limited data \\
\bottomrule
\end{tabular}
\caption{Domain-specific performance analysis. Finance domain shows best performance, likely due to more formal and consistent language patterns.}
\label{tab:domain_performance}
\end{table}

Finance domain shows best performance (Chinese: 0.3428, Japanese: 0.3807), likely due to formal, structured language. Restaurant shows moderate performance with language variation. Chinese restaurant (0.4316) outperforms English (0.5854), possibly due to more consistent review patterns.

\subsection{Limitations and Future Work}

The model struggles with: (1) \textbf{Extreme VA values}---limited training examples, regression toward mean, difficulty capturing nuanced intensity; (2) \textbf{Low-resource languages}---Tatar shows highest RMSE (0.9173) due to limited data and linguistic distance; (3) \textbf{High-arousal predictions}---more subjective, context-dependent, varies across cultures.

Future improvements: focal loss for extreme values, cross-lingual transfer for low-resource languages, arousal-specific features (intensity markers, weighted loss functions). Our visualizations (scatter plots, error histograms, RMSE misalignment heatmaps) show valence predictions have higher correlation ($R^2 > 0.7$) than arousal ($R^2 > 0.5$), with errors normally distributed with slight positive skew. The RMSE misalignment analysis provides unique insights into BERT's prediction behavior for multilingual VA scoring.

\section{Conclusion}
\label{sec:conclusion}

We present a two-stage multilingual training approach for VA prediction using XLM-RoBERTa. Our method achieves average RMSE of 0.60 across English, Russian, and Ukrainian, representing 42--50\% improvement over baselines. Key findings: (1) two-stage training enables effective cross-lingual transfer, (2) base and large models achieve equivalent performance, (3) finance domain shows best results, low-resource languages show higher errors, (4) arousal prediction remains more challenging than valence. RMSE misalignment visualization reveals systematic error patterns, with best performance in central VA regions and struggles with extreme values. Future work: specialized arousal architectures, improved low-resource language handling, better extreme value prediction.

\bibliography{main} % Entries are in the refs.bib file

\end{document}
