\documentclass[11pt]{article}
\usepackage{amsmath}
% Change "review" to "final" to generate the final (sometimes called camera-ready) version.
% Change to "preprint" to generate a non-anonymous version with page numbers.
\usepackage{acl}
\bibliographystyle{acl_natbib}
% Standard package includes
\usepackage{times}
\usepackage{latexsym}

% For proper rendering and hyphenation of words containing Latin characters (including in bib files)
\usepackage[T1]{fontenc}
% For Vietnamese characters
% \usepackage[T5]{fontenc}
% See https://www.latex-project.org/help/documentation/encguide.pdf for other character sets

% This assumes your files are encoded as UTF8
\usepackage[utf8]{inputenc}

% This is not strictly necessary, and may be commented out,
% but it will improve the layout of the manuscript,
% and will typically save some space.
\usepackage{microtype}

% This is also not strictly necessary, and may be commented out.
% However, it will improve the aesthetics of text in
% the typewriter font.
\usepackage{inconsolata}

%Including images in your LaTeX document requires adding
%additional package(s)
\usepackage{graphicx}

% If the title and author information does not fit in the area allocated, uncomment the following
%
\setlength\titlebox{8cm}
%
% and set <dim> to something 5cm or larger.

\title{blue at SemEval-2026 Task 5: \\ 
NarrBERT : Narrative-Aware BERT for Word Sense Disambiguation}

% Author information can be set in various styles:
% For several authors from the same institution:
% \author{Author 1 \and ... \and Author n \\
%         Address line \\ ... \\ Address line}
% if the names do not fit well on one line use
%         Author 1 \\ {\bf Author 2} \\ ... \\ {\bf Author n} \\
% For authors from different institutions:
% \author{Author 1 \\ Address line \\  ... \\ Address line
%         \And  ... \And
%         Author n \\ Address line \\ ... \\ Address line}
% To start a separate ``row'' of authors use \AND, as in
% \author{Author 1 \\ Address line \\  ... \\ Address line
%         \AND
%         Author 2 \\ Address line \\ ... \\ Address line \And
%         Author 3 \\ Address line \\ ... \\ Address line}
\author{
  \textbf{Rhea Singhal} \\
  Thapar Institute of Eng. \& Tech. \\
  Patiala, Punjab, India \\
  \texttt{rsinghal\_be23@thapar.edu} \And
  \textbf{Krish Sharma} \\
  Thapar Institute of Eng. \& Tech. \\
  Patiala, Punjab, India \\
  \texttt{ksharma8\_be23@thapar.edu} \AND
  \textbf{Lakksh Sharma} \\
  Thapar Institute of Eng. \& Tech. \\
  Patiala, Punjab, India \\
  \texttt{lsharma\_be23@thapar.edu} \And
  \textbf{Jatin Bedi} \\
  Thapar Institute of Eng. \& Tech. \\
  Patiala, Punjab, India \\
  \texttt{jatin.bedi@thapar.edu}
}
%\author{
%  \textbf{First Author\textsuperscript{1}},
%  \textbf{Second Author\textsuperscript{1,2}},
%  \textbf{Third T. Author\textsuperscript{1}},
%  \textbf{Fourth Author\textsuperscript{1}},
%\\
%  \textbf{Fifth Author\textsuperscript{1,2}},
%  \textbf{Sixth Author\textsuperscript{1}},
%  \textbf{Seventh Author\textsuperscript{1}},
%  \textbf{Eighth Author \textsuperscript{1,2,3,4}},
%\\
%  \textbf{Ninth Author\textsuperscript{1}},
%  \textbf{Tenth Author\textsuperscript{1}},
%  \textbf{Eleventh E. Author\textsuperscript{1,2,3,4,5}},
%  \textbf{Twelfth Author\textsuperscript{1}},
%\\
%  \textbf{Thirteenth Author\textsuperscript{3}},
%  \textbf{Fourteenth F. Author\textsuperscript{2,4}},
%  \textbf{Fifteenth Author\textsuperscript{1}},
%  \textbf{Sixteenth Author\textsuperscript{1}},
%\\
%  \textbf{Seventeenth S. Author\textsuperscript{4,5}},
%  \textbf{Eighteenth Author\textsuperscript{3,4}},
%  \textbf{Nineteenth N. Author\textsuperscript{2,5}},
%  \textbf{Twentieth Author\textsuperscript{1}}
%\\
%\\
%  \textsuperscript{1}Affiliation 1,
%  \textsuperscript{2}Affiliation 2,
%  \textsuperscript{3}Affiliation 3,
%  \textsuperscript{4}Affiliation 4,
%  \textsuperscript{5}Affiliation 5
%\\
%  \small{
%    \textbf{Correspondence:} \href{mailto:email@domain}{email@domain}
%  }
%}

\begin{document}
\maketitle
\begin{abstract}
This paper outlines the method submitted by team blue for the SemEval-2026 Task 5: Rating Plausibility of Word Senses in Ambiguous Sentences through Narrative (AmbiStory). The task requires predicting reasonable scores that match human thoughts and judgments instead of just picking a single correct sense as the output. This means that contextual reasoning with fine-grain contextual modeling is vital. In order to tackle this problem, we suggest a BERT-based cross-encoder regression model. This model encodes the entire narrative context, which includes the precontext, the ambiguous sentence, and the ending, along with candidate sense definitions and example usages. Unlike bi-encoder sentence-level methods, our model allows for token-level interaction between story cues and sense meanings. This interaction helps capture subtle narrative disambiguation signals. We conduct a systematic exploration of model architectures and training strategies, progressing from a sentence-transformer baseline to an optimised BERT cross-encoder. On the development set, our best configuration achieves a Spearman rank correlation of 0.66. On the official test set, the system achieves a Spearman correlation of \textbf{0.4866} and an Accuracy-within-Standard-Deviation of \textbf{0.6613}, substantially outperforming sentence-transformer bi-encoder baselines.
\end{abstract}

\section{Introduction}

Word Sense Disambiguation (WSD) in narrative contexts goes beyond traditional classification methods. Human interpretation of ambiguity is often more complicated than simply fitting it into categories.SemEval-2026 Task 5: Rating Plausibility of Word Senses in Ambiguous Stories through Narrative Understanding (AmbiStory) \cite{semeval2026-task-5} captures this idea. It requires systems to predict continuous plausibility scores that reflect human judgments. The cues for disambiguation in this task are often spread throughout the entire narrative, including the story's ending, making it inadequate to focus only on sentence-level modeling.

Figure~\ref{fig:task_overview} shows the overall task layout. Given a five-sentence narrative and a candidate sense definition, the system must provide a plausibility score ranging from 1 to 5 to match human annotations.

This task presents several challenges. Plausibility judgments can vary among annotators. The evaluation metrics focus on matching human disagreement. Furthermore, the narrative context needs to be understood as a whole. Initial experiments with sentence-transformer bi-encoders showed limited ability to model the detailed interactions between stories and sense descriptions, especially when disambiguation relies on subtle developments in the narrative.
\footnote{\url{https://github.com/RheaSinghal/SemEval-Task5}}

To solve these challenges, we suggest a narrative-aware BERT cross-encoder trained with a regression goal. Our model encodes the entire story context along with the candidate sense definition, allowing detailed token-level interaction between narrative cues and semantic gloss information. Unlike bi-encoder methods that process each input separately, our design directly models the relationship between context and meaning. This approach helps capture subtle resolution signals introduced at the end of the story. 



\begin{figure}[h]

    \centering
    \includegraphics[width=\linewidth]{latex/am1.png}
    \caption{Overview of the AmbiStory plausibility task. The model uses the full 5-sentence narrative to project a continuous score onto a candidate sense definition.}
    \label{fig:task_overview}
\end{figure}



% \section{Background and Related Work}

% \subsection{Task Description}
% The \textbf{AmbiStory} shared task (SemEval-2026 Task 5) reframes Word Sense Disambiguation (WSD) as a continuous plausibility estimation task intead of being a discrete classification problem. While traditional WSD systems aim to select a single ``correct'' sense from a predefined inventory \cite{Navigli:09}, AmbiStory acknowledges that human interpretation of ambiguous meanings is often graded and context-dependent \cite{Erk:09}.

% The dataset consists of short, five-sentence stories. Each story follows a specific structure: a \textit{pre-context} (sentences 1--3) establishes the scene; an \textit{ambiguous sentence} (sentence 4) contains a target homonym with two distinct meanings; and an \textit{ending} (sentence 5) typically resolves the ambiguity by implying one specific sense. 

% Formally, given a narrative context $C$ and a candidate sense definition $D$ of the target homonym, the system must predict a real-valued plausibility score $y \in [1, 5]$. This score reflects the degree to which sense $D$ is appropriate within context $C$. Evaluation is performed using Spearman rank correlation to measure the ordering quality of predictions against human judgments, and Accuracy within Standard Deviation (Acc-within-SD) to measure the alignment with human consensus.


% \subsection{From Discrete to Graded WSD}
% Standard WSD benchmarks, such as the Senseval and SemEval series \cite{Raganato:17}, typically treat word senses as mutually exclusive labels. Models trained on these datasets are optimized to maximize the probability of the gold-standard sense while minimizing others. However, this ``winner-takes-all'' approach fails to capture cases where multiple senses might be partially plausible, or where the context is intentionally underspecified.

% \citeauthor{Erk:09} \shortcite{Erk:09} argued early on that sense applicability is better modeled as a graded phenomenon. This perspective aligns with recent findings in cognitive linguistics, suggesting that humans retain multiple competing interpretations of ambiguous words until disambiguating evidence (such as the story ending in AmbiStory) becomes available \cite{Rodd:05}. Our approach leverages this theoretical grounding by utilizing a regression objective rather than soft-max classification, allowing the model to express uncertainty and varying degrees of fit.

% \subsection{Neural Approaches to Narrative Context}
% The shift from sentence-level to document-level WSD requires architectures capable of handling long-range dependencies. While early neural WSD models relied on LSTMs \cite{Yuan:16}, the introduction of Transformers \cite{Vaswani:17} and pre-trained language models like BERT \cite{Devlin:19} revolutionized the field. 

% A critical innovation relevant to our work is the \textbf{Gloss-BERT} approach proposed by \citeauthor{Huang:19} \shortcite{Huang:19}, which constructs input pairs containing both the target sentence and the dictionary gloss (definition) of the candidate sense. This transforms WSD into a sentence-pair classification task (Context vs. Gloss). We extend this methodology by replacing the standard sentence input with the full five-sentence narrative.

% Furthermore, we distinguish between \textit{Bi-Encoder} architectures, which map contexts and senses to independent vector spaces \cite{Reimers:19}, and \textit{Cross-Encoders}, which process them jointly. Although Bi-Encoders are computationally efficient for retrieval, \citeauthor{Hadiwinoto:19} \shortcite{Hadiwinoto:19} demonstrated that Cross-Encoders significantly outperform them in WSD tasks because they allow the self-attention mechanism to directly model the interaction between specific contextual cues and sense definitions. Our system adopts this Cross-Encoder strategy to capture the subtle semantic shifts introduced by the story endings.

% Finally, the AmbiStory format shares structural similarities with the ROCStories corpus \cite{Mostafazadeh:16}, which pioneered the evaluation of commonsense reasoning through five-sentence narratives. Just as the Story Cloze Test requires predicting a coherent ending, AmbiStory requires using the ending to retroactively value the plausibility of a prior ambiguity, a task that demands robust narrative consistency checking \cite{Wang:18}.
\section{Background and Related Work}
Traditional Word Sense Disambiguation (WSD) treats sense prediction as a discrete classification problem, assuming mutual exclusivity among senses \cite{Navigli:09, Raganato:17}. . However, since sense applicability is inherently graded rather than strictly categorical \cite{Erk:09}, AmbiStory \cite{semeval2026-task-5} frames WSD as a continuous plausibility estimation task over narratives, allowing multiple senses to be partially compatible.

Neural approaches to WSD have evolved from LSTM-based architectures \cite{Yuan:16} to Transformer-based models \cite{Vaswani:17}. Pre-trained models like BERT \cite{Devlin:19} significantly improved contextual learning, enabling stronger semantic performance. A relevant development is Gloss-BERT \cite{Huang:19}, which reformulates WSD as a sentence-pair task by jointly encoding a target sentence and candidate sense gloss. This allows contextual tokens to directly attend to definition tokens, improving disambiguation.

Recent studies further distinguish between \textit{Bi-Encoder} and \textit{Cross-Encoder} architectures \cite{Reimers:19, Hadiwinoto:19}. Bi-Encoders encode context and gloss independently and compare them in embedding space, offering computational efficiency but limiting fine-grained interaction. In contrast, Cross-Encoders jointly process both inputs, enabling token-level attention between contextual cues and sense definitions. Cross-Encoders have been shown to outperform Bi-Encoders in tasks requiring precise semantic alignment.

Our work builds upon the Gloss-BERT framework by extending the input from a single sentence to a full five-sentence narrative, incorporating precontext and story resolution. This modification enables the model to evaluate sense plausibility based on the complete narrative trajectory rather than localized lexical cues.


\section{System Overview}
\label{sec:system}


% --- INSERT DIAGRAM HERE ---
\begin{figure*}[t]
    \centering
    \includegraphics[width=0.85\linewidth]{latex/aaa.jpg}
    \caption{\textbf{System Architecture:} The BERT Cross-Encoder (left) jointly processes the narrative context and candidate sense. The raw regression output is then passed through a linear calibration layer (right) to align the scores with the human 1--5 plausibility scale.}
    \label{fig:system_arch}
\end{figure*}
% ---------------------------

The full layout of our approach is shown in Figure~\ref{fig:system_arch}. Our system treats the graded word sense plausibility task as a semantic regression task. It models the direct interaction between narrative trajectory and sense semantics.

To accomplish this, we created a unified BERT-based Cross-Encoder architecture.
% This architecture processes the story and the sense together, followed by a linear calibration stage to match the continuous outputs with human rating distributions.

\subsection{Component 1: Cross-Encoder Regression Architecture}
The core engine of our system is \texttt{bert-base-uncased}. We chose it because it captures deep bidirectional context. Unlike Bi-Encoder architectures that compress the story and meaning into separate vectors and lose detailed interaction, our Cross-Encoder approach makes the model focus on every token interaction between the narrative cues and the meaning definition.

Given a narrative context $C$ (which includes the pre-context, ambiguous sentence, and ending) and a candidate meaning $S$ (which includes the definition and example usage), the input sequence is built as:
\begin{equation}
    X_{input} = \texttt{[CLS]} \oplus C \oplus \texttt{[SEP]} \oplus S \oplus \texttt{[SEP]}
\end{equation}
The model processes this sequence to produce a contextualized embedding for the \texttt{[CLS]} token, $\mathbf{h}_{[\texttt{CLS}]}$. This representation is projected via a linear regression head to a scalar score $\hat{y}_{raw}$:
\begin{equation}
    \hat{y}_{raw} = \mathbf{W} \cdot \mathbf{h}_{[\texttt{CLS}]} + b
\end{equation}
The model is trained using Mean Squared Error (MSE) loss compared to the human average ratings, enabling it to learn the \textit{degree} of plausibility instead of just a correct or incorrect label.

\subsection{Component 2: Distributional Calibration}

While the Cross-Encoder effectively ranks senses, the raw regression outputs from BERT often suffer from ``scale compression,`` clustering around the mean of 2.5 to 3.5 instead of using the full 1 to 5 range. To solve this, we use a linear calibration step trained on the development set.

For a raw prediction $\hat{y}_{raw}$, the final calibrated score $\hat{y}_{final}$ is computed as:
\begin{equation}
    \hat{y}_{final} = \alpha \cdot \hat{y}_{raw} + \beta
\end{equation}
where $\alpha$ (scaling factor) and $\beta$ (shift) are coefficients fitted via least-squares regression. This component serves as a distribution aligner. It makes sure that highly plausible senses get scores close to 5.0, while implausible ones get scores near 1.0.

\subsection{Illustrative Example}
To illustrate the system's reasoning, consider a story containing the ambiguous word \textbf{``bank''}:
\begin{quote}
    \textit{Context:} ``John walked down the path. He saw the water flowing quickly. He sat down on the \textbf{bank}.''
\end{quote}

\begin{itemize}
    \item \textbf{Candidate A (Financial Institution):} The definition involves ``money'' and ``deposits.'' Although the word ``bank'' matches, the Cross-Encoder attention mechanism detects a semantic conflict between ``water/flowing'' in the context and ``financial'' in the definition. 
    \item \textbf{Candidate B (River Side):} The definition involves ``sloped land'' and ``water edge.'' The model attends to the explicit overlap of ``water'' in the story and ``water'' in the definition.
    \item \textbf{Result:} The raw BERT score might be $2.1$ for A and $3.8$ for B. After calibration, the system outputs $\hat{y}_{final}(A) \approx 1.2$ (Implausible) and $\hat{y}_{final}(B) \approx 4.9$ (Highly Plausible), matching human intuition.
\end{itemize}

\section{Experimental Setup}

\subsection{Data and Preprocessing}
We used the official AmbiStory dataset for SemEval-2026 Task 5. To create the input for our Cross-Encoder, we flattened the narrative structure. We combined the \textit{pre-context}, \textit{ambiguous sentence}, and \textit{ending} to form the first segment, and we paired the \textit{sense definition} with the \textit{example usage} to create the second segment.

We applied the standard WordPiece tokenizer tied to \texttt{bert-base-uncased}. To ensure we didn't lose the important disambiguation cues at the end of the story, we set the maximum sequence length to 512 tokens. The average length of the stories plus definitions rarely exceeds this limit, so there was minimum truncation , mainly affecting the example usage field rather than the narrative itself.

\subsection{Configuration and Hyperparameters}
All experiments were conducted using the PyTorch framework and the Hugging Face \texttt{transformers} library.

\begin{itemize}
    \item \textbf{Model Architecture:} We used \texttt{bert-base-uncased} with a regression head (\texttt{num\_labels=1}). The model trained to minimize the Mean Squared Error (MSE) loss between the predicted scalar and the human average rating.
    
    \item \textbf{Training Dynamics:} We applied the AdamW optimizer with a learning rate of $1\times10^{-5}$ and a linear decay schedule. Because of GPU memory limits, we set a per-device batch size of 8 with gradient accumulation steps at 2, giving us an effective batch size of 16. The model trained for 10 epochs with mixed-precision (FP16) enabled to speed up convergence.
    
    \item \textbf{Checkpoint Selection:} To avoid overfitting, we assessed the model on the development set at the end of each epoch. We saved the checkpoint that got the highest Spearman rank correlation, focusing on ranking quality instead of just MSE loss.
    
    \item \textbf{Calibration:} The linear calibration parameters ($\alpha, \beta$) were fitted using Ordinary Least Squares (OLS) regression on the development set predictions after the main training phase.
\end{itemize}


\subsection{Evaluation Metrics}
Following the official shared task guidelines, we report performance on two key metrics:
\begin{enumerate}
    \item \textbf{Spearman Rank Correlation ($\rho$):}This is our main measure of ranking quality, checks how well the model arranges the likelihood of different meanings compared to human intuition.
    \item \textbf{Accuracy within Standard Deviation:} This metric takes human subjectivity into account. A prediction is seen as correct if it falls within the range $[y_{human} - \sigma, y_{human} + \sigma]$, where $\sigma$ is the standard deviation of the human ratings for that specific sample.
\end{enumerate}


% \section{Results and Analysis}

% \subsection{Main Quantitative Findings}
% Our system, submitted under the team name \textbf{blue}, achieved a Spearman rank correlation coefficient of \textbf{0.66} on the development set for Task 5. This result confirms the effectiveness of framing Word Sense Disambiguation (WSD) as a graded regression task rather than a binary classification problem.

% \begin{table}[t]
% \centering
% % This command forces the table to fit the column width
% \resizebox{\columnwidth}{!}{%
% \begin{tabular}{llc}
% \hline
% \textbf{Model Configuration} & \textbf{Architecture} & \textbf{Score ($\rho$)} \\ \hline
% Baseline: Sentence-Transformer & Bi-Encoder & 0.52 \\
% Cross-Encoder (Initial) & BERT-Base & 0.58 \\
% \textbf{Cross-Encoder (Tuned)} & \textbf{BERT-Base} & \textbf{0.66} \\ \hline
% \end{tabular}%
% }
% \caption{\textbf{Ablation Study:} Performance comparison on the development split. The substantial improvement from 0.52 to 0.66 highlights the necessity of joint context-sense encoding.}
% \label{tab:ablation}
% \end{table}


% Table \ref{tab:ablation} presents a detailed ablation study, isolating the performance of each component in our pipeline. The results demonstrate a clear hierarchy of capability: standard bi-encoder models perform poorly due to information compression, while cross-encoder architectures provide a significant boost by enabling token-level interaction between the narrative and the candidate sense.

% \subsection{Quantitative Analysis}
% The most significant finding from our experiments is the performance gap between the Bi-Encoder Baseline and the Tuned Cross-Encoder (0.52 vs. 0.66). This margin suggests that approximately 14\% of the task's difficulty lies solely in the \textit{interaction bottleneck}. 

% When a model is forced to compress a 5-sentence story into a single vector (as Sentence-Transformers do), it loses the fine-grained signals required to distinguish between subtle ambiguity cues. In contrast, the Cross-Encoder allows the definition of the target sense to directly attend to the specific "trigger words" in the story ending, effectively verifying the sense against the narrative outcome.

% Furthermore, our experiments with calibration revealed that while raw BERT outputs correlate well with human rankings, they often suffer from scale compression (predicting values between 2.0--4.0). The application of linear calibration expanded this range to match the human 1.0--5.0 distribution, significantly improving the secondary metric, Accuracy-within-SD.

% \subsection{Error Analysis}
% To better understand our system's limitations, we manually inspected a sample of 50 pairs where our model predicted a plausibility score significantly divergent from the human average. We identified two primary error subtypes:

% \begin{enumerate}
%     \item \textbf{The ``Lexical Overlap'' Trap:} In several cases, the candidate sense definition contained keywords that also appeared in the story, but in a different context. Our system often marked these as highly plausible (scores $>$ 4.5) due to the surface-level lexical matching, failing to recognize that the actual semantic meaning was incongruent with the narrative flow.
    
%     \item \textbf{Underspecified Endings:} The system struggled with stories where the ending did not explicitly resolve the ambiguity (the ``None'' case). While human annotators typically assign low-confidence scores (approx. 2.5--3.0) to both senses in these scenarios, our model often hallucinated a resolution, confidently assigning a high score to one sense based on weak biases in the pre-context.
% \end{enumerate}
\section{Results and Analysis}

\subsection{Official Evaluation Results}
Our system was evaluated using the official SemEval-2026 Task 5 scoring script. On the hidden test set, our submission achieved a \textbf{Spearman rank correlation of 0.4866} ($p < 1.9\times10^{-56}$) and an \textbf{Accuracy-within-Standard-Deviation of 0.6613} (615/930 instances).

Spearman correlation measures the quality of plausibility ranking relative to human judgments, while Acc-within-SD evaluates whether predictions fall within the annotator disagreement range for each instance. The Acc-within-SD of 0.6613 indicates that the system's predictions align with the human consensus distribution for approximately 66\% of test samples, which is notably higher than the na\"ive majority-class rate, suggesting the model captures meaningful plausibility signals even where exact ranking is difficult.


\subsection{Development Set Performance and Ablation Study}

During development, we conducted systematic experiments to evaluate architectural choices and training strategies. Table~\ref{tab:ablation} summarizes the performance progression.

\begin{table}[t]
\centering
\small
\begin{tabular}{llc}
\hline
\textbf{Model Configuration} & \textbf{Architecture} & \textbf{Spearman ($\rho$)} \\ \hline
Sentence-Transformer & Bi-Encoder & 0.52 \\
Cross-Encoder (Initial) & BERT-Base & 0.58 \\
\textbf{Cross-Encoder (Tuned)} & \textbf{BERT-Base} & \textbf{0.66} \\
Cross-Encoder & BERT-Large & 0.64 \\ \hline
\end{tabular}
\caption{Development set performance comparison across model variants.}
\label{tab:ablation}
\end{table}
The results show that switching from a bi-encoder to a cross-encoder architecture significantly improves the performance. The Sentence-Transformer baseline had a Spearman correlation of 0.52, which means it was not effective at modelling the fine-grained interaction between the narrative context and the candidate senses. Replacing this with a BERT-based cross-encoder improved performance to 0.58, which shows how important it is to encode both context and sense together.

Further hyperparameter tuning, including gradient accumulation, mixed-precision training, and Spearman-based checkpointing, raised the development set performance to 0.66. We also tested BERT-Large, scoring 0.64. However, the tuned BERT-Base configuration provided a better balance between performance and efficiency.

\subsection{Test-Development Performance Gap}

The tuned cross-encoder achieved a Spearman correlation of 0.66 on the development set, while the official test set score of 0.4866 indicates a notable drop. This gap reflects moderate overfitting to the development distribution. To mitigate this, we employed Spearman-based checkpoint selection rather than minimising training loss directly, and experimented with increased dropout (0.2) and weight decay ($\lambda = 0.01$). This did not fully close the gap, suggesting that the drop may partly reflect distributional differences between the development and test narratives rather than overfitting alone.

It is worth noting that despite the drop in Spearman correlation, the system achieved a comparatively high Acc-within-SD of 0.6613. This suggests that while the exact ordering of senses may shift across annotation instances, the predicted plausibility scores remain broadly consistent with the human consensus distribution.distribution.

\subsection{Quantitative Insights}

The big difference in performance between the bi-encoder baseline (0.52) and the tuned cross-encoder (0.66) shows importance of model token-level interactions. Putting all five sentences into one embedding, like in sentence-transformer architectures, creates information bottlenecks that make it hard to see subtle cues that help you figure out what something means. The cross-encoder, on the contrary, lets the sense definition directly respond to certain lexical triggers and narrative resolutions.

Also, linear post-hoc calibration was helpful for matching raw regression outputs with the 1 to 5 rating scale used by people. Predictions tended to cluster in a narrow mid-range (2.0 to 4.0) without calibration. Calibration, on the other hand, spread out the distribution to better match how people score things, which improved the Accuracy-within-SD metric.


\section{Conclusion}

We presented a BERT-based cross-encoder regression system for SemEval-2026 Task 5, addressing the limitations of discrete WSD in modelling human plausibility judgments. On the development set, the tuned BERT-Base cross-encoder with linear calibration achieved a Spearman rank correlation of 0.66, substantially outperforming the sentence-transformer bi-encoder baseline ($\rho = 0.52$). On the official test set, the system achieved a Spearman correlation of 0.4866 and an Accuracy-within-SD of 0.6613. Our experiments demonstrate that for plausibility estimation, ``reading the ending'' necessitates joint attention between the narrative and the definition, rather than mere semantic similarity.

% \section*{Acknowledgments}
% We thank the organizers of SemEval-2026 for their valuable feedback and the provided AmbiStory dataset. We also acknowledge the computing resources provided by the Thapar Institute of Engineering and Technology.

\bibliographystyle{acl_natbib}
\bibliography{custom}



\end{document}

