\subsection{LLM-judge to human annotation alignment}

\begin{table*}[t]
    \centering
    \input{latex/prelim}
    \label{results}
\end{table*}

% For inter-annotator agreement, at least two annotators agreed in 95.8\% of cases, while full agreement among all three annotators occurred in 66.3\% of cases. Table \ref{tab:preliminary_results} presents the accuracy of various LLM judges in aligning with human annotations across different evaluation axes. We also summarize the mean and standard deviation of the performance across the 5 LLMs. Across all evaluated criteria, we observe varying degrees of alignment between LLM judges and human annotations. Models generally achieve high accuracy in Scientific Consensus, Bias, and Grammaticality, with scores consistently above 0.90. However, performance drops significantly in Empathy, where all models struggle to align with human judgments, achieving an average accuracy of just 0.113. Missing Content and Extent of Harm also exhibit considerable variability, with standard deviations of 0.318 and 0.210, respectively, indicating greater disagreement between models and human annotators. Interestingly, while Likelihood of Harm maintains a relatively high mean accuracy (0.737), individual model scores vary widely, suggesting inconsistency in judgment. \textcolor{red}{Once the results are finalized, can you summarize surprising findings and/or areas where it is consistent with what was found in related work?}

For inter-annotator agreement, at least two annotators agreed in 95.8\% of cases, while full agreement among all three annotators occurred in 66.3\% of cases (see Appendix \ref{sec:appendix_B} for details). Table \ref{tab:preliminary_results} presents the accuracy of various LLM judges in aligning with human annotations across different evaluation axes. We also summarize the mean performance across the five LLMs.

Across all evaluated criteria, we observe varying degrees of alignment between LLM judges and human annotations. Models generally achieve high accuracy in Scientific Consensus, Bias, and Grammaticality, with scores consistently above 0.96. Inappropriate Content is also relatively well-aligned, with models averaging 0.95. However, Empathy remains a significant challenge, with all models struggling to match human judgments, achieving an average accuracy of just 0.49. Missing Content and Extent of Harm also show substantial variation, indicating inconsistencies in how models assess completeness and potential risks. Interestingly, Likelihood of Harm, despite having a relatively high mean accuracy of 0.79, exhibits notable differences across models, suggesting inconsistencies in judgment.

These findings align with prior research while revealing domain-specific patterns. The high alignment in factual assessments mirrors \citet{kanithi2024mediccomprehensiveframeworkevaluating}'s observations, while our lower alignment in Empathy extends \citet{szymanski2024limitationsllmasajudgeapproachevaluating}'s findings on LLM limitations in specialized contexts. Our results are positioned between their 60-64\% agreement in mental health domains and the 86\% found in general contexts \citep{zheng2023judgingllmasajudgemtbenchchatbot}, highlighting medical safety evaluation's unique challenges.

\subsection{Adversarial questions results}
Overall, all models performed well in classifying samples as "Opposed to Consensus," reinforcing confidence that models can accurately assess and differentiate between consensus-aligned and non-aligned responses. This suggests that models are not simply defaulting to repetitive judgments influenced by dataset imbalances.

\begin{figure}[t]
    \centering
    \includegraphics[width=0.48\textwidth]{latex/color_failed_example.png}
    \caption{Example of adversarial question failure.}
    \label{fig:fail_example}
\end{figure}

Figure~\ref{fig:fail_example} shows the only adversarial question where models failed. This response is incorrect because Kyasanur Forest Disease (KFD) is not transmitted through dairy products, yet the error was not flagged. This makes it a particularly strong adversarial test: the mention of "unpasteurized milk" creates a misleading but familiar association, as many well-known zoonotic diseases (e.g., brucellosis, Q fever) spread this way \citep{sebastianski2022disease}. 