Clinical trials are prospective studies that aim to compare the effectiveness of an intervention against a control group in clinical patients \cite{friedman2015fundamentals}. ClinicalTrials.gov\footnote{\href{https://clinicaltrials.gov/}{https://clinicaltrials.gov/}} hosts more than 480,000 clinical trials, making it challenging to analyze and extract information from them manually. Natural language inference has emerged as a valuable tool for interpreting evidence from clinical trials \cite{jullien-etal-2023-nli4ct}.

% Clinical trials are prospective studies that aim to compare the effectiveness of an intervention (therapeutic drugs, vaccines, procedures, devices, etc.) against a control group in clinical patients \cite{friedman2015fundamentals}. These trials are important in determining which interventions are safe and effective, the incidence of adverse effects, and in developing personalized medicine practices. ClinicalTrials.gov\footnote{\href{https://clinicaltrials.gov/}{https://clinicaltrials.gov/}} hosts more than 480,000 clinical trials, making it challenging to analyze and extract information from them manually. Additionally, clinical contradictions are often found in medical corpora due to a large amount of newly developed studies \cite{makhervaks2023clinical}. To address these issues, various computational approaches have been proposed to automatically extract information \cite{kiritchenko2010exact} and to interpret evidence from clinical trials using natural language inference \cite{jullien-etal-2023-nli4ct}. 

The second task of SemEval 2024 focuses on improving the understanding of clinical trial data through the second edition of NLI4CT (Natural Language Inference for Clinical Trials) \cite{jullien-etal-2024-semeval}. This challenge is specifically designed to test the natural language inference capabilities of large language models (LLMs) and their ability to understand clinical text. The data used for the challenge was carefully annotated by clinical domain experts, and semantic interventions were performed to evaluate the safety and robustness of the models. 
%Submitted systems are evaluated using metrics designed to measure the robustness of their predictions, including consistency and faithfulness, in addition to the standard F1-score.

We employed LLMs, achieving the best results with SOLAR-Instruct, and focused on two key strategies:
\begin{itemize}
   \item  \textbf{Targeted Summarization}: Summarizing both CTRs and the hypothesis (retaining only the first 'trial' sentence) aided the model's focus on essential information.
   \item  \textbf{Tailored Prompting}: We used both zero-shot and two-shot prompting, tailoring prompts to individual CTR sections for optimal results.
\end{itemize}

We were surprised to find that causal models significantly outperform masked language models on this type of task. This is probably because the task requires reasoning capabilities that BERT-based models do not have. Our model's biggest challenges were with numerical reasoning and rephrasing (discussed in Section \ref{sec:discussion}), but despite those, we still secured the 14th place (out of 32) in the leaderboard. We make our code publicly available on GitHub\footnote{\url{https://github.com/ClaudiuCreanga/sem-eval-2024-task-2}}.


