
The prompt templates used to obtain the final leaderboard results for SOLAR are shown in tables \ref{tab:eval_prompt} and
\ref{tab:summary_prompt}.

We started with a single prompt template for all sections and summaries. When the results did not further improve,
we analyzed the F1-score of each section, shown in Tables \ref{tab:init_results_train} and \ref{tab:init_results_dev}.
Due to time constraints, we only create
summaries for the first 50 examples in the train set.

% aici nu sunt sigur, a fost oarecum intenționat din lipsă de timp
While the single CTR summaries for the results section depend on the hypothesis, due to an implementation choice, % defect (?),
all examples use the same CTR summary, regardless of the hypothesis (only the first hypothesis is used). We believe
that this issue is not essential, since all the hypotheses for a CTR focus on the same information.

Initial summary prompt: ``Instruction: You are given a clinical trial report. You must summarize the report. Use abbreviations. Be brief. Report: \{premise\}. Summary (max 350 words):''.

Initial evaluation prompt: ``Instruction: You are given a Clinical Trial Report and a hypothesis.
\#\#Report: \{premise\}. \#\#Hypothesis: \{hypothesis\}. \#\#Can the hypothesis be inferred from the report? Respond only with Yes or No. \#\#Response (Yes or No):''. This prompt was also used for our experiments with LLaMa-2 and Mistral.

% TODO: de actualizat cu noile rezultate, rerulez script-ul cu un prompt inițial și apoi cu prompt-urile din paper - complet pt dev și train (first 50) să fiu sigur că cifrele de mai jos se reproduc (ish) și să justifice de ce am umblat mai mult doar la unele prompts;
% pt test sper să fie suficient doar rerulat evaluarea, fără rezumate

\begin{table}[h]
    \centering
    \begin{tabular}{lr}
        Section                     & F1     \\
        \toprule
        Eligibility (Single)        & 0.6637 \\ % mail: 0.5050
        Eligibility (Comparison)    & 0.5274 \\ % mail: 0.5508
        Intervention (Single)       & 0.7306 \\ % mail: 0.7342
        Intervention (Comparison)   & 0.7751 \\ % mail: 0.7887
        Results (Single)            & 0.7141 \\ % mail: 0.7550
        Results (Comparison)        & 0.7525 \\ % mail: 0.6991
        Adverse events (Single)     & 0.7141 \\ % mail: 0.7141
        Adverse events (Comparison) & 0.6805 \\ % mail: 0.7177
    \end{tabular}
    \caption{Initial results for the train set (first 50 examples)}
    \label{tab:init_results_train}
\end{table}
% total: 0.7040

\begin{table}[ht]
    \centering
    \begin{tabular}{lr}
        Section                     & F1     \\
        \toprule
        Eligibility (Single)        & 0.8178 \\ % mail: 0.6757; nu e chiar ok, a dat prea bine... 0.7953 cu fewshot
        Eligibility (Comparison)    & 0.8285 \\ % mail: 0.5804; 0.9160 cu fewshot
        Intervention (Single)       & 0.7678 \\ % mail: 0.8461; 0.8461 cu fewshot pe init prompt
        Intervention (Comparison)   & 0.6000 \\ % mail: 0.6969; 0.8 cu fewshot
        Results (Single)            & 0.7368 \\ % mail: 0.8421; 0.7103 cu fewshot
        Results (Comparison)        & 0.7749 \\ % mail: 0.7777; 0.8328 cu fewshot
        Adverse events (Single)     & 0.5835 \\ % mail: 0.6862; 0.6190 cu fewshot
        Adverse events (Comparison) & 0.6969 \\ % mail: 0.6969; 0.8496 cu fewshot
    \end{tabular}
    \caption{Initial results for the development set}
    \label{tab:init_results_dev}
\end{table}
% inițial rezumatele erau ~128 tokens și făcute cu modelul q3, probabil (și) din cauza asta discrepanța la scor
% eventual nu mai includem tabelele init_results_train și init_results_dev
%
% total fără fewshot: 0.7346, ceea ce reproduce rezultatele inițiale unde aveam F1 ~73-74
% problema a fost că am plecat cu prompts de la 0 când am făcut prompts pe secțiuni
% total cu fewshot: 0.7748, deci saltul cel mai mare pare că a fost din faptul că am făcut fewshot, nu din prompts

\begin{table*}[htbp]
    \centering
    \begin{tabular}{lp{10.59cm}}
        Section (CTR type)              & Prompt \\
        \toprule
        Eligibility (Single)        & {Instruction: You are given clinical trial criteria and a statement that may or may not be contradictory. Regarding the inclusion and exclusion criteria, is the statement correct? Respond only with Yes or No.\newline\#\# Criteria: \{premise\}.\newline\#\# Statement: \{hypothesis\}.\newline\#\# Response (Yes or No):} \\
        \midrule
        Eligibility (Comparison)    & {Instruction: You are given clinical trial criteria for a primary and a secondary trial, and a statement. Regarding the inclusion and exclusion criteria, is the statement correct for each trial? Respond only with Yes or No.\newline\#\# Criteria: \{premise\}.\newline\#\# Statement: \{hypothesis\}.\newline\#\# Response (Yes or No):} \\
        \midrule
        Intervention (Single)       & {Instruction: You are given a CTR and a statement. Can the statement be deduced from the CTR? Focus on the interventions. Respond only with Yes or No.\newline\#\#CTR: \{premise\}.\newline\#\#Statement: \{hypothesis\}.\newline\#\#Response (Yes or No):} \\
        \midrule
        Intervention (Comparison)   & (same prompt template as single CTR for interventions) \\
        \midrule
        Results (Single)            & {Instruction: You are given the results of a CTR and a statement. Can the statement be deduced from the CTR in terms of number of participants, measures and results? Respond only with Yes or No.\newline\#\#CTR: \{premise\}.\newline\#\#Statement: \{hypothesis\}.\newline\#\#Response (Yes or No):} \\
        \midrule
        Results (Comparison)        & {Instruction: You are given the results of a CTR and a statement. Can the statement be deduced from the CTR? Respond only with Yes or No.\newline\#\#CTR: \{premise\}.\newline\#\#Statement: \{hypothesis\}.\newline\#\#Response (Yes or No):} \\
        \midrule
        Adverse events (Single)     & {Instruction: You are given a CTR and a statement. Regarding the adverse events, signs and symptoms observed in the CTR, is the statement correct? Respond only with Yes or No.\newline\#\#CTR: \{premise\}.\newline\#\#Statement: \{hypothesis\}.\newline\#\#Response (Yes or No):} \\
        \midrule
        Adverse events (Comparison) & {(same prompt template as single CTR for adverse events)}
    \end{tabular}
    \caption{Evaluation templates for each CTR section}
    \label{tab:eval_prompt}
\end{table*}


\begin{table*}[htbp]
    \centering
    \begin{tabular}{lp{10.59cm}}
        Section (CTR type)              & Prompt \\
        \toprule
        Eligibility (Single)        & {Instruction: You are given the eligibility criteria for a clinical trial report. You must summarize the report focusing on inclusion and exclusion criteria.\newline Report: \{premise\}.\newline Use short sentences. Summary: } \\
        \midrule
        Eligibility (Comparison)    & {(same prompt template as single CTR for eligibility)} \\
        \midrule
        Intervention (Single)       & {Instruction: You are given the intervention information for a clinical trial report. Each report contains 1-2 cohorts, which receive different treatments, or have different characteristics. You must summarize the report focusing on the type, dosage, frequency, and duration of treatments being studied.\newline Report: \{premise\}.\newline Use short sentences to group by cohort. Summary:} \\
        \midrule
        Intervention (Comparison)   & {Instruction: You are given the intervention information for a clinical trial report. Each report contains 1-2 cohorts, which receive different treatments, or have different characteristics. You must summarize the report focusing on the type, dosage, frequency, and duration of treatments being studied.\newline Report: \{premise\}.\newline Use short sentences to group by cohort and group by primary trial and secondary trial. Summary:} \\
        \midrule
        Results (Single)            & {Instruction: You are given the results of a CTR and a statement. Extract all the relevant information from the CTR that is related to the statement.\newline Report: \{premise\}. Statement: \{hypothesis\}.\newline Answer:} \\
        \midrule
        Results (Comparison)        & {Instruction: You are given the results of two clinical trials. Each trial contains 1-2 cohorts, which receive different treatments, or have different characteristics. You must summarize the report for each trial, focusing on number of participants, outcome measures, units, results.\newline Report: \{premise\}.\newline Use short sentences and keep all numeric values. Summary:} \\
        \midrule
        Adverse events (Single)     & {Instruction: You are given the adverse events of a clinical trial report. Each report contains 1-2 cohorts, which receive different treatments, or have different characteristics. You must summarize the report focusing on the adverse events, signs and symptoms observed in patients.\newline Report: \{premise\}.\newline Use short sentences. Summary:} \\
        \midrule
        Adverse events (Comparison) & {Instruction: You are given the adverse events of a clinical trial report. Each report contains 1-2 cohorts, which receive different treatments, or have different characteristics. You must summarize the report focusing on the adverse events, signs and symptoms observed in patients.\newline Report: \{premise\}.\newline Use short sentences and group by primary trial and secondary trial. Summary:}
    \end{tabular}
    \caption{Summarization templates for each CTR section. The results section (single CTR) is the only one for which summaries depend on the hypothesis due to lack of time to rerun the summaries for the test set.}
    \label{tab:summary_prompt}
\end{table*}
