% The best performing systems were ....

% cel mai important mi se pare faptul că modelul pare să prezică mai mult contradicții deoarece informația nu e repetată cuvânt cu cuvânt => trebuie dezvoltată în continuare evaluarea pe reformulări pt că acum modelul pare să fenteze evaluarea

In this paper, we presented our approach for the SemEval 2024 Task 2 \citep{jullien-etal-2024-semeval} aimed at understanding large language models behavior in clinical contexts. We explored several types of models and prompting techniques in order to determine whether fine-tuning is more feasible than zero-shot or few-shot prompting in a limited resource setting.

Our findings suggest that, while LLMs exhibit remarkable clinical NLI capabilities
% even without explicit medical pre-training or fine-tuning
at a surface level, the proposed metrics and interventions uncover a tendency of the models to take shortcuts
and rely on simple heuristics, especially when faced with semantic-preserving changes.
We intend to investigate further methods of evaluating the reliability of large language models in future work. 

To address the inherent weak numerical reasoning of our model (and all generative models), a promising strategy is to offload complex mathematical hypotheses to a specialized model like xVal \cite{Golkar}. This approach involves representing numbers as individual digits (e.g., 123 becomes ["1", "2", "3"]), replacing them with a generic [NUM] token, and scaling the token according to the original numerical value. Their results showed a 70-fold improvement over standard models. We can extend this strategy to create an ensemble where other weaknesses of our model (like rephrases) are offloaded to specialized models. 


% - efortul computațional este mult mai mare pentru a îmbunătăți un prompt generic comparativ cu utilizarea de prompts specializate; considerăm că este o abordare necesară deoarece conținutul diferă destul de mult între secțiuni



% Ideas for improvements could include ...

% pt ce ziceam cu ce exemple ar trebui folosite, s-a făcut și se cheamă "demonstration selection" - find the best examples for few-shot prompting (a.k.a. in-context learning)
% https://arxiv.org/pdf/2301.00234.pdf
