SubmissionNumber#=%=#194
FinalPaperTitle#=%=#Lisbon Computational Linguists at SemEval-2024 Task 2: Using a Mistral-7B Model and Data Augmentation
ShortPaperTitle#=%=#
NumberOfPages#=%=#8
CopyrightSigned#=%=#Artur Guimarães
JobTitle#==#
Organization#==#INESC-ID, Rua Alves Redol, nº9 1000-029, Lisboa, Portugal
Abstract#==#ABSTRACT: This paper describes our approach to the SemEval-2024 safe biomedical Natural Language Inference for Clinical Trials (NLI4CT) task, which concerns classifying statements about Clinical Trial Reports (CTRs). We explored the capabilities of Mistral-7B, a generalistic open-source Large Language Model (LLM). 

   We developed a prompt for the NLI4CT task, and fine-tuning a quantized version of the model using a slightly augmented version of the training dataset. The experimental results show that this approach can produce notable results in terms of the macro F1-score, while having limitations in terms of faithfulness and consistency. All the developed code is publicly available on a GitHub repository.
Author{1}{Firstname}#=%=#Artur RAA
Author{1}{Lastname}#=%=#Guimarães
Author{1}{Username}#=%=#araag2
Author{1}{Email}#=%=#artur.guimas@gmail.com
Author{1}{Affiliation}#=%=#IST
Author{2}{Firstname}#=%=#Bruno
Author{2}{Lastname}#=%=#Martins
Author{2}{Email}#=%=#bgmartins@gmail.com
Author{2}{Affiliation}#=%=#INESC-ID and IST, University of Lisbon
Author{3}{Firstname}#=%=#João
Author{3}{Lastname}#=%=#Magalhães
Author{3}{Email}#=%=#jmag@fct.unl.pt
Author{3}{Affiliation}#=%=#NOVA-LINCS and NOVA, University of Lisbon

==========
èéáğö