SubmissionNumber#=%=#101
FinalPaperTitle#=%=#UMUTeam at SemEval-2024 Task 6: Leveraging Zero-Shot Learning for Detecting Hallucinations and Related Observable Overgeneration Mistakes
ShortPaperTitle#=%=#
NumberOfPages#=%=#7
CopyrightSigned#=%=#José Antonio García Díaz
JobTitle#==#
Organization#==#Universidad de Murcia
Abstract#==#In these working notes we describe the UMUTeam's participation in SemEval-2024 shared task 6, which aims at detecting grammatically correct output of Natural Language Generation with incorrect semantic information in two different setups: model-aware and model-agnostic tracks. The task is consists of three subtasks with different model setups. Our approach is based on exploiting the zero-shot classification capability of the Large Language Models LLaMa-2, Tulu and Mistral, through prompt engineering. Our system ranked eighteenth in the model-aware setup with an accuracy of 78.4% and 29th in the model-agnostic setup with an accuracy of 76.9333%.
Author{1}{Firstname}#=%=#ronghao
Author{1}{Lastname}#=%=#pan
Author{1}{Username}#=%=#ronghaopan
Author{1}{Email}#=%=#ronghao.pan@um.es
Author{1}{Affiliation}#=%=#Universidad de Murcia
Author{2}{Firstname}#=%=#José Antonio
Author{2}{Lastname}#=%=#García-Díaz
Author{2}{Username}#=%=#joseagd
Author{2}{Email}#=%=#joseantonio.garcia8@um.es
Author{2}{Affiliation}#=%=#Universidad de Murcia
Author{3}{Firstname}#=%=#Tomás
Author{3}{Lastname}#=%=#Bernal-Beltrán
Author{3}{Email}#=%=#tomas.bernalb@um.es
Author{3}{Affiliation}#=%=#Universidad de Murcia
Author{4}{Firstname}#=%=#Rafael
Author{4}{Lastname}#=%=#Valencia-García
Author{4}{Username}#=%=#valencia
Author{4}{Email}#=%=#valencia@um.es
Author{4}{Affiliation}#=%=#Universidad de Murcia

==========
èéáğö