SubmissionNumber#=%=#89
FinalPaperTitle#=%=#IRIT-Berger-Levrault at SemEval-2024: How Sensitive Sentence Embeddings are to Hallucinations?
ShortPaperTitle#=%=#
NumberOfPages#=%=#6
CopyrightSigned#=%=#Nihed Bendahman
JobTitle#==#
Organization#==#
Abstract#==#This article presents our participation to Task 6 of SemEval-2024, named SHROOM (a Shared-task on Hallucinations and Related Observable Overgeneration Mistakes), which aims at detecting hallucinations. We propose two types of approaches for the task: the first one is based on sentence embeddings and cosine similarity metric, and the second one uses LLMs (Large Language Model). We found that LLMs fail to improve the performance achieved by embedding generation models. The latter outperform the baseline provided by the organizers, and our best system achieves 78% accuracy.
Author{1}{Firstname}#=%=#Nihed
Author{1}{Lastname}#=%=#Bendahman
Author{1}{Username}#=%=#nihed_bendahman
Author{1}{Email}#=%=#nihed.bendahman@irit.fr
Author{1}{Affiliation}#=%=#IRIT/Berger-Levrault
Author{2}{Firstname}#=%=#Karen
Author{2}{Lastname}#=%=#Pinel-Sauvagnat
Author{2}{Username}#=%=#ksauvagnat
Author{2}{Email}#=%=#karen.sauvagnat@irit.fr
Author{2}{Affiliation}#=%=#IRIT
Author{3}{Firstname}#=%=#Gilles
Author{3}{Lastname}#=%=#Hubert
Author{3}{Username}#=%=#gilleshub
Author{3}{Email}#=%=#hubert@irit.fr
Author{3}{Affiliation}#=%=#IRIT
Author{4}{Firstname}#=%=#Mokhtar
Author{4}{Lastname}#=%=#BILLAMI
Author{4}{Username}#=%=#mokhtarlive
Author{4}{Email}#=%=#mokhtarlive@live.fr
Author{4}{Affiliation}#=%=#Informatique

==========
èéáğö