SubmissionNumber#=%=#162
FinalPaperTitle#=%=#DUTh at SemEval-2024 Task 6: Comparing Pre-trained Models on Sentence Similarity Evaluation for Detecting of Hallucinations and Related Observable Overgeneration Mistakes
ShortPaperTitle#=%=#
NumberOfPages#=%=#7
CopyrightSigned#=%=#Ioannis Maslaris
JobTitle#==#
Organization#==#Database & Information Retrieval research unit,
Department of Electrical & Computer Engineering,
Democritus University of Thrace, Greece
Abstract#==#In this paper, we present our approach to
SemEval-2024 Task 6: SHROOM, a Sharedtask on Hallucinations and Related Observable
Overgeneration Mistakes, which aims to determine weather AI generated text is semantically
correct or incorrect. This work is a comparative study of Large Language Models (LLMs)
in the context of the task, shedding light on
their effectiveness and nuances. We present a
system that leverages pre-trained LLMs, such
as LaBSE, T5, and DistilUSE, for binary classification of given sentences into ‘Hallucination'
or ‘Not Hallucination' classes by evaluating
the model's output against the reference correct text. Moreover, beyond utilizing labeled
datasets, our methodology integrates synthetic
label creation in unlabeled datasets, followed
by the prediction of test labels.
Author{1}{Firstname}#=%=#Ioanna
Author{1}{Lastname}#=%=#Iordanidou
Author{1}{Username}#=%=#ioannaior
Author{1}{Email}#=%=#ioaniord1@ee.duth.gr
Author{1}{Affiliation}#=%=#Democritus University of Thrace
Author{2}{Firstname}#=%=#Ioannis
Author{2}{Lastname}#=%=#Maslaris
Author{2}{Username}#=%=#yms1
Author{2}{Email}#=%=#yannismslr@gmail.com
Author{2}{Affiliation}#=%=#Democritus University of Thrace
Author{3}{Firstname}#=%=#Avi
Author{3}{Lastname}#=%=#Arampatzis
Author{3}{Username}#=%=#avi1
Author{3}{Email}#=%=#yannis.maslatis@gmail.com
Author{3}{Affiliation}#=%=#Democritus University of Thrace

==========
èéáğö