SubmissionNumber#=%=#146
FinalPaperTitle#=%=#NootNoot At SemEval-2024 Task 6: Hallucinations and Related Observable Overgeneration Mistakes Detection
ShortPaperTitle#=%=#
NumberOfPages#=%=#5
CopyrightSigned#=%=#Sankalp Bahad
JobTitle#==#
Organization#==#
Abstract#==#Semantic hallucinations in neural language gen-
eration systems pose a significant challenge to
the reliability and accuracy of natural language
processing applications. Current neural mod-
els often produce fluent but incorrect outputs,
undermining the usefulness of generated text.
In this study, we address the task of detecting
semantic hallucinations through the SHROOM
(Semantic Hallucinations Real Or Mistakes)
dataset, encompassing data from diverse NLG
tasks such as definition modeling, machine
translation, and paraphrase generation. We in-
vestigate three methodologies: fine-tuning on
labelled training data, fine-tuning on labelled
validation data, and a zero-shot approach using
the Mixtral 8x7b instruct model. Our results
demonstrate the effectiveness of these method-
ologies in identifying semantic hallucinations,
with the zero-shot approach showing compet-
itive performance without additional training.
Our findings highlight the importance of robust
detection mechanisms for ensuring the accu-
racy and reliability of neural language genera-
tion systems.
Author{1}{Firstname}#=%=#Sankalp Sanjay
Author{1}{Lastname}#=%=#Bahad
Author{1}{Username}#=%=#sankalp_bahad
Author{1}{Email}#=%=#sankalp.bahad@research.iiit.ac.in
Author{1}{Affiliation}#=%=#IIIT Hyderabad
Author{2}{Firstname}#=%=#Yash
Author{2}{Lastname}#=%=#Bhaskar
Author{2}{Email}#=%=#yash.bhaskar@research.iiit.ac.in
Author{2}{Affiliation}#=%=#IIIT Hyderabad
Author{3}{Firstname}#=%=#Parameswari
Author{3}{Lastname}#=%=#Krishnamurthy
Author{3}{Username}#=%=#parameswari
Author{3}{Email}#=%=#parameshkrishnaa@gmail.com
Author{3}{Affiliation}#=%=#Assistant Professor, IIIT Hyderabad

==========
èéáğö