SubmissionNumber#=%=#257
FinalPaperTitle#=%=#Maha Bhaashya at SemEval-2024 Task 6: Zero-Shot Multi-task Hallucination Detection
ShortPaperTitle#=%=#
NumberOfPages#=%=#5
CopyrightSigned#=%=#Patanjali Bhamidipati
JobTitle#==#
Organization#==#IIIT Hyderabad
Abstract#==#In recent studies, the extensive utilization of
large language models has underscored the importance of robust evaluation methodologies
for assessing text generation quality and relevance to specific tasks. This has revealed
a prevalent issue known as hallucination, an
emergent condition in the model where generated text lacks faithfulness to the source and
deviates from the evaluation criteria. In this
study, we formally define hallucination and propose a framework for its quantitative detection
in a zero-shot setting, leveraging our definition
and the assumption that model outputs entail
task and sample specific inputs. In detecting
hallucinations, our solution achieves an accuracy of 0.78 in a model-aware setting and 0.61
in a model-agnostic setting. Notably, our solution maintains computational efficiency, requiring far less computational resources than other SOTA approaches, aligning with the trend
towards lightweight and compressed models.
Author{1}{Firstname}#=%=#Patanjali
Author{1}{Lastname}#=%=#Bhamidipati
Author{1}{Username}#=%=#patanjali-b
Author{1}{Email}#=%=#patanjali.b@research.iiit.ac.in
Author{1}{Affiliation}#=%=#International Institute of Information Technology Hyderabad
Author{2}{Firstname}#=%=#Advaith
Author{2}{Lastname}#=%=#Malladi
Author{2}{Username}#=%=#advaithmalladi
Author{2}{Email}#=%=#advaith.malladi@research.iiit.ac.in
Author{2}{Affiliation}#=%=#International Institute of Information Technology, Hyderabad
Author{3}{Firstname}#=%=#Manish
Author{3}{Lastname}#=%=#Shrivastava
Author{3}{Username}#=%=#manshri
Author{3}{Email}#=%=#m.shrivastava@iiit.ac.in
Author{3}{Affiliation}#=%=#International Institute of Information Technology Hyderabad
Author{4}{Firstname}#=%=#Radhika
Author{4}{Lastname}#=%=#Mamidi
Author{4}{Username}#=%=#radhika
Author{4}{Email}#=%=#radhika.mamidi@iiit.ac.in
Author{4}{Affiliation}#=%=#Language Technologies Research Centre, IIIT Hyderabad

==========
èéáğö