SubmissionNumber#=%=#53
FinalPaperTitle#=%=#Halu-NLP at SemEval-2024 Task 6: MetaCheckGPT - A Multi-task Hallucination Detection using LLM uncertainty and meta-models
ShortPaperTitle#=%=#
NumberOfPages#=%=#7
CopyrightSigned#=%=#Rahul mehta
JobTitle#==#
Organization#==#
Abstract#==#Hallucinations in large language models
(LLMs) have recently become a significant
problem. A recent effort in this direction
is a shared task at Semeval 2024 Task 6,
SHROOM, a Shared-task on Hallucinations
and Related Observable Overgeneration Mis-
takes. This paper describes our winning so-
lution ranked 1st and 2nd in the 2 sub-tasks
of model agnostic and model aware tracks re-
spectively. We propose a meta-regressor based
ensemble of LLMs based on a random forest
algorithm that achieves the highest scores on
the leader board. We also experiment with var-
ious transformer based models and black box
methods like ChatGPT, Vectara, and others. In
addition, we perform an error analysis com-
paring ChatGPT against our best model which
shows the limitations of the former
Author{1}{Firstname}#=%=#Rahul
Author{1}{Lastname}#=%=#Mehta
Author{1}{Username}#=%=#novice8
Author{1}{Email}#=%=#rahul.mehta@research.iiit.ac.in
Author{1}{Affiliation}#=%=#IIIT Hyderabad
Author{2}{Firstname}#=%=#Andrew
Author{2}{Lastname}#=%=#Hoblitzell
Author{2}{Username}#=%=#ahoblitz
Author{2}{Email}#=%=#ahoblitz@gmail.com
Author{2}{Affiliation}#=%=#Purdue University
Author{3}{Firstname}#=%=#Jack
Author{3}{Lastname}#=%=#O'Keefe
Author{3}{Email}#=%=#jackokeefe2024@u.northwestern.edu
Author{3}{Affiliation}#=%=#Northwestern University
Author{4}{Firstname}#=%=#Hyeju
Author{4}{Lastname}#=%=#Jang
Author{4}{Email}#=%=#hyejuj@iu.edu
Author{4}{Affiliation}#=%=#Indiana University
Author{5}{Firstname}#=%=#Vasudeva
Author{5}{Lastname}#=%=#Varma
Author{5}{Email}#=%=#vv@iiit.ac.in
Author{5}{Affiliation}#=%=#International Institute of Information Technology - Hyderabad

==========
èéáğö