SubmissionNumber#=%=#23
FinalPaperTitle#=%=#HalluSafe at SemEval-2024 Task 6: An NLI-based Approach to Make LLMs Safer by Better Detecting Hallucinations and Overgeneration Mistakes
ShortPaperTitle#=%=#
NumberOfPages#=%=#9
CopyrightSigned#=%=#Zahra Rahimi
JobTitle#==#
Organization#==#
Abstract#==#The advancement of large language models (LLMs), their ability to produce eloquent and fluent content, and their vast knowledge have resulted in their usage in various tasks and applications. Despite generating fluent content, this content can contain fabricated or false information. This problem is known as hallucination and has reduced the confidence in the output of LLMs. In this work, we have used Natural Language Inference to train classifiers for hallucination detection to tackle SemEval-2024 Task 6-SHROOM (Mickus et al., 2024) which is defined in three sub-tasks: Paraphrase Generation, Machine Translation, and Definition Modeling. We have also conducted experiments on LLMs to evaluate their ability to detect hallucinated outputs. We have achieved 75.93% and 78.33% accuracy for the modelaware and model-agnostic tracks, respectively. The shared links of our models and the codes are available on GitHub.
Author{1}{Firstname}#=%=#Zahra
Author{1}{Lastname}#=%=#Rahimi
Author{1}{Username}#=%=#zarahimi33
Author{1}{Email}#=%=#zarahimi@sharif.edu
Author{1}{Affiliation}#=%=#Sharif University of Technology
Author{2}{Firstname}#=%=#Hamidreza
Author{2}{Lastname}#=%=#Amirzadeh
Author{2}{Username}#=%=#hamiiidreza
Author{2}{Email}#=%=#hamid.amirzadeh78@sharif.edu
Author{2}{Affiliation}#=%=#Sharif University of Technology
Author{3}{Firstname}#=%=#Alireza
Author{3}{Lastname}#=%=#Sohrabi
Author{3}{Username}#=%=#sohraabi
Author{3}{Email}#=%=#lirezasohrabi@gmail.com
Author{3}{Affiliation}#=%=#Sharif University of Technology
Author{4}{Firstname}#=%=#Zeinab Sadat
Author{4}{Lastname}#=%=#Taghavi
Author{4}{Username}#=%=#zeinabtaghavi
Author{4}{Email}#=%=#zeinabtaghavi1377@gmail.com
Author{4}{Affiliation}#=%=#MSc student
Author{5}{Firstname}#=%=#Hossein
Author{5}{Lastname}#=%=#Sameti
Author{5}{Username}#=%=#sameti
Author{5}{Email}#=%=#sameti@sharif.edu
Author{5}{Affiliation}#=%=#Sharif University of Technology

==========
èéáğö