@inproceedings{obiso-etal-2024-harmonee,
    title = "{H}a{RM}o{NEE} at {S}em{E}val-2024 Task 6: Tuning-based Approaches to Hallucination Recognition",
    author = "Obiso, Timothy  and
      Tu, Jingxuan  and
      Pustejovsky, James",
    editor = {Ojha, Atul Kr.  and
      Do{\u{g}}ru{\"o}z, A. Seza  and
      Tayyar Madabushi, Harish  and
      Da San Martino, Giovanni  and
      Rosenthal, Sara  and
      Ros{\'a}, Aiala},
    booktitle = "Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)",
    month = jun,
    year = "2024",
    address = "Mexico City, Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/Add-Cong-Liu-Florida-Atlantic-University-author-id/2024.semeval-1.191/",
    doi = "10.18653/v1/2024.semeval-1.191",
    pages = "1322--1331",
    abstract = "This paper presents the Hallucination Recognition Model for New Experiment Evaluation (HaRMoNEE) team`s winning ({\#}1) and {\#}10 submissions for SemEval-2024 Task 6: Shared- task on Hallucinations and Related Observable Overgeneration Mistakes (SHROOM)`s two subtasks. This task challenged its participants to design systems to detect hallucinations in Large Language Model (LLM) outputs. Team HaRMoNEE proposes two architectures: (1) fine-tuning an off-the-shelf transformer-based model and (2) prompt tuning large-scale Large Language Models (LLMs). One submission from the fine-tuning approach outperformed all other submissions for the model-aware subtask; one submission from the prompt-tuning approach is the 10th-best submission on the leaderboard for the model-agnostic subtask. Our systems also include pre-processing, system-specific tuning, post-processing, and evaluation."
}