@inproceedings{gogoulou-etal-2025-llms,
    title = "Can {LLM}s Detect Intrinsic Hallucinations in Paraphrasing and Machine Translation?",
    author = {Gogoulou, Evangelia  and
      Zahra, Shorouq  and
      Guillou, Liane  and
      D{\"u}rlich, Luise  and
      Nivre, Joakim},
    editor = "Dhole, Kaustubh  and
      Clinciu, Miruna",
    booktitle = "Proceedings of the Fourth Workshop on Generation, Evaluation and Metrics (GEM{\texttwosuperior})",
    month = jul,
    year = "2025",
    address = "Vienna, Austria and virtual meeting",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/corrections-2025-08/2025.gem-1.13/",
    pages = "161--177",
    ISBN = "979-8-89176-261-9",
    abstract = "A frequently observed problem with LLMs is their tendency to generate output that is nonsensical, illogical, or factually incorrect, often referred to broadly as ``hallucination''. Building on the recently proposed HalluciGen task for hallucination detection and generation, we evaluate a suite of open-access LLMs on their ability to detect intrinsic hallucinations in two conditional generation tasks: translation and paraphrasing. We study how model performance varies across tasks and languages and we investigate the impact of model size, instruction-tuning, and prompt choice. We find that performance varies across models but is consistent across prompts. Finally, we find that NLI models perform comparably well, suggesting that LLM-based detectors are not the only viable option for this specific task."
}