SciBERT Meets Contrastive Learning: A Solution for Scientific Hallucination Detection

Crivoi Carla, Ana Sabina Uban


Abstract
As AI systems become more involved in scientific research, there is growing concern about the accuracy of their outputs. Tools powered by large language models can generate summaries and answers that appear well-formed, but sometimes include claims that are not actually supported by the cited references. In this paper, we focus on identifying these hallucinated claims. We propose a system built on SciBERT and contrastive learning to detect whether a scientific claim can be inferred from the referenced content. Our method was evaluated in the SciHal 2025 shared task, which includes both coarse and fine-grained hallucination labels. The results show that our model performs well on supported and clearly unsupported claims, but struggles with ambiguous or low-resource categories. These findings highlight both the promise and the limitations of current models in improving the trustworthiness of AI-generated scientific content.
Anthology ID:
2025.sdp-1.32
Volume:
Proceedings of the Fifth Workshop on Scholarly Document Processing (SDP 2025)
Month:
July
Year:
2025
Address:
Vienna, Austria
Editors:
Tirthankar Ghosal, Philipp Mayr, Amanpreet Singh, Aakanksha Naik, Georg Rehm, Dayne Freitag, Dan Li, Sonja Schimmler, Anita De Waard
Venues:
sdp | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
336–343
Language:
URL:
https://preview.aclanthology.org/landing_page/2025.sdp-1.32/
DOI:
10.18653/v1/2025.sdp-1.32
Bibkey:
Cite (ACL):
Crivoi Carla and Ana Sabina Uban. 2025. SciBERT Meets Contrastive Learning: A Solution for Scientific Hallucination Detection. In Proceedings of the Fifth Workshop on Scholarly Document Processing (SDP 2025), pages 336–343, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
SciBERT Meets Contrastive Learning: A Solution for Scientific Hallucination Detection (Carla & Uban, sdp 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/landing_page/2025.sdp-1.32.pdf