Valerii Olisov
2025
SmurfCat at SemEval-2025 Task 3: Bridging External Knowledge and Model Uncertainty for Enhanced Hallucination Detection
Elisei Rykov
|
Valerii Olisov
|
Maksim Savkin
|
Artem Vazhentsev
|
Kseniia Titova
|
Alexander Panchenko
|
Vasily Konovalov
|
Julia Belikova
Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)
The Multilingual shared-task on Hallucinations and Related Observable Overgeneration Mistakes in the SemEval-2025 competition aims to detect hallucination spans in the outputs of instruction-tuned LLMs in a multilingual context. In this paper, we address the detection of span hallucinations by applying an ensemble of approaches. In particular, we synthesized a PsiloQA dataset and fine-tuned LLM to detect hallucination spans. In addition, we combined this approach with a white-box method based on uncertainty quantification techniques. Using our combined pipeline, we achieved 3rd place in detecting span hallucinations in Arabic, Catalan, Finnish, Italian, and ranked within the top ten for the rest of the languages.
Search
Fix author
Co-authors
- Julia Belikova 1
- Vasily Konovalov 1
- Alexander Panchenko 1
- Elisei Rykov 1
- Maksim Savkin 1
- show all...