2025
pdf
bib
abs
When the Dictionary Strikes Back: A Case Study on Slovak Migration Location Term Extraction and NER via Rule-Based vs. LLM Methods
Miroslav Blšták
|
Jaroslav Kopčan
|
Marek Suppa
|
Samuel Havran
|
Andrej Findor
|
Martin Takac
|
Marian Simko
Proceedings of the 10th Workshop on Slavic Natural Language Processing (Slavic NLP 2025)
This study explores the task of automatically extracting migration-related locations (source and destination) from media articles, focusing on the challenges posed by Slovak, a low-resource and morphologically complex language. We present the first comparative analysis of rule-based dictionary approaches (NLP4SK) versus Large Language Models (LLMs, e.g. SlovakBERT, GPT-4o) for both geographical relevance classification (Slovakia-focused migration) and specific source/target location extraction. To facilitate this research and future work, we introduce the first manually annotated Slovak dataset tailored for migration-focused locality detection. Our results show that while a fine-tuned SlovakBERT model achieves high accuracy for classification, specialized rule-based methods still have the potential to outperform LLMs for specific extraction tasks, though improved LLM performance with few-shot examples suggests future competitiveness as research in this area continues to evolve.
pdf
bib
abs
o-MEGA: Optimized Methods for Explanation Generation and Analysis
Ľuboš Kriš
|
Jaroslav Kopčan
|
Qiwei Peng
|
Andrej Ridzik
|
Marcel Veselý
|
Martin Tamajka
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
The proliferation of transformer-based language models has revolutionized NLP domain while simultaneously introduced significant challenges regarding model transparency and trustworthiness. The complexity of achieving explainable systems in this domain is evidenced by the extensive array of explanation methods and evaluation metrics developed by researchers. To address the challenge of selecting optimal explainability approaches, we present o-mega, a hyperparameter optimization tool designed to automatically identify the most effective explainable AI methods and their configurations within the semantic matching domain. We evaluate o-mega on a post-claim matching pipeline using a curated dataset of social media posts paired with refuting claims. Our tool systematically explores different explainable methods and their hyperparameters, demonstrating improved transparency in automated fact-checking systems. As a result, such automated optimization of explanation methods can significantly enhance the interpretability of claim-matching models in critical applications such as misinformation detection, contributing to more trustworthy and transparent AI systems.
pdf
bib
abs
SemEval-2025 Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrieval
Qiwei Peng
|
Robert Moro
|
Michal Gregor
|
Ivan Srba
|
Simon Ostermann
|
Marian Simko
|
Juraj Podrouzek
|
Matúš Mesarčík
|
Jaroslav Kopčan
|
Anders Søgaard
Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)
The rapid spread of online disinformation presents a global challenge, and machine learning has been widely explored as a potential solution. However, multilingual settings and low-resource languages are often neglected in this field. To address this gap, we conducted a shared task on multilingual claim retrieval at SemEval 2025, aimed at identifying fact-checked claims that match newly encountered claims expressed in social media posts across different languages. The task includes two subtracks: 1) a monolingual track, where social posts and claims are in the same language 2) a crosslingual track, where social posts and claims might be in different languages. A total of 179 participants registered for the task contributing to 52 test submissions. 23 out of 31 teams have submitted their system papers. In this paper, we report the best-performing systems as well as the most common and the most effective approaches across both subtracks. This shared task, along with its dataset and participating systems, provides valuable insights into multilingual claim retrieval and automated fact-checking, supporting future research in this field.