2021
pdf
abs
Evaluation Datasets for Cross-lingual Semantic Textual Similarity
Tomáš Hercig
|
Pavel Kral
Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)
Semantic textual similarity (STS) systems estimate the degree of the meaning similarity between two sentences. Cross-lingual STS systems estimate the degree of the meaning similarity between two sentences, each in a different language. State-of-the-art algorithms usually employ a strongly supervised, resource-rich approach difficult to use for poorly-resourced languages. However, any approach needs to have evaluation data to confirm the results. In order to simplify the evaluation process for poorly-resourced languages (in terms of STS evaluation datasets), we present new datasets for cross-lingual and monolingual STS for languages without this evaluation data. We also present the results of several state-of-the-art methods on these data which can be used as a baseline for further research. We believe that this article will not only extend the current STS research to other languages, but will also encourage competition on this new evaluation data.
2020
pdf
abs
UWB@FinTOC-2020 Shared Task: Financial Document Title Detection
Tomáš Hercig
|
Pavel Kral
Proceedings of the 1st Joint Workshop on Financial Narrative Processing and MultiLing Financial Summarisation
This paper describes our system created for the Financial Document Structure Extraction Shared Task (FinTOC-2020): Title Detection. We rely on the Apache PDFBox library to extract text and all additional information e.g. font type and font size from the financial prospectuses. Our constrained system uses only the provided training data without any additional external resources. Our system is based on the Maximum Entropy classifier and various features including font type and font size. Our system achieves F1 score 81% and #1 place in the French track and F1 score 77% and #2 place among 5 participating teams in the English track.
2019
pdf
abs
Machine Learning Approach to Fact-Checking in West Slavic Languages
Pavel Přibáň
|
Tomáš Hercig
|
Josef Steinberger
Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019)
Fake news detection and closely-related fact-checking have recently attracted a lot of attention. Automatization of these tasks has been already studied for English. For other languages, only a few studies can be found (e.g. (Baly et al., 2018)), and to the best of our knowledge, no research has been conducted for West Slavic languages. In this paper, we present datasets for Czech, Polish, and Slovak. We also ran initial experiments which set a baseline for further research into this area.
2018
pdf
abs
UWB at SemEval-2018 Task 1: Emotion Intensity Detection in Tweets
Pavel Přibáň
|
Tomáš Hercig
|
Ladislav Lenc
Proceedings of the 12th International Workshop on Semantic Evaluation
This paper describes our system created for the SemEval-2018 Task 1: Affect in Tweets (AIT-2018). We participated in both the regression and the ordinal classification subtasks for emotion intensity detection in English, Arabic, and Spanish. For the regression subtask we use the AffectiveTweets system with added features using various word embeddings, lexicons, and LDA. For the ordinal classification we additionally use our Brainy system with features using parse tree, POS tags, and morphological features. The most beneficial features apart from word and character n-grams include word embeddings, POS count and morphological features.
pdf
abs
UWB at SemEval-2018 Task 3: Irony detection in English tweets
Tomáš Hercig
Proceedings of the 12th International Workshop on Semantic Evaluation
This paper describes our system created for the SemEval-2018 Task 3: Irony detection in English tweets. Our strongly constrained system uses only the provided training data without any additional external resources. Our system is based on Maximum Entropy classifier and various features using parse tree, POS tags, and morphological features. Even without additional lexicons and word embeddings we achieved fourth place in Subtask A and seventh in Subtask B in terms of accuracy.
pdf
abs
UWB at SemEval-2018 Task 10: Capturing Discriminative Attributes from Word Distributions
Tomáš Brychcín
|
Tomáš Hercig
|
Josef Steinberger
|
Michal Konkol
Proceedings of the 12th International Workshop on Semantic Evaluation
We present our UWB system for the task of capturing discriminative attributes at SemEval 2018. Given two words and an attribute, the system decides, whether this attribute is discriminative between the words or not. Assuming Distributional Hypothesis, i.e., a word meaning is related to the distribution across contexts, we introduce several approaches to compare word contextual information. We experiment with state-of-the-art semantic spaces and with simple co-occurrence statistics. We show the word distribution in the corpus has potential for detecting discriminative attributes. Our system achieves F1 score 72.1% and is ranked #4 among 26 submitted systems.
2017
pdf
abs
Geographical Evaluation of Word Embeddings
Michal Konkol
|
Tomáš Brychcín
|
Michal Nykl
|
Tomáš Hercig
Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Word embeddings are commonly compared either with human-annotated word similarities or through improvements in natural language processing tasks. We propose a novel principle which compares the information from word embeddings with reality. We implement this principle by comparing the information in the word embeddings with geographical positions of cities. Our evaluation linearly transforms the semantic space to optimally fit the real positions of cities and measures the deviation between the position given by word embeddings and the real position. A set of well-known word embeddings with state-of-the-art results were evaluated. We also introduce a visualization that helps with error analysis.
pdf
abs
The Impact of Figurative Language on Sentiment Analysis
Tomáš Hercig
|
Ladislav Lenc
Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017
Figurative language such as irony, sarcasm, and metaphor is considered a significant challenge in sentiment analysis. These figurative devices can sculpt the affect of an utterance and test the limits of sentiment analysis of supposedly literal texts. We explore the effect of figurative language on sentiment analysis. We incorporate the figurative language indicators into the sentiment analysis process and compare the results with and without the additional information about them. We evaluate on the SemEval-2015 Task 11 data and outperform the first team with our convolutional neural network model and additional training data in terms of mean squared error and we follow closely behind the first place in terms of cosine similarity.
pdf
abs
Cross-lingual Flames Detection in News Discussions
Josef Steinberger
|
Tomáš Brychcín
|
Tomáš Hercig
|
Peter Krejzl
Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017
We introduce Flames Detector, an online system for measuring flames, i.e. strong negative feelings or emotions, insults or other verbal offences, in news commentaries across five languages. It is designed to assist journalists, public institutions or discussion moderators to detect news topics which evoke wrangles. We propose a machine learning approach to flames detection and calculate an aggregated score for a set of comment threads. The demo application shows the most flaming topics of the current period in several language variants. The search functionality gives a possibility to measure flames in any topic specified by a query. The evaluation shows that the flame detection in discussions is a difficult task, however, the application can already reveal interesting information about the actual news discussions.
2016
pdf
UWB at SemEval-2016 Task 5: Aspect Based Sentiment Analysis
Tomáš Hercig
|
Tomáš Brychcín
|
Lukáš Svoboda
|
Michal Konkol
Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016)