AIT_FHSTP at GermEval 2021: Automatic Fact Claiming Detection with Multilingual Transformer Models
Jaqueline Böck, Daria Liakhovets, Mina Schütz, Armin Kirchknopf, Djordje Slijepčević, Matthias Zeppelzauer, Alexander Schindler
Abstract
Spreading ones opinion on the internet is becoming more and more important. A problem is that in many discussions people often argue with supposed facts. This year’s GermEval 2021 focuses on this topic by incorporating a shared task on the identification of fact-claiming comments. This paper presents the contribution of the AIT FHSTP team at the GermEval 2021 benchmark for task 3: “identifying fact-claiming comments in social media texts”. Our methodological approaches are based on transformers and incorporate 3 different models: multilingual BERT, GottBERT and XML-RoBERTa. To solve the fact claiming task, we fine-tuned these transformers with external data and the data provided by the GermEval task organizers. Our multilingual BERT model achieved a precision-score of 72.71%, a recall of 72.96% and an F1-Score of 72.84% on the GermEval test set. Our fine-tuned XML-RoBERTa model achieved a precision-score of 68.45%, a recall of 70.11% and a F1-Score of 69.27%. Our best model is GottBERT (i.e., a BERT transformer pre-trained on German texts) fine-tuned on the GermEval 2021 data. This transformer achieved a precision of 74.13%, a recall of 75.11% and an F1-Score of 74.62% on the test set.- Anthology ID:
- 2021.germeval-1.11
- Volume:
- Proceedings of the GermEval 2021 Shared Task on the Identification of Toxic, Engaging, and Fact-Claiming Comments
- Month:
- September
- Year:
- 2021
- Address:
- Duesseldorf, Germany
- Editors:
- Julian Risch, Anke Stoll, Lena Wilms, Michael Wiegand
- Venue:
- GermEval
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 76–82
- Language:
- URL:
- https://aclanthology.org/2021.germeval-1.11
- DOI:
- Cite (ACL):
- Jaqueline Böck, Daria Liakhovets, Mina Schütz, Armin Kirchknopf, Djordje Slijepčević, Matthias Zeppelzauer, and Alexander Schindler. 2021. AIT_FHSTP at GermEval 2021: Automatic Fact Claiming Detection with Multilingual Transformer Models. In Proceedings of the GermEval 2021 Shared Task on the Identification of Toxic, Engaging, and Fact-Claiming Comments, pages 76–82, Duesseldorf, Germany. Association for Computational Linguistics.
- Cite (Informal):
- AIT_FHSTP at GermEval 2021: Automatic Fact Claiming Detection with Multilingual Transformer Models (Böck et al., GermEval 2021)
- PDF:
- https://preview.aclanthology.org/teach-a-man-to-fish/2021.germeval-1.11.pdf
- Data
- ClaimBuster