ParsFEVER: a Dataset for Farsi Fact Extraction and Verification
Majid Zarharan, Mahsa Ghaderan, Amin Pourdabiri, Zahra Sayedi, Behrouz Minaei-Bidgoli, Sauleh Eetemadi, Mohammad Taher Pilehvar
Abstract
Training and evaluation of automatic fact extraction and verification techniques require large amounts of annotated data which might not be available for low-resource languages. This paper presents ParsFEVER: the first publicly available Farsi dataset for fact extraction and verification. We adopt the construction procedure of the standard English dataset for the task, i.e., FEVER, and improve it for the case of low-resource languages. Specifically, claims are extracted from sentences that are carefully selected to be more informative. The dataset comprises nearly 23K manually-annotated claims. Over 65% of the claims in ParsFEVER are many-hop (require evidence from multiple sources), making the dataset a challenging benchmark (only 13% of the claims in FEVER are many-hop). Also, despite having a smaller training set (around one-ninth of that in Fever), a model trained on ParsFEVER attains similar downstream performance, indicating the quality of the dataset. We release the dataset and the annotation guidelines at https://github.com/Zarharan/ParsFEVER.- Anthology ID:
- 2021.starsem-1.9
- Volume:
- Proceedings of *SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics
- Month:
- August
- Year:
- 2021
- Address:
- Online
- Editors:
- Lun-Wei Ku, Vivi Nastase, Ivan Vulić
- Venue:
- *SEM
- SIG:
- SIGLEX
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 99–104
- Language:
- URL:
- https://aclanthology.org/2021.starsem-1.9
- DOI:
- 10.18653/v1/2021.starsem-1.9
- Cite (ACL):
- Majid Zarharan, Mahsa Ghaderan, Amin Pourdabiri, Zahra Sayedi, Behrouz Minaei-Bidgoli, Sauleh Eetemadi, and Mohammad Taher Pilehvar. 2021. ParsFEVER: a Dataset for Farsi Fact Extraction and Verification. In Proceedings of *SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics, pages 99–104, Online. Association for Computational Linguistics.
- Cite (Informal):
- ParsFEVER: a Dataset for Farsi Fact Extraction and Verification (Zarharan et al., *SEM 2021)
- PDF:
- https://preview.aclanthology.org/naacl24-info/2021.starsem-1.9.pdf
- Code
- zarharan/parsfever
- Data
- FEVER, HoVer, LIAR