Abstract
We describe our submission for the Breaker phase of the second Fact Extraction and VERification (FEVER) Shared Task. Our adversarial data can be explained by two perspectives. First, we aimed at testing model’s ability to retrieve evidence, when appropriate query terms could not be easily generated from the claim. Second, we test model’s ability to precisely understand the implications of the texts, which we expect to be rare in FEVER 1.0 dataset. Overall, we suggested six types of adversarial attacks. The evaluation on the submitted systems showed that the systems were only able get both the evidence and label correct in 20% of the data. We also demonstrate our adversarial run analysis in the data development process.- Anthology ID:
- D19-6615
- Volume:
- Proceedings of the Second Workshop on Fact Extraction and VERification (FEVER)
- Month:
- November
- Year:
- 2019
- Address:
- Hong Kong, China
- Editors:
- James Thorne, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulos, Arpit Mittal
- Venue:
- WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 99–104
- Language:
- URL:
- https://aclanthology.org/D19-6615
- DOI:
- 10.18653/v1/D19-6615
- Cite (ACL):
- Youngwoo Kim and James Allan. 2019. FEVER Breaker’s Run of Team NbAuzDrLqg. In Proceedings of the Second Workshop on Fact Extraction and VERification (FEVER), pages 99–104, Hong Kong, China. Association for Computational Linguistics.
- Cite (Informal):
- FEVER Breaker’s Run of Team NbAuzDrLqg (Kim & Allan, 2019)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-5/D19-6615.pdf