Abstract
This study evaluates the performance of four information extraction tools (extractors) on identifying health claims in health news headlines. A health claim is defined as a triplet: IV (what is being manipulated), DV (what is being measured) and their relation. Tools that can identify health claims provide the foundation for evaluating the accuracy of these claims against authoritative resources. The evaluation result shows that 26% headlines do not in-clude health claims, and all extractors face difficulty separating them from the rest. For those with health claims, OPENIE-5.0 performed the best with F-measure at 0.6 level for ex-tracting “IV-relation-DV”. However, the characteristic linguistic structures in health news headlines, such as incomplete sentences and non-verb relations, pose particular challenge to existing tools.- Anthology ID:
- W18-4305
- Volume:
- Proceedings of the Workshop Events and Stories in the News 2018
- Month:
- August
- Year:
- 2018
- Address:
- Santa Fe, New Mexico, U.S.A
- Editors:
- Tommaso Caselli, Ben Miller, Marieke van Erp, Piek Vossen, Martha Palmer, Eduard Hovy, Teruko Mitamura, David Caswell, Susan W. Brown, Claire Bonial
- Venue:
- EventStory
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 34–43
- Language:
- URL:
- https://aclanthology.org/W18-4305
- DOI:
- Cite (ACL):
- Shi Yuan and Bei Yu. 2018. An Evaluation of Information Extraction Tools for Identifying Health Claims in News Headlines. In Proceedings of the Workshop Events and Stories in the News 2018, pages 34–43, Santa Fe, New Mexico, U.S.A. Association for Computational Linguistics.
- Cite (Informal):
- An Evaluation of Information Extraction Tools for Identifying Health Claims in News Headlines (Yuan & Yu, EventStory 2018)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/W18-4305.pdf