Re-Examining FactBank: Predicting the Author’s Presentation of Factuality

John Murzaku, Peter Zeng, Magdalena Markowska, Owen Rambow


Abstract
We present a corrected version of a subset of the FactBank data set. Previously published results on FactBank are no longer valid. We perform experiments on FactBank using multiple training paradigms, data smoothing techniques, and polarity classifiers. We argue that f-measure is an important alternative evaluation metric for factuality. We provide new state-of-the-art results for four corpora including FactBank. We perform an error analysis on Factbank combined with two similar corpora.
Anthology ID:
2022.coling-1.66
Volume:
Proceedings of the 29th International Conference on Computational Linguistics
Month:
October
Year:
2022
Address:
Gyeongju, Republic of Korea
Editors:
Nicoletta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin-Hsi Chen, Lucia Donatelli, Heng Ji, Sadao Kurohashi, Patrizia Paggio, Nianwen Xue, Seokhwan Kim, Younggyun Hahm, Zhong He, Tony Kyungil Lee, Enrico Santus, Francis Bond, Seung-Hoon Na
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
786–796
Language:
URL:
https://aclanthology.org/2022.coling-1.66
DOI:
Bibkey:
Cite (ACL):
John Murzaku, Peter Zeng, Magdalena Markowska, and Owen Rambow. 2022. Re-Examining FactBank: Predicting the Author’s Presentation of Factuality. In Proceedings of the 29th International Conference on Computational Linguistics, pages 786–796, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Cite (Informal):
Re-Examining FactBank: Predicting the Author’s Presentation of Factuality (Murzaku et al., COLING 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/naacl-24-ws-corrections/2022.coling-1.66.pdf
Data
MegaVeridicality