Predicting Document Coverage for Relation Extraction

Sneha Singhania, Simon Razniewski, Gerhard Weikum


Abstract
This paper presents a new task of predicting the coverage of a text document for relation extraction (RE): Does the document contain many relational tuples for a given entity? Coverage predictions are useful in selecting the best documents for knowledge base construction with large input corpora. To study this problem, we present a dataset of 31,366 diverse documents for 520 entities. We analyze the correlation of document coverage with features like length, entity mention frequency, Alexa rank, language complexity, and information retrieval scores. Each of these features has only moderate predictive power. We employ methods combining features with statistical models like TF-IDF and language models like BERT. The model combining features and BERT, HERB, achieves an F1 score of up to 46%. We demonstrate the utility of coverage predictions on two use cases: KB construction and claim refutation.
Anthology ID:
2022.tacl-1.12
Volume:
Transactions of the Association for Computational Linguistics, Volume 10
Month:
Year:
2022
Address:
Cambridge, MA
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
207–223
Language:
URL:
https://aclanthology.org/2022.tacl-1.12
DOI:
10.1162/tacl_a_00456
Bibkey:
Cite (ACL):
Sneha Singhania, Simon Razniewski, and Gerhard Weikum. 2022. Predicting Document Coverage for Relation Extraction. Transactions of the Association for Computational Linguistics, 10:207–223.
Cite (Informal):
Predicting Document Coverage for Relation Extraction (Singhania et al., TACL 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/emnlp-22-attachments/2022.tacl-1.12.pdf
Video:
 https://preview.aclanthology.org/emnlp-22-attachments/2022.tacl-1.12.mp4