Abstract
Automatic construction of large knowledge graphs (KG) by mining web-scale text datasets has received considerable attention recently. Estimating accuracy of such automatically constructed KGs is a challenging problem due to their size and diversity. This important problem has largely been ignored in prior research – we fill this gap and propose KGEval. KGEval uses coupling constraints to bind facts and crowdsources those few that can infer large parts of the graph. We demonstrate that the objective optimized by KGEval is submodular and NP-hard, allowing guarantees for our approximation algorithm. Through experiments on real-world datasets, we demonstrate that KGEval best estimates KG accuracy compared to other baselines, while requiring significantly lesser number of human evaluations.- Anthology ID:
- D17-1183
- Volume:
- Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
- Month:
- September
- Year:
- 2017
- Address:
- Copenhagen, Denmark
- Venue:
- EMNLP
- SIG:
- SIGDAT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1741–1750
- Language:
- URL:
- https://aclanthology.org/D17-1183
- DOI:
- 10.18653/v1/D17-1183
- Cite (ACL):
- Prakhar Ojha and Partha Talukdar. 2017. KGEval: Accuracy Estimation of Automatically Constructed Knowledge Graphs. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1741–1750, Copenhagen, Denmark. Association for Computational Linguistics.
- Cite (Informal):
- KGEval: Accuracy Estimation of Automatically Constructed Knowledge Graphs (Ojha & Talukdar, EMNLP 2017)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/D17-1183.pdf