Complex Logical Query Answering by Calibrating Knowledge Graph Completion Models

Changyi Xiao, Yixin Cao


Abstract
Complex logical query answering (CLQA) is a challenging task that involves finding answer entities for complex logical queries over incomplete knowledge graphs (KGs). Previous research has explored the use of pre-trained knowledge graph completion (KGC) models, which can predict the missing facts in KGs, to answer complex logical queries. However, KGC models are typically evaluated using ranking evaluation metrics, which may result in values of predictions of KGC models that are not well-calibrated. In this paper, we propose a method for calibrating KGC models, namely CKGC, which enables KGC models to adapt to answering complex logical queries. Notably, CKGC is lightweight and effective. The adaptation function is simple, allowing the model to quickly converge during the adaptation process. The core concept of CKGC is to map the values of predictions of KGC models to the range [0, 1], ensuring that values associated with true facts are close to 1, while values linked to false facts are close to 0. Through experiments on three benchmark datasets, we demonstrate that our proposed calibration method can significantly boost model performance in the CLQA task. Moreover, our approach can enhance the performance of CLQA while preserving the ranking evaluation metrics of KGC models. The code is available at https://github.com/changyi7231/CKGC.
Anthology ID:
2024.findings-acl.819
Volume:
Findings of the Association for Computational Linguistics ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand and virtual meeting
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
13792–13803
Language:
URL:
https://aclanthology.org/2024.findings-acl.819
DOI:
10.18653/v1/2024.findings-acl.819
Bibkey:
Cite (ACL):
Changyi Xiao and Yixin Cao. 2024. Complex Logical Query Answering by Calibrating Knowledge Graph Completion Models. In Findings of the Association for Computational Linguistics ACL 2024, pages 13792–13803, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
Complex Logical Query Answering by Calibrating Knowledge Graph Completion Models (Xiao & Cao, Findings 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-5/2024.findings-acl.819.pdf