Tucker Decomposition with Frequency Attention for Temporal Knowledge Graph Completion

Likang Xiao, Richong Zhang, Zijie Chen, Junfan Chen

Abstract
Temporal Knowledge Graph Completion aims to complete missing entities or relations under temporal constraints. Previous tensor decomposition-based models for TKGC only independently consider the combination of one single relation with one single timestamp, ignoring the global nature of the embedding. We propose a Frequency Attention (FA) model to capture the global temporal dependencies between one relation and the entire timestamp. Specifically, we use Discrete Cosine Transform (DCT) to capture the frequency of the timestamp embedding and further compute the frequency attention weight to scale embedding. Meanwhile, the previous temporal tucker decomposition method uses a simple norm regularization to constrain the core tensor, which limits the optimization performance. Thus, we propose Orthogonal Regularization (OR) variants for the core tensor, which can limit the non-superdiagonal elements of the 3-rd core tensor. Experiments on three standard TKGC datasets demonstrate that our method outperforms the state-of-the-art results on several metrics. The results suggest that the direct-current component is not the best feature for TKG representation learning. Additional analysis shows the effectiveness of our FA and OR models, even with smaller embedding dimensions.
Anthology ID:
2023.findings-acl.458
Volume:
Findings of the Association for Computational Linguistics: ACL 2023
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7286–7300
Language:
URL:
https://aclanthology.org/2023.findings-acl.458
DOI:
10.18653/v1/2023.findings-acl.458
Bibkey:
Cite (ACL):
Likang Xiao, Richong Zhang, Zijie Chen, and Junfan Chen. 2023. Tucker Decomposition with Frequency Attention for Temporal Knowledge Graph Completion. In Findings of the Association for Computational Linguistics: ACL 2023, pages 7286–7300, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Tucker Decomposition with Frequency Attention for Temporal Knowledge Graph Completion (Xiao et al., Findings 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/dois-2013-emnlp/2023.findings-acl.458.pdf
PDF Search