Joint Knowledge Base Completion and Question Answering by Combining Large Language Models and Small Language Models
Yinan Liu, Dongying Lin, Sigang Luo, Xiaochun Yang, Bin Wang
Abstract
Knowledge Bases (KBs) play a key role in various applications. As two representative KB-related tasks, knowledge base completion (KBC) and knowledge base question answering (KBQA) are closely related and inherently complementary with each other. Thus, it will be beneficial to solve the task of joint KBC and KBQA to make them reinforce each other. However, existing studies usually rely on the small language model (SLM) to enhance them jointly, and the large language model (LLM)’s strong reasoning ability is ignored. In this paper, by combining the strengths of the LLM with the SLM, we propose a novel framework JCQL, which can make these two tasks enhance each other in an iterative manner. To make KBC enhance KBQA, we augment the LLM agent-based KBQA model’s reasoning paths by incorporating an SLM-trained KBC model as an action of the agent, alleviating the LLM’s hallucination and high computational costs issue in KBQA. To make KBQA enhance KBC, we incrementally fine-tune the KBC model by leveraging KBQA’s reasoning paths as its supplementary training data, improving the ability of the SLM in KBC. Extensive experiments over two public benchmark data sets demonstrate that JCQL surpasses all baselines for both KBC and KBQA tasks.- Anthology ID:
- 2026.acl-long.1704
- Volume:
- Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Month:
- July
- Year:
- 2026
- Address:
- San Diego, California, United States
- Editors:
- Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 36756–36775
- Language:
- URL:
- https://preview.aclanthology.org/ingest-acl/2026.acl-long.1704/
- DOI:
- Cite (ACL):
- Yinan Liu, Dongying Lin, Sigang Luo, Xiaochun Yang, and Bin Wang. 2026. Joint Knowledge Base Completion and Question Answering by Combining Large Language Models and Small Language Models. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 36756–36775, San Diego, California, United States. Association for Computational Linguistics.
- Cite (Informal):
- Joint Knowledge Base Completion and Question Answering by Combining Large Language Models and Small Language Models (Liu et al., ACL 2026)
- PDF:
- https://preview.aclanthology.org/ingest-acl/2026.acl-long.1704.pdf