Retrieval-Augmented Knowledge Integration into Language Models: A Survey
Yuxuan Chen, Daniel Röder, Justus-Jonas Erker, Leonhard Hennig, Philippe Thomas, Sebastian Möller, Roland Roller
Abstract
This survey analyses how external knowledge can be integrated into language models in the context of retrieval-augmentation.The main goal of this work is to give an overview of: (1) Which external knowledge can be augmented? (2) Given a knowledge source, how to retrieve from it and then integrate the retrieved knowledge? To achieve this, we define and give a mathematical formulation of retrieval-augmented knowledge integration (RAKI). We discuss retrieval and integration techniques separately in detail, for each of the following knowledge formats: knowledge graph, tabular and natural language.- Anthology ID:
- 2024.knowllm-1.5
- Volume:
- Proceedings of the 1st Workshop on Towards Knowledgeable Language Models (KnowLLM 2024)
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Sha Li, Manling Li, Michael JQ Zhang, Eunsol Choi, Mor Geva, Peter Hase, Heng Ji
- Venues:
- KnowLLM | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 45–63
- Language:
- URL:
- https://aclanthology.org/2024.knowllm-1.5
- DOI:
- Cite (ACL):
- Yuxuan Chen, Daniel Röder, Justus-Jonas Erker, Leonhard Hennig, Philippe Thomas, Sebastian Möller, and Roland Roller. 2024. Retrieval-Augmented Knowledge Integration into Language Models: A Survey. In Proceedings of the 1st Workshop on Towards Knowledgeable Language Models (KnowLLM 2024), pages 45–63, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- Retrieval-Augmented Knowledge Integration into Language Models: A Survey (Chen et al., KnowLLM-WS 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.knowllm-1.5.pdf