Abstract
This work describes an approach to develop Knowledge Graph Question Answering (KGQA) system for TextGraphs-17 shared task. The task focuses on the fusion of Large Language Models (LLMs) with Knowledge Graphs (KGs). The goal is to select a KG entity (out of several candidates) which corresponds to an answer given a textual question. Our approach applies LLM to identify the correct answer among the list of possible candidates. We confirm that integrating external information is particularly beneficial when the subject entities are not well-known, and using RAG can negatively impact the performance of LLM on questions related to popular entities, as the retrieved context might be misleading. With our result, we achieved 2nd place in the post-evaluation phase.- Anthology ID:
- 2024.textgraphs-1.15
- Volume:
- Proceedings of TextGraphs-17: Graph-based Methods for Natural Language Processing
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Dmitry Ustalov, Yanjun Gao, Alexander Panchenko, Elena Tutubalina, Irina Nikishina, Arti Ramesh, Andrey Sakhovskiy, Ricardo Usbeck, Gerald Penn, Marco Valentino
- Venues:
- TextGraphs | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 154–160
- Language:
- URL:
- https://aclanthology.org/2024.textgraphs-1.15
- DOI:
- Cite (ACL):
- Julia Belikova, Evegeniy Beliakin, and Vasily Konovalov. 2024. JellyBell at TextGraphs-17 Shared Task: Fusing Large Language Models with External Knowledge for Enhanced Question Answering. In Proceedings of TextGraphs-17: Graph-based Methods for Natural Language Processing, pages 154–160, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- JellyBell at TextGraphs-17 Shared Task: Fusing Large Language Models with External Knowledge for Enhanced Question Answering (Belikova et al., TextGraphs-WS 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.textgraphs-1.15.pdf