Abstract
In this paper, we present our solution to the TextGraphs-17 Shared Task on Text-Graph Representations for KGQA. GPT-4 alone, with chain-of-thought reasoning and a given set of answers, achieves an F1 score of 0.78. By employing subgraph size as a feature, Wikidata answer description as an additional context, and question rephrasing technique, we further strengthen this result. These tricks help to answer questions that were not initially answered and to eliminate irrelevant, identical answers. We have managed to achieve an F1 score of 0.83 and took 2nd place, improving the score by 0.05 over the baseline. An open implementation of our method is available on GitHub.- Anthology ID:
- 2024.textgraphs-1.14
- Volume:
- Proceedings of TextGraphs-17: Graph-based Methods for Natural Language Processing
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Dmitry Ustalov, Yanjun Gao, Alexander Panchenko, Elena Tutubalina, Irina Nikishina, Arti Ramesh, Andrey Sakhovskiy, Ricardo Usbeck, Gerald Penn, Marco Valentino
- Venues:
- TextGraphs | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 149–153
- Language:
- URL:
- https://aclanthology.org/2024.textgraphs-1.14
- DOI:
- Cite (ACL):
- Maria Lysyuk and Pavel Braslavski. 2024. Skoltech at TextGraphs-17 Shared Task: Finding GPT-4 Prompting Strategies for Multiple Choice Questions. In Proceedings of TextGraphs-17: Graph-based Methods for Natural Language Processing, pages 149–153, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- Skoltech at TextGraphs-17 Shared Task: Finding GPT-4 Prompting Strategies for Multiple Choice Questions (Lysyuk & Braslavski, TextGraphs-WS 2024)
- PDF:
- https://preview.aclanthology.org/dois-2013-emnlp/2024.textgraphs-1.14.pdf