Zero-shot Graph Reasoning via Retrieval Augmented Framework with LLMs

Hanqing Li, Sharika Mahadevan, Kiran Jyothi Sheena, Henry Liang, Diego Klabjan


Abstract
We propose a new, training-free method, Graph Reasoning via Retrieval Augmented Framework (GRRAF), that harnesses retrieval-augmented generation (RAG) alongside the code-generation capabilities of large language models (LLMs) to address a wide range of graph reasoning tasks. In GRRAF, the target graph is stored in a graph database, and the LLM is prompted to generate executable code queries that retrieve the necessary information. This approach circumvents the limitations of existing methods that require extensive finetuning or depend on predefined algorithms, and it incorporates an error feedback loop with a time-out mechanism to ensure both correctness and efficiency. Experimental evaluations on the GraphInstruct dataset reveal that GRRAF achieves 100% accuracy on most graph reasoning tasks, including cycle detection, bipartite graph checks, shortest path computation, and maximum flow, while maintaining consistent token costs regardless of graph sizes. Imperfect but still very high performance is observed on subgraph matching. Notably, GRRAF scales effectively to large graphs with up to 10,000 nodes.
Anthology ID:
2025.findings-emnlp.924
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2025
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, Violet Peng
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
17041–17054
Language:
URL:
https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.924/
DOI:
10.18653/v1/2025.findings-emnlp.924
Bibkey:
Cite (ACL):
Hanqing Li, Sharika Mahadevan, Kiran Jyothi Sheena, Henry Liang, and Diego Klabjan. 2025. Zero-shot Graph Reasoning via Retrieval Augmented Framework with LLMs. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 17041–17054, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
Zero-shot Graph Reasoning via Retrieval Augmented Framework with LLMs (Li et al., Findings 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.924.pdf
Checklist:
 2025.findings-emnlp.924.checklist.pdf