From Query to Logic: Ontology-Driven Multi-Hop Reasoning in LLMs

Haonan Bian, Yutao Qi, Rui Yang, Yuanxi Che, Jiaqian Wang, Heming Xia, Ranran Zhen


Abstract
Large Language Models (LLMs), despite their success in question answering, exhibit limitations in complex multi-hop question answering (MQA) tasks that necessitate non-linear, structured reasoning. This limitation stems from their inability to adequately capture deep conceptual relationships between entities. To overcome this challenge, we present ORACLE (Ontology-driven Reasoning And Chain for Logical Elucidation), a training-free framework that combines LLMs’ generative capabilities with the structural benefits of knowledge graphs. Our approach operates through three stages: (1) dynamic construction of question-specific knowledge ontologies using LLMs, (2) transformation of these ontologies into First-Order Logic (FOL) reasoning chains, and (3) systematic decomposition of the original query into logically coherent sub-questions. Extensive experiments across a diverse set of models and standard MQA benchmarks demonstrate that our framework achieves competitive performance while producing more interpretable reasoning chains.
Anthology ID:
2026.findings-acl.687
Volume:
Findings of the Association for Computational Linguistics: ACL 2026
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
14035–14051
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.687/
DOI:
Bibkey:
Cite (ACL):
Haonan Bian, Yutao Qi, Rui Yang, Yuanxi Che, Jiaqian Wang, Heming Xia, and Ranran Zhen. 2026. From Query to Logic: Ontology-Driven Multi-Hop Reasoning in LLMs. In Findings of the Association for Computational Linguistics: ACL 2026, pages 14035–14051, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
From Query to Logic: Ontology-Driven Multi-Hop Reasoning in LLMs (Bian et al., Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.687.pdf
Checklist:
 2026.findings-acl.687.checklist.pdf