From Query to Logic: Ontology-Driven Multi-Hop Reasoning in LLMs
Haonan Bian, Yutao Qi, Rui Yang, Yuanxi Che, Jiaqian Wang, Heming Xia, Ranran Zhen
Abstract
Large Language Models (LLMs), despite their success in question answering, exhibit limitations in complex multi-hop question answering (MQA) tasks that necessitate non-linear, structured reasoning. This limitation stems from their inability to adequately capture deep conceptual relationships between entities. To overcome this challenge, we present ORACLE (Ontology-driven Reasoning And Chain for Logical Elucidation), a training-free framework that combines LLMs’ generative capabilities with the structural benefits of knowledge graphs. Our approach operates through three stages: (1) dynamic construction of question-specific knowledge ontologies using LLMs, (2) transformation of these ontologies into First-Order Logic (FOL) reasoning chains, and (3) systematic decomposition of the original query into logically coherent sub-questions. Extensive experiments across a diverse set of models and standard MQA benchmarks demonstrate that our framework achieves competitive performance while producing more interpretable reasoning chains.- Anthology ID:
- 2026.findings-acl.687
- Volume:
- Findings of the Association for Computational Linguistics: ACL 2026
- Month:
- July
- Year:
- 2026
- Address:
- San Diego, California, United States
- Editors:
- Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 14035–14051
- Language:
- URL:
- https://preview.aclanthology.org/ingest-acl/2026.findings-acl.687/
- DOI:
- Cite (ACL):
- Haonan Bian, Yutao Qi, Rui Yang, Yuanxi Che, Jiaqian Wang, Heming Xia, and Ranran Zhen. 2026. From Query to Logic: Ontology-Driven Multi-Hop Reasoning in LLMs. In Findings of the Association for Computational Linguistics: ACL 2026, pages 14035–14051, San Diego, California, United States. Association for Computational Linguistics.
- Cite (Informal):
- From Query to Logic: Ontology-Driven Multi-Hop Reasoning in LLMs (Bian et al., Findings 2026)
- PDF:
- https://preview.aclanthology.org/ingest-acl/2026.findings-acl.687.pdf