Yuanxi Che
2026
From Query to Logic: Ontology-Driven Multi-Hop Reasoning in LLMs
Haonan Bian | Yutao Qi | Rui Yang | Yuanxi Che | Jiaqian Wang | Heming Xia | Ranran Zhen
Findings of the Association for Computational Linguistics: ACL 2026
Haonan Bian | Yutao Qi | Rui Yang | Yuanxi Che | Jiaqian Wang | Heming Xia | Ranran Zhen
Findings of the Association for Computational Linguistics: ACL 2026
Large Language Models (LLMs), despite their success in question answering, exhibit limitations in complex multi-hop question answering (MQA) tasks that necessitate non-linear, structured reasoning. This limitation stems from their inability to adequately capture deep conceptual relationships between entities. To overcome this challenge, we present ORACLE (Ontology-driven Reasoning And Chain for Logical Elucidation), a training-free framework that combines LLMs’ generative capabilities with the structural benefits of knowledge graphs. Our approach operates through three stages: (1) dynamic construction of question-specific knowledge ontologies using LLMs, (2) transformation of these ontologies into First-Order Logic (FOL) reasoning chains, and (3) systematic decomposition of the original query into logically coherent sub-questions. Extensive experiments across a diverse set of models and standard MQA benchmarks demonstrate that our framework achieves competitive performance while producing more interpretable reasoning chains.