@inproceedings{liu-etal-2025-beyond,
    title = "Beyond the Answer: Advancing Multi-Hop {QA} with Fine-Grained Graph Reasoning and Evaluation",
    author = "Liu, Qichuan  and
      Zhang, Chentao  and
      Zheng, Chenfeng  and
      Hu, Guosheng  and
      Li, Xiaodong  and
      Zhang, Zhihong",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.1142/",
    pages = "23433--23456",
    ISBN = "979-8-89176-251-0",
    abstract = "Recent advancements in large language models (LLMs) have significantly improved the performance of multi-hop question answering (MHQA) systems. Despite the success of MHQA systems, the evaluation of MHQA is not deeply investigated. Existing evaluations mainly focus on comparing the final answers of the reasoning method and given ground-truths. We argue that the reasoning process should also be evaluated because wrong reasoning process can also lead to the correct final answers. Motivated by this, we propose a ``Planner-Executor-Reasoner'' (PER) architecture, which forms the core of the Plan-anchored Data Preprocessing (PER-DP) and the Plan-guided Multi-Hop QA (PER-QA).The former provides the ground-truth of intermediate reasoning steps and final answers, and the latter offers them of a reasoning method. Moreover, we design a fine-grained evaluation metric called Plan-aligned Stepwise Evaluation (PSE), which evaluates the intermediate reasoning steps from two aspects: planning and solving. Extensive experiments on ten types of questions demonstrate competitive reasoning performance, improved explainability of the MHQA system, and uncover issues such as ``fortuitous reasoning continuance'' and ``latent reasoning suspension'' in RAG-based MHQA systems. Besides, we also demonstrate the potential of our approach in data contamination scenarios."
}