Towards Graph-hop Retrieval and Reasoning in Complex Question Answering over Textual Database
Minjun Zhu, Yixuan Weng, Shizhu He, Kang Liu, Haifeng Liu, Yang jun Jun, Jun Zhao
Abstract
In textual question answering (TQA) systems, complex questions often require retrieving multiple textual fact chains with multiple reasoning steps. While existing benchmarks are limited to single-chain or single-hop retrieval scenarios. In this paper, we propose to conduct Graph-Hop —— a novel multi-chains and multi-hops retrieval and reasoning paradigm in complex question answering. We construct a new benchmark called ReasonGraphQA, which provides explicit and fine-grained evidence graphs for complex question to support comprehensive and detailed reasoning. In order to further study how graph-based evidential reasoning can be performed, we explore what form of Graph-Hop works best for generating textual evidence explanations in knowledge reasoning and question answering. We have thoroughly evaluated existing evidence retrieval and reasoning models on the ReasonGraphQA. Experiments highlight Graph-Hop is a promising direction for answering complex questions, but it still has certain limitations. We have further studied mitigation strategies to meet these challenges and discuss future directions.- Anthology ID:
- 2024.lrec-main.1437
- Volume:
- Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
- Month:
- May
- Year:
- 2024
- Address:
- Torino, Italia
- Editors:
- Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
- Venues:
- LREC | COLING
- SIG:
- Publisher:
- ELRA and ICCL
- Note:
- Pages:
- 16539–16549
- Language:
- URL:
- https://aclanthology.org/2024.lrec-main.1437
- DOI:
- Cite (ACL):
- Minjun Zhu, Yixuan Weng, Shizhu He, Kang Liu, Haifeng Liu, Yang jun Jun, and Jun Zhao. 2024. Towards Graph-hop Retrieval and Reasoning in Complex Question Answering over Textual Database. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 16539–16549, Torino, Italia. ELRA and ICCL.
- Cite (Informal):
- Towards Graph-hop Retrieval and Reasoning in Complex Question Answering over Textual Database (Zhu et al., LREC-COLING 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.lrec-main.1437.pdf