Abstract
We propose a method to teach an automated agent to learn how to search for multi-hop paths of relations between entities in an open domain. The method learns a policy for directing existing information retrieval and machine reading resources to focus on relevant regions of a corpus. The approach formulates the learning problem as a Markov decision process with a state representation that encodes the dynamics of the search process and a reward structure that minimizes the number of documents that must be processed while still finding multi-hop paths. We implement the method in an actor-critic reinforcement learning algorithm and evaluate it on a dataset of search problems derived from a subset of English Wikipedia. The algorithm finds a family of policies that succeeds in extracting the desired information while processing fewer documents compared to several baseline heuristic algorithms.- Anthology ID:
- 2022.suki-1.4
- Volume:
- Proceedings of the Workshop on Structured and Unstructured Knowledge Integration (SUKI)
- Month:
- July
- Year:
- 2022
- Address:
- Seattle, USA
- Editors:
- Wenhu Chen, Xinyun Chen, Zhiyu Chen, Ziyu Yao, Michihiro Yasunaga, Tao Yu, Rui Zhang
- Venue:
- SUKI
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 26–35
- Language:
- URL:
- https://aclanthology.org/2022.suki-1.4
- DOI:
- 10.18653/v1/2022.suki-1.4
- Cite (ACL):
- Enrique Noriega-Atala, Mihai Surdeanu, and Clayton Morrison. 2022. Learning Open Domain Multi-hop Search Using Reinforcement Learning. In Proceedings of the Workshop on Structured and Unstructured Knowledge Integration (SUKI), pages 26–35, Seattle, USA. Association for Computational Linguistics.
- Cite (Informal):
- Learning Open Domain Multi-hop Search Using Reinforcement Learning (Noriega-Atala et al., SUKI 2022)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2022.suki-1.4.pdf
- Data
- WikiHop