Abstract
Despite readily memorizing world knowledge about entities, pre-trained language models (LMs) struggle to compose together two or more facts to perform multi-hop reasoning in question-answering tasks. In this work, we propose techniques that improve upon this limitation by relying on random-walks over structured knowledge graphs. Specifically, we use soft-prompts to guide LMs to chain together their encoded knowledge by learning to map multi-hop questions to random-walk paths that lead to the answer. Applying our methods on two T5 LMs shows substantial improvements over standard tuning approaches in answering questions that require multi-hop reasoning.- Anthology ID:
- 2023.findings-acl.62
- Volume:
- Findings of the Association for Computational Linguistics: ACL 2023
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 972–985
- Language:
- URL:
- https://aclanthology.org/2023.findings-acl.62
- DOI:
- 10.18653/v1/2023.findings-acl.62
- Cite (ACL):
- Kanishka Misra, Cicero Nogueira dos Santos, and Siamak Shakeri. 2023. Triggering Multi-Hop Reasoning for Question Answering in Language Models using Soft Prompts and Random Walks. In Findings of the Association for Computational Linguistics: ACL 2023, pages 972–985, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Triggering Multi-Hop Reasoning for Question Answering in Language Models using Soft Prompts and Random Walks (Misra et al., Findings 2023)
- PDF:
- https://preview.aclanthology.org/revert-3132-ingestion-checklist/2023.findings-acl.62.pdf