TextGraphs 2019 Shared Task on Multi-Hop Inference for Explanation Regeneration

Peter Jansen, Dmitry Ustalov

Abstract
While automated question answering systems are increasingly able to retrieve answers to natural language questions, their ability to generate detailed human-readable explanations for their answers is still quite limited. The Shared Task on Multi-Hop Inference for Explanation Regeneration tasks participants with regenerating detailed gold explanations for standardized elementary science exam questions by selecting facts from a knowledge base of semi-structured tables. Each explanation contains between 1 and 16 interconnected facts that form an “explanation graph” spanning core scientific knowledge and detailed world knowledge. It is expected that successfully combining these facts to generate detailed explanations will require advancing methods in multi-hop inference and information combination, and will make use of the supervised training data provided by the WorldTree explanation corpus. The top-performing system achieved a mean average precision (MAP) of 0.56, substantially advancing the state-of-the-art over a baseline information retrieval model. Detailed extended analyses of all submitted systems showed large relative improvements in accessing the most challenging multi-hop inference problems, while absolute performance remains low, highlighting the difficulty of generating detailed explanations through multi-hop reasoning.
Anthology ID:
D19-5309
Volume:
Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-13)
Month:
November
Year:
2019
Address:
Hong Kong
Venue:
TextGraphs
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
63–77
Language:
URL:
https://aclanthology.org/D19-5309
DOI:
10.18653/v1/D19-5309
Bibkey:
Cite (ACL):
Peter Jansen and Dmitry Ustalov. 2019. TextGraphs 2019 Shared Task on Multi-Hop Inference for Explanation Regeneration. In Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-13), pages 63–77, Hong Kong. Association for Computational Linguistics.
Cite (Informal):
TextGraphs 2019 Shared Task on Multi-Hop Inference for Explanation Regeneration (Jansen & Ustalov, TextGraphs 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/remove-xml-comments/D19-5309.pdf
PDF Search