Bingsheng Yao
2021
Narrative Question Answering with Cutting-Edge Open-Domain QA Techniques: A Comprehensive Study
Xiangyang Mou
|
Chenghao Yang
|
Mo Yu
|
Bingsheng Yao
|
Xiaoxiao Guo
|
Saloni Potdar
|
Hui Su
Transactions of the Association for Computational Linguistics, Volume 9
Abstract Recent advancements in open-domain question answering (ODQA), that is, finding answers from large open-domain corpus like Wikipedia, have led to human-level performance on many datasets. However, progress in QA over book stories (Book QA) lags despite its similar task formulation to ODQA. This work provides a comprehensive and quantitative analysis about the difficulty of Book QA: (1) We benchmark the research on the NarrativeQA dataset with extensive experiments with cutting-edge ODQA techniques. This quantifies the challenges Book QA poses, as well as advances the published state-of-the-art with a ∼7% absolute improvement on ROUGE-L. (2) We further analyze the detailed challenges in Book QA through human studies.1 Our findings indicate that the event-centric questions dominate this task, which exemplifies the inability of existing QA models to handle event-oriented scenarios.
2020
Frustratingly Hard Evidence Retrieval for QA Over Books
Xiangyang Mou
|
Mo Yu
|
Bingsheng Yao
|
Chenghao Yang
|
Xiaoxiao Guo
|
Saloni Potdar
|
Hui Su
Proceedings of the First Joint Workshop on Narrative Understanding, Storylines, and Events
A lot of progress has been made to improve question answering (QA) in recent years, but the special problem of QA over narrative book stories has not been explored in-depth. We formulate BookQA as an open-domain QA task given its similar dependency on evidence retrieval. We further investigate how state-of-the-art open-domain QA approaches can help BookQA. Besides achieving state-of-the-art on the NarrativeQA benchmark, our study also reveals the difficulty of evidence retrieval in books with a wealth of experiments and analysis - which necessitates future effort on novel solutions for evidence retrieval in BookQA.
Search
Co-authors
- Xiangyang Mou 2
- Mo Yu 2
- Chenghao Yang 2
- Xiaoxiao Guo 2
- Saloni Potdar 2
- show all...
- Hui Su 2