Question Generation for Reading Comprehension Assessment by Modeling How and What to Ask

Bilal Ghanem, Lauren Lutz Coleman, Julia Rivard Dexter, Spencer von der Ohe, Alona Fyshe


Abstract
Reading is integral to everyday life, and yet learning to read is a struggle for many young learners. During lessons, teachers can use comprehension questions to increase engagement, test reading skills, and improve retention. Historically such questions were written by skilled teachers, but recently language models have been used to generate comprehension questions. However, many existing Question Generation (QG) systems focus on generating extractive questions from the text, and have no way to control the type of the generated question. In this paper, we study QG for reading comprehension where inferential questions are critical and extractive techniques cannot be used. We propose a two-step model (HTA-WTA) that takes advantage of previous datasets, and can generate questions for a specific targeted comprehension skill. We propose a new reading comprehension dataset that contains questions annotated with story-based reading comprehension skills (SBRCS), allowing for a more complete reader assessment. Across several experiments, our results show that HTA-WTA outperforms multiple strong baselines on this new dataset. We show that the HTA-WTA model tests for strong SCRS by asking deep inferential questions.
Anthology ID:
2022.findings-acl.168
Volume:
Findings of the Association for Computational Linguistics: ACL 2022
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2131–2146
Language:
URL:
https://aclanthology.org/2022.findings-acl.168
DOI:
10.18653/v1/2022.findings-acl.168
Bibkey:
Cite (ACL):
Bilal Ghanem, Lauren Lutz Coleman, Julia Rivard Dexter, Spencer von der Ohe, and Alona Fyshe. 2022. Question Generation for Reading Comprehension Assessment by Modeling How and What to Ask. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2131–2146, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Question Generation for Reading Comprehension Assessment by Modeling How and What to Ask (Ghanem et al., Findings 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/naacl-24-ws-corrections/2022.findings-acl.168.pdf
Video:
 https://preview.aclanthology.org/naacl-24-ws-corrections/2022.findings-acl.168.mp4
Data
CosmosQASQuAD