Comparing Neural Question Generation Architectures for Reading Comprehension
E. Margaret Perkoff, Abhidip Bhattacharyya, Jon Cai, Jie Cao
Abstract
In recent decades, there has been a significant push to leverage technology to aid both teachers and students in the classroom. Language processing advancements have been harnessed to provide better tutoring services, automated feedback to teachers, improved peer-to-peer feedback mechanisms, and measures of student comprehension for reading. Automated question generation systems have the potential to significantly reduce teachers’ workload in the latter. In this paper, we compare three differ- ent neural architectures for question generation across two types of reading material: narratives and textbooks. For each architecture, we explore the benefits of including question attributes in the input representation. Our models show that a T5 architecture has the best overall performance, with a RougeL score of 0.536 on a narrative corpus and 0.316 on a textbook corpus. We break down the results by attribute and discover that the attribute can improve the quality of some types of generated questions, including Action and Character, but this is not true for all models.- Anthology ID:
- 2023.bea-1.47
- Volume:
- Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023)
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Ekaterina Kochmar, Jill Burstein, Andrea Horbach, Ronja Laarmann-Quante, Nitin Madnani, Anaïs Tack, Victoria Yaneva, Zheng Yuan, Torsten Zesch
- Venue:
- BEA
- SIG:
- SIGEDU
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 556–566
- Language:
- URL:
- https://aclanthology.org/2023.bea-1.47
- DOI:
- 10.18653/v1/2023.bea-1.47
- Cite (ACL):
- E. Margaret Perkoff, Abhidip Bhattacharyya, Jon Cai, and Jie Cao. 2023. Comparing Neural Question Generation Architectures for Reading Comprehension. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 556–566, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Comparing Neural Question Generation Architectures for Reading Comprehension (Perkoff et al., BEA 2023)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/2023.bea-1.47.pdf