DT-QDC: A Dataset for Question Comprehension in Online Test
Sijin Wu, Yujiu Yang, Nicholas Yung, Zhengchen Shen, Zeyang Lei
Abstract
With the transformation of education from the traditional classroom environment to online education and assessment, it is more and more important to accurately assess the difficulty of questions than ever. As teachers may not be able to follow the student’s performance and learning behavior closely, a well-defined method to measure the difficulty of questions to guide learning is necessary. In this paper, we explore the concept of question difficulty and provide our new Chinese DT-QDC dataset. This is currently the largest and only Chinese question dataset, and it also has enriched attributes and difficulty labels. Additional attributes such as keywords, chapter, and question type would allow models to understand questions more precisely. We proposed the MTMS-BERT and ORMS-BERT, which can improve the judgment of difficulty from different views. The proposed methods outperforms different baselines by 7.79% on F1-score and 15.92% on MAE, 28.26% on MSE on the new DT-QDC dataset, laying the foundation for the question difficulty comprehension task.- Anthology ID:
- 2020.coling-main.569
- Volume:
- Proceedings of the 28th International Conference on Computational Linguistics
- Month:
- December
- Year:
- 2020
- Address:
- Barcelona, Spain (Online)
- Editors:
- Donia Scott, Nuria Bel, Chengqing Zong
- Venue:
- COLING
- SIG:
- Publisher:
- International Committee on Computational Linguistics
- Note:
- Pages:
- 6470–6480
- Language:
- URL:
- https://aclanthology.org/2020.coling-main.569
- DOI:
- 10.18653/v1/2020.coling-main.569
- Cite (ACL):
- Sijin Wu, Yujiu Yang, Nicholas Yung, Zhengchen Shen, and Zeyang Lei. 2020. DT-QDC: A Dataset for Question Comprehension in Online Test. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6470–6480, Barcelona, Spain (Online). International Committee on Computational Linguistics.
- Cite (Informal):
- DT-QDC: A Dataset for Question Comprehension in Online Test (Wu et al., COLING 2020)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2020.coling-main.569.pdf
- Code
- wusj18/DT-QDC