Abstract
Automatic short answer grading (ASAG) techniques are designed to automatically assess short answers written in natural language. Apart from MCQs, evaluating free text answer is essential to assess the knowledge and understanding of children in the subject. But assessing descriptive answers in low resource languages in a linguistically diverse country like India poses significant hurdles. To solve this assessment problem and advance NLP research in regional Indian languages, we present the Science Answer Assessment (ScAA) dataset of children’s answers in the age group of 8-14. ScAA dataset is a 2-way (correct/incorrect) labeled dataset and contains 10,988 and 1,955 pairs of natural answers along with model answers for Hindi and Marathi respectively for 32 questions. We benchmark various state-of-the-art ASAG methods, and show the data presents a strong challenge for future research.- Anthology ID:
- 2020.icon-main.58
- Volume:
- Proceedings of the 17th International Conference on Natural Language Processing (ICON)
- Month:
- December
- Year:
- 2020
- Address:
- Indian Institute of Technology Patna, Patna, India
- Editors:
- Pushpak Bhattacharyya, Dipti Misra Sharma, Rajeev Sangal
- Venue:
- ICON
- SIG:
- Publisher:
- NLP Association of India (NLPAI)
- Note:
- Pages:
- 430–436
- Language:
- URL:
- https://aclanthology.org/2020.icon-main.58
- DOI:
- Cite (ACL):
- Dolly Agarwal, Somya Gupta, and Nishant Baghel. 2020. ScAA: A Dataset for Automated Short Answer Grading of Children’s free-text Answers in Hindi and Marathi. In Proceedings of the 17th International Conference on Natural Language Processing (ICON), pages 430–436, Indian Institute of Technology Patna, Patna, India. NLP Association of India (NLPAI).
- Cite (Informal):
- ScAA: A Dataset for Automated Short Answer Grading of Children’s free-text Answers in Hindi and Marathi (Agarwal et al., ICON 2020)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/2020.icon-main.58.pdf