Paragraph Similarity Matches for Generating Multiple-choice Test Items

Halyna Maslak, Ruslan Mitkov


Abstract
Multiple-choice questions (MCQs) are widely used in knowledge assessment in educational institutions, during work interviews, in entertainment quizzes and games. Although the research on the automatic or semi-automatic generation of multiple-choice test items has been conducted since the beginning of this millennium, most approaches focus on generating questions from a single sentence. In this research, a state-of-the-art method of creating questions based on multiple sentences is introduced. It was inspired by semantic similarity matches used in the translation memory component of translation management systems. The performance of two deep learning algorithms, doc2vec and SBERT, is compared for the paragraph similarity task. The experiments are performed on the ad-hoc corpus within the EU domain. For the automatic evaluation, a smaller corpus of manually selected matching paragraphs has been compiled. The results prove the good performance of Sentence Embeddings for the given task.
Anthology ID:
2021.ranlp-srw.15
Volume:
Proceedings of the Student Research Workshop Associated with RANLP 2021
Month:
September
Year:
2021
Address:
Online
Venue:
RANLP
SIG:
Publisher:
INCOMA Ltd.
Note:
Pages:
99–108
Language:
URL:
https://aclanthology.org/2021.ranlp-srw.15
DOI:
Bibkey:
Cite (ACL):
Halyna Maslak and Ruslan Mitkov. 2021. Paragraph Similarity Matches for Generating Multiple-choice Test Items. In Proceedings of the Student Research Workshop Associated with RANLP 2021, pages 99–108, Online. INCOMA Ltd..
Cite (Informal):
Paragraph Similarity Matches for Generating Multiple-choice Test Items (Maslak & Mitkov, RANLP 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/update-css-js/2021.ranlp-srw.15.pdf