Agenda-Driven Question Generation: A Case Study in the Courtroom Domain

Yi Fung, Anoop Kumar, Aram Galstyan, Heng Ji, Prem Natarajan

Abstract
This paper introduces a novel problem of automated question generation for courtroom examinations, CourtQG. While question generation has been studied in domains such as educational testing and product description, CourtQG poses several unique challenges owing to its non-cooperative and agenda-driven nature. Specifically, not only the generated questions need to be relevant to the case and underlying context, they also have to achieve certain objectives such as challenging the opponent’s arguments and/or revealing potential inconsistencies in their answers. We propose to leverage large language models (LLM) for CourtQG by fine-tuning them on two auxiliary tasks, agenda explanation (i.e., uncovering the underlying intents) and question type prediction. We additionally propose cold-start generation of questions from background documents without relying on examination history. We construct a dataset to evaluate our proposed method and show that it generates better questions according to standard metrics when compared to several baselines.
Anthology ID:
2024.lrec-main.49
Volume:
Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
Month:
May
Year:
2024
Address:
Torino, Italia
Editors:
Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
Venues:
LREC | COLING
SIG:
Publisher:
ELRA and ICCL
Note:
Pages:
572–583
Language:
URL:
https://aclanthology.org/2024.lrec-main.49
DOI:
Bibkey:
Cite (ACL):
Yi Fung, Anoop Kumar, Aram Galstyan, Heng Ji, and Prem Natarajan. 2024. Agenda-Driven Question Generation: A Case Study in the Courtroom Domain. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 572–583, Torino, Italia. ELRA and ICCL.
Cite (Informal):
Agenda-Driven Question Generation: A Case Study in the Courtroom Domain (Fung et al., LREC-COLING 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-4/2024.lrec-main.49.pdf
PDF Search