FAMULUS: Interactive Annotation and Feedback Generation for Teaching Diagnostic Reasoning
Jonas Pfeiffer, Christian M. Meyer, Claudia Schulz, Jan Kiesewetter, Jan Zottmann, Michael Sailer, Elisabeth Bauer, Frank Fischer, Martin R. Fischer, Iryna Gurevych
Abstract
Our proposed system FAMULUS helps students learn to diagnose based on automatic feedback in virtual patient simulations, and it supports instructors in labeling training data. Diagnosing is an exceptionally difficult skill to obtain but vital for many different professions (e.g., medical doctors, teachers). Previous case simulation systems are limited to multiple-choice questions and thus cannot give constructive individualized feedback on a student’s diagnostic reasoning process. Given initially only limited data, we leverage a (replaceable) NLP model to both support experts in their further data annotation with automatic suggestions, and we provide automatic feedback for students. We argue that because the central model consistently improves, our interactive approach encourages both students and instructors to recurrently use the tool, and thus accelerate the speed of data creation and annotation. We show results from two user studies on diagnostic reasoning in medicine and teacher education and outline how our system can be extended to further use cases.- Anthology ID:
- D19-3013
- Volume:
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations
- Month:
- November
- Year:
- 2019
- Address:
- Hong Kong, China
- Editors:
- Sebastian Padó, Ruihong Huang
- Venues:
- EMNLP | IJCNLP
- SIG:
- SIGDAT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 73–78
- Language:
- URL:
- https://aclanthology.org/D19-3013
- DOI:
- 10.18653/v1/D19-3013
- Cite (ACL):
- Jonas Pfeiffer, Christian M. Meyer, Claudia Schulz, Jan Kiesewetter, Jan Zottmann, Michael Sailer, Elisabeth Bauer, Frank Fischer, Martin R. Fischer, and Iryna Gurevych. 2019. FAMULUS: Interactive Annotation and Feedback Generation for Teaching Diagnostic Reasoning. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations, pages 73–78, Hong Kong, China. Association for Computational Linguistics.
- Cite (Informal):
- FAMULUS: Interactive Annotation and Feedback Generation for Teaching Diagnostic Reasoning (Pfeiffer et al., EMNLP-IJCNLP 2019)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/D19-3013.pdf
- Data
- Famulus