Evaluating the Reliability of Human–AI Collaborative Scoring of Written Arguments Using Rational Force Model
Noriko Takahashi, Abraham Onuorah, Alina Reznitskaya, Evgeny Chukharev, Ariel Sykes, Michele Flammia, Joe Oyler
Abstract
This study aims to improve the reliability of a new AI collaborative scoring system used to assess the quality of students’ written arguments. The system draws on the Rational Force Model and focuses on classifying the functional relation of each proposition in terms of support, opposition, acceptability, and relevance.- Anthology ID:
- 2025.aimecon-wip.16
- Volume:
- Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress
- Month:
- October
- Year:
- 2025
- Address:
- Wyndham Grand Pittsburgh, Downtown, Pittsburgh, Pennsylvania, United States
- Editors:
- Joshua Wilson, Christopher Ormerod, Magdalen Beiting Parrish
- Venue:
- AIME-Con
- SIG:
- Publisher:
- National Council on Measurement in Education (NCME)
- Note:
- Pages:
- 135–140
- Language:
- URL:
- https://preview.aclanthology.org/ingest-emnlp/2025.aimecon-wip.16/
- DOI:
- Cite (ACL):
- Noriko Takahashi, Abraham Onuorah, Alina Reznitskaya, Evgeny Chukharev, Ariel Sykes, Michele Flammia, and Joe Oyler. 2025. Evaluating the Reliability of Human–AI Collaborative Scoring of Written Arguments Using Rational Force Model. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress, pages 135–140, Wyndham Grand Pittsburgh, Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
- Cite (Informal):
- Evaluating the Reliability of Human–AI Collaborative Scoring of Written Arguments Using Rational Force Model (Takahashi et al., AIME-Con 2025)
- PDF:
- https://preview.aclanthology.org/ingest-emnlp/2025.aimecon-wip.16.pdf