Pre-trained Transformer Models for Standard-to-Standard Alignment Study

Hye-Jeong Choi, Reese Butterfuss, Meng Fan


Abstract
The current study evaluated the accuracy of five pre-trained large language models (LLMs) in matching human judgment for standard-to-standard alignment study. Results demonstrated comparable performance LLMs across despite differences in scale and computational demands. Additionally, incorporating domain labels as auxiliary information did not enhance LLMs performance. These findings provide initial evidence for the viability of open-source LLMs to facilitate alignment study and offer insights into the utility of auxiliary information.
Anthology ID:
2025.aimecon-main.33
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers
Month:
October
Year:
2025
Address:
Wyndham Grand Pittsburgh, Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
306–311
Language:
URL:
https://preview.aclanthology.org/ingest-emnlp/2025.aimecon-main.33/
DOI:
Bibkey:
Cite (ACL):
Hye-Jeong Choi, Reese Butterfuss, and Meng Fan. 2025. Pre-trained Transformer Models for Standard-to-Standard Alignment Study. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers, pages 306–311, Wyndham Grand Pittsburgh, Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
Pre-trained Transformer Models for Standard-to-Standard Alignment Study (Choi et al., AIME-Con 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-emnlp/2025.aimecon-main.33.pdf