Anastasia Smirnova
2025
Numeric Information in Elementary School Texts Generated by LLMs vs Human Experts
Anastasia Smirnova
|
Erin S. Lee
|
Shiying Li
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers
We analyze GPT-4o’s ability to represent numeric information in texts for elementary school children and assess it with respect to the human baseline. We show that both humans and GPT-4o reduce the amount of numeric information when adapting informational texts for children but GPT-4o retains more complex numeric types than humans do.
Linguistic proficiency of humans and LLMs in Japanese: Effects of task demands and content
May Lynn Reese
|
Anastasia Smirnova
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers
We evaluate linguistic proficiency of humans and LLMs on pronoun resolution in Japanese, using the Winograd Schema Challenge dataset. Humans outperform LLMs in the baseline condition, but we find evidence for task demand effectss in both humans and LLMs. We also found that LLMs surpass human performance in scenarios referencing US culture, providing strong evidence for content effects.