Exploring LLM-Based Assessment of Italian Middle School Writing: A Pilot Study

Adriana Mirabella, Dominique Brunato


Abstract
This study investigates the use of ChatGPT for Automated Essay Scoring (AES) in assessing Italian middle school students’ written texts. Using rubrics targeting grammar, coherence and argumentation, we compare AI-generated feedback with that of a human teacher on a newly collected corpus of students’ essays. Despite some differences, ChatGPT provided detailed and timely feedback that complements the teacher’s role. These findings underscore the potential of generative AI to improve the assessment of writing, providing useful insights for educators and supporting students in developing their writing skills.
Anthology ID:
2025.bea-1.51
Volume:
Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025)
Month:
July
Year:
2025
Address:
Vienna, Austria
Editors:
Ekaterina Kochmar, Bashar Alhafni, Marie Bexte, Jill Burstein, Andrea Horbach, Ronja Laarmann-Quante, Anaïs Tack, Victoria Yaneva, Zheng Yuan
Venues:
BEA | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
708–715
Language:
URL:
https://preview.aclanthology.org/landing_page/2025.bea-1.51/
DOI:
Bibkey:
Cite (ACL):
Adriana Mirabella and Dominique Brunato. 2025. Exploring LLM-Based Assessment of Italian Middle School Writing: A Pilot Study. In Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025), pages 708–715, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
Exploring LLM-Based Assessment of Italian Middle School Writing: A Pilot Study (Mirabella & Brunato, BEA 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/landing_page/2025.bea-1.51.pdf