Through the Sentence Lens: Explainable Essay Scoring through Fine-Grained Predictions

Daniel Mora Melanchthon, Stefan Keller, Andrea Horbach


Abstract
Beyond performance, model transparency is a crucial factor in Automated Essay Scoring, yet current systems often lack explainability, limiting their pedagogical value and users’ trust. Existing explainability methods, such as gradient-based attribution or feature-importance approaches, either produce counterintuitive explanations or are too complex for classroom use. To address this limitation, we make use of fine-grained prediction at the sentence level as a way to enhance explainability. We propose ablation strategies to derive sentence-level pseudo scores from essay-level gold scores and use them to train sentence-level models. We evaluate their performance against essay-level baselines on two datasets (ASAP and MEWS), and compare their sentence-level output to a human baseline. Results indicate a trade-off between essay-level performance and sentence-level granularity. For the language quality trait, most sentence-level models achieve performance comparable to the essay-level baseline, whereas for content, the approach yields more positive results on prompts with shorter
Anthology ID:
2026.bea-1.16
Volume:
Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ekaterina Kochmar, Bashar Alhafni, Stefano Bannò, Marie Bexte, Jill Burstein, Andrea Horbach, Ronja Laarmann-Quante, Anais Tack, Victoria Yaneva, Zheng Yuan
Venues:
BEA | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
221–233
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.bea-1.16/
DOI:
Bibkey:
Cite (ACL):
Daniel Mora Melanchthon, Stefan Keller, and Andrea Horbach. 2026. Through the Sentence Lens: Explainable Essay Scoring through Fine-Grained Predictions. In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), pages 221–233, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
Through the Sentence Lens: Explainable Essay Scoring through Fine-Grained Predictions (Melanchthon et al., BEA 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.bea-1.16.pdf