@inproceedings{schaller-etal-2026-keyscore,
    title = "{KEYSCORE} {---} Keystroke-enhanced Automated Essay Scoring",
    author = {Schaller, Nils-Jonathan  and
      Melanchthon, Daniel Mora  and
      Jansen, Thorben  and
      K{\"o}ller, Olaf  and
      Horbach, Andrea},
    editor = "Kochmar, Ekaterina  and
      Alhafni, Bashar  and
      Bann{\`o}, Stefano  and
      Bexte, Marie  and
      Burstein, Jill  and
      Horbach, Andrea  and
      Laarmann-Quante, Ronja  and
      Tack, Anais  and
      Yaneva, Victoria  and
      Yuan, Zheng",
    booktitle = "Proceedings of the 21st Workshop on Innovative Use of {NLP} for Building Educational Applications ({BEA} 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.bea-1.24/",
    pages = "347--357",
    ISBN = "979-8-89176-409-5",
    abstract = "We investigate the predictive power of keystroke logging data for automated essay scoring using the newly collected PISA FLA writing process dataset. Based on 3,882 writing sessions, we extract a comprehensive set of keystroke-based process features, including temporal measures, pause and burst patterns, deletion behavior, production efficiency, and navigation activity and evaluate their ability to predict holistic essay scores on a 0{--}5 scale. We specifically compare process-feature-based models with content-based scoring approaches trained on data written with and without the help of an AI chatbot, and investigate how predictive power evolves over the course of a writing session by training models at multiple time thresholds.Our analysis reveals that keystroke features provide genuine early predictive signal, capturing aspects of writing fluency and revision behavior that distinguish writers before their texts are long enough to score conventionally. Additionally, our results suggest that process-based scoring is a viable complement to product-based approaches, with promise for formative, real-time feedback during writing."
}