@inproceedings{ozten-wilkens-2026-evaluating,
    title = "Evaluating Transformer Model Family Representations Through Automated Essay Scoring",
    author = "Ozten, Akchay  and
      Wilkens, Rodrigo",
    editor = "Shardlow, Matthew  and
      Fran{\c{c}}ois, Thomas  and
      Amaro, Raquel  and
      Baptista, Jorge  and
      Cardon, R{\'e}mi  and
      Ribeiro, Eug{\'e}nio  and
      Saggion, Horacio  and
      Stodden, Regina  and
      Todirascu, Amalia  and
      Wilkens, Rodrigo",
    booktitle = "Proceedings of the Joint Workshop on Readability and Text Simplification ({READI}x{TSAR}) @ {LREC} 2026",
    month = may,
    year = "2026",
    address = "Palma, Mallorca (Spain)",
    publisher = "ELRA Language Resources Association (ELRA)",
    url = "https://preview.aclanthology.org/ingest-lrec/2026.readi-1.11/",
    doi = "10.63317/35ytj3qcrwv4",
    pages = "142--150",
    abstract = "Large Language Models have become central to Automated Essay Scoring (AES), typically through fine-tuned transformer encoders or prompt-based applications of decoder models. However, the representational capacity of decoder models as frozen embedding extractors remains largely unexplored. In this paper, we present a controlled comparison between encoder and decoder transformer embeddings for prompt-agnostic AES. Using regression models, we evaluate frozen representations across two English datasets. We analyzed scaling effects and the impact of integrating explicit linguistic features in hybrid configurations. Our results show that decoder embeddings consistently outperform encoder embeddings in embedding-only settings, with gains generalizing across holistic essay scoring and proficiency prediction. Scaling effects are modest, and hybrid models that combine contextual embeddings with linguistic features yield further improvements. Notably, frozen decoder embeddings achieve performance competitive with a fine-tuned BERT. These findings highlight the importance of representation-level properties in essay scoring."
}