@inproceedings{mayfield-black-2020-fine,
    title = "Should You Fine-Tune {BERT} for Automated Essay Scoring?",
    author = "Mayfield, Elijah  and
      Black, Alan W",
    editor = "Burstein, Jill  and
      Kochmar, Ekaterina  and
      Leacock, Claudia  and
      Madnani, Nitin  and
      Pil{\'a}n, Ildik{\'o}  and
      Yannakoudakis, Helen  and
      Zesch, Torsten",
    booktitle = "Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications",
    month = jul,
    year = "2020",
    address = "Seattle, WA, USA {\textrightarrow} Online",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2020.bea-1.15/",
    doi = "10.18653/v1/2020.bea-1.15",
    pages = "151--162",
    abstract = "Most natural language processing research now recommends large Transformer-based models with fine-tuning for supervised classification tasks; older strategies like bag-of-words features and linear models have fallen out of favor. Here we investigate whether, in automated essay scoring (AES) research, deep neural models are an appropriate technological choice. We find that fine-tuning BERT produces similar performance to classical models at significant additional cost. We argue that while state-of-the-art strategies do match existing best results, they come with opportunity costs in computational resources. We conclude with a review of promising areas for research on student essays where the unique characteristics of Transformers may provide benefits over classical methods to justify the costs."
}