@inproceedings{singh-etal-2026-evaluating,
    title = "Evaluating Customized vs. Generalist Transformer-based Models for Legal Contract Classification",
    author = "Singh, Amrita  and
      Karaca, H. Suhan  and
      Joshi, Aditya  and
      Paik, Hye-young  and
      Jiang, Jiaojiao",
    editor = "Mysore, Sheshera  and
      Kumar, Sachin  and
      Balachandran, Vidhisha  and
      Hayati, Shirley Anugrah  and
      Brahman, Faeze  and
      Moussa, Hanane Nour  and
      Salemi, Alireza",
    booktitle = "Proceedings of the Second Workshop on Customizable {NLP}: Progress and Challenges in Customizing {NLP} for a Domain, Application, Group, or Individual ({C}ustom{NLP}4{U})",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.customnlp4u-1.5/",
    pages = "44--54",
    ISBN = "979-8-89176-396-8",
    abstract = "Despite advances in legal NLP, no comprehensive evaluation of Transformer-based models customized for legal tasks (referred to as `legal-specific' models in this paper) exists for contract classification tasks. To address this gap, we present an evaluation of 13 legal-specific transformer-based models on 3 English-language contract classification tasks and compare them with 9 generalist models. The results show that legal-specific models consistently outperform generalist models, especially on tasks requiring nuanced legal understanding. They also help reduce misclassification of rare classes in imbalanced datasets. Legal-BERT and Contracts-BERT establish new SOTAs on two of the three tasks, despite having 69{\%} fewer parameters than the best-performing generalist models. We also identify CaseLaw-BERT and LexLM as strong additional baselines for contract classification. Our results highlight the shortcomings of generalist models, emphasizing the need for domain-specific customization, particularly in the context of legal applications."
}