@inproceedings{chen-etal-2026-jurisbench,
    title = "{J}uris{B}ench: A Deep Benchmark for Assessing Large Language Models in Professional Legal Practice",
    author = "Chen, Ziang  and
      Li, Guannan  and
      Ji, Fanlin  and
      Kang, Yipeng  and
      Li, Jiaqi  and
      Zhang, Muhan  and
      Zhang, Yangtao  and
      Tianjiao, Li  and
      Wang, Jiannan  and
      Guo, Xin  and
      Zhu, Song-Chun  and
      Ling, Bin",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1666/",
    pages = "35994--36018",
    ISBN = "979-8-89176-390-6",
    abstract = "Large Language Models (LLMs) have demonstrated strong cross-domain capabilities, yet their competence in specialized professional tasks remains underexamined. Existing legal benchmarks evaluate isolated tasks or exam-style questions, failing to capture the procedural interdependencies and adjudicative rigor inherent in professional practice. To bridge this gap, we construct JurisBench, a vertical, depth-oriented, domain-specific benchmark designed to evaluate LLMs across key stages of Chinese civil litigation. JurisBench introduces a Linear Depth Simulation track that mirrors the cognitive workflow of professional judges through four sequential, dependency-aware phases: Cause of Action prediction, Focus of Disputes identification, Rationale of the Judgment generation, and Result of the Judgment determination. Results reveal an ``illusion of competence'': state-of-the-art models exhibit marked performance degradation in end-to-end pipelines due to cascading error propagation. We identify precise statutory grounding as a persistent bottleneck, highlighting a critical gap between fluent linguistic output and judicial reliability. JurisBench shifts evaluation from isolated legal knowledge to workflow-level task execution, providing a diagnostic framework for legal AI and a template for benchmark design in specialized domains."
}