@inproceedings{akyurek-etal-2026-prbench,
    title = "{PRB}ench: Large-Scale Expert Rubrics for Evaluating High-Stakes Professional Reasoning",
    author = {Aky{\"u}rek, Afra Feyza  and
      Gosai, Advait  and
      Zhang, Chen Bo Calvin  and
      Gupta, Vipul  and
      Jeong, Jaehwan  and
      Gunjal, Anisha  and
      Rabbani, Tahseen  and
      Mazzone, Maria  and
      IV, David Randolph  and
      Meymand, Mohammad Mahmoudi  and
      Chattha, Gurshaan  and
      Rodriguez, Paula  and
      Buendia, Diego A. Mares  and
      Singh, Pavit  and
      Liu, Michael  and
      Chawla, Subodh  and
      Cline, Peter  and
      Ogaz, Lucy  and
      Montoya, Ernesto Gabriel Hern{\'a}ndez  and
      Wang, Zihao  and
      Bhatter, Pavi  and
      Ayestaran, Marcos  and
      Liu, Bing  and
      He, Yunzhong},
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1958/",
    pages = "42297--42325",
    ISBN = "979-8-89176-390-6",
    abstract = "Frontier model progress is often measured using academic benchmarks that provide a limited view of performance on open-ended, economically consequential tasks in high-stakes professional domains where practical returns matter most. We introduce Professional Reasoning Bench (PRBench), a realistic, open-ended, and difficult benchmark of real-world problems in Finance and Law. We open-source its 1,100 expert-authored tasks and 19,356 expert-curated criteria, making it the largest public, rubric-based benchmark for both legal and finance domains. We recruit 182 qualified professionals, holding JDs, CFAs, or 6+ years of experience, who contributed questions inspired by their actual workflows. This process yields significant diversity, with tasks spanning 114 countries and 47 US jurisdictions. Our expert-curated rubrics are validated through a rigorous quality pipeline, including independent expert validation. Subsequent evaluation of 20 leading models reveals substantial room for improvement, with top scores of only 0.39 (Finance) and 0.37 (Legal) on our Hard subsets. We further catalog associated economic impacts of the prompts and analyze performance using human-annotated rubric categories. Common failure modes include inaccurate judgments, a lack of process transparency and incomplete reasoning, highlighting critical gaps in their reliability for professional adoption."
}