@inproceedings{wang-etal-2026-towards-ip,
    title = "Towards {IP} Intelligence: Benchmarking Large Language Models on Intellectual Property Knowledge and Practice",
    author = "Wang, Qiyao  and
      Chen, Guhong  and
      Wang, Hongbo  and
      Liu, Huaren  and
      Zhu, Minghui  and
      Qin, Zhifei  and
      Linwei, Li  and
      Yue, Yilin  and
      Wang, Shiqiang  and
      Li, Jiayan  and
      Yihang, Wu  and
      Liu, Ziqiang  and
      Chen, Longze  and
      Luo, Run  and
      Fan, Liyang  and
      Li, Jiaming  and
      Zhang, Lei  and
      Xu, Kan  and
      Alinejad-Rokny, Hamid  and
      Li, Chengming  and
      Ni, Shiwen  and
      Lin, Yuan  and
      Yang, Min",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1000/",
    pages = "20007--20052",
    ISBN = "979-8-89176-395-1",
    abstract = "Intellectual Property (IP) is a highly specialized domain that integrates technical and legal knowledge, making it inherently complex and knowledge-intensive. Recent advancements in LLMs have demonstrated their potential to handle IP tasks, enabling more efficient analysis, understanding, and generation of IP-related content. However, existing datasets and benchmarks focus narrowly on patents or cover limited aspects of the IP field, lacking alignment with real-world scenarios. To bridge this gap, we introduce **IPBench**, the first comprehensive IP task taxonomy and a large-scale bilingual benchmark encompassing **8 IP mechanisms and 20 distinct tasks**, designed to evaluate LLMs in real-world IP practice. We benchmark **19 main LLMs**, ranging from general purpose to domain-specific, including chat-oriented and reasoning-focused models, under zero-shot, few-shot, and chain-of-thought settings. Our results show that even the top-performing model, DeepSeek-V3, achieves only 75.8{\%} accuracy, indicating significant room for improvement. Notably, open-source IP and law-oriented models lag behind closed-source general-purpose models. To foster future research, we publicly release IPBench, and will expand it with additional tasks to better reflect real-world complexities and support model advancements in the IP domain. We provide the data, code in the supplementary materials."
}