@inproceedings{wang-etal-2026-livecannbench,
    title = "{L}ive{CANNB}ench: Benchmark {SWE} {AI} Coding for Ascend {CANN}",
    author = "Wang, Sijie  and
      Zhao, Kai  and
      Tay, Wee Peng  and
      Zhang, Shuo  and
      Liu, Chengwen  and
      Guo, Quanjiang  and
      Junhao, Ren  and
      Li, Xin  and
      Lian, Heng  and
      Lei, Jingdi  and
      She, Rui  and
      Wang, Huacan  and
      Chen, Ronghao",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1143/",
    pages = "22788--22803",
    ISBN = "979-8-89176-395-1",
    abstract = "AI coding has emerged as a core application of large language models (LLMs), evolving from single-file coding tasks towards complex software engineering (SWE) scenarios. Recent advances in agents have enabled multi-file, multi-language, and dependency-aware AI coding, significantly expanding the scope of AI-assisted software development. While a variety of benchmarks have been proposed to evaluate coding capabilities in general-purpose or GPU coding ecosystems such as CUDA and ROCm, systematic evaluation for Huawei Ascend CANN remains largely underexplored. In this work, we propose LiveCANNBench, an SWE-level benchmark designed for AI coding in the CANN software stack. LiveCANNBench is constructed from real-world CANN repositories and consists of over 400 task instances spanning multi-file, multi-language, and execution-aware coding challenges. Unlike existing static benchmarks that primarily focus on kernel-level code generation, LiveCANNBench adopts a live benchmarking paradigm, effectively mitigating data leakage and enabling more reliable evaluation of modern coding agents."
}