@inproceedings{yang-etal-2026-abc,
    title = "{ABC}-Bench: Benchmarking Agentic Backend Coding in Real-World Development",
    author = "Yang, Jie  and
      Guo, Honglin  and
      Ji, Li  and
      Zhou, Jiazheng  and
      Zheng, Rui  and
      Lei, Zhikai  and
      Zhang, Shuo  and
      Xi, Zhiheng  and
      Liu, Shichun  and
      Wang, Yuxin  and
      Wang, Bo  and
      Zheng, Yining  and
      Gui, Tao  and
      Qiu, Xipeng",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1142/",
    pages = "22765--22787",
    ISBN = "979-8-89176-395-1",
    abstract = "The evolution of Large Language Models (LLMs) into autonomous agents has expanded the scope of AI coding from localized code generation to complex, repository-level, and execution-driven problem solving. However, current benchmarks predominantly evaluate code logic in static contexts, neglecting the dynamic, full-process requirements of real-world engineering, particularly in backend development which demands rigorous environment configuration and service deployment. To address this gap, we introduce ABC-Bench, a benchmark explicitly designed to evaluate agentic backend coding within a realistic, executable workflow. Using a scalable automated pipeline, we curated 224 practical tasks spanning 8 languages and 19 frameworks from open-source repositories. Distinct from previous evaluations, ABC-Bench require the agents to manage the entire development lifecycle from repository exploration to instantiating containerized services and pass the external end-to-end API tests. Our extensive evaluation reveals that even state-of-the-art models struggle to deliver reliable performance on these holistic tasks, highlighting a substantial disparity between current model capabilities and the demands of practical backend engineering."
}