@inproceedings{liu-etal-2026-mdeval,
    title = "{M}d{E}val: Massively Multilingual Code Debugging",
    author = "Liu, Shukai  and
      Chai, Linzheng  and
      Yang, Jian  and
      Shi, Jiajun  and
      Zhu, He  and
      Wang, Liran  and
      Ke, Jin  and
      Zhang, Wei  and
      Zhu, Hualei  and
      Guo, Shuyue  and
      Sun, Tao  and
      Liu, Jiaheng  and
      Duan, Yunlong  and
      Hao, Yu  and
      Yang, Liqun  and
      Niu, Guanglin  and
      Zhang, Ge  and
      Li, Zhoujun",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1041/",
    pages = "20780--20797",
    ISBN = "979-8-89176-395-1",
    abstract = "Code large language models (LLMs) have made significant progress in code debugging by directly generating the correct code based on the buggy code snippet. Programming benchmarks, typically consisting of buggy code snippets and their associated test cases, are used to assess the debugging capabilities of LLMs. However, many existing benchmarks primarily focus on Python and are often limited in terms of language diversity (e.g., DebugBench and DebugEval). To advancethe field of multilingual debugging with LLMs, we propose the first massively multilingual debugging benchmark, which includes 3.9K test samples of 20 programming languages and covers the automated program repair (APR) task, the bug localization(BL) task, and the bug identification (BI) task. In addition, we introduce the debugging instruction corpora MdEval-Instruct by injecting bugs into the correct multilingual queries and solutions (xDebugGen). Further, a multilingual debugger xDebugCoder trained on MdEval-Instruct as a strong baseline specifically to handle bugs of a wide range of programming languages (e.g. ``Missing Mut'' in language Rust and ``Misused Macro Definition'' in language C). Our extensive experiments on MdEval reveal a notable performance gap between open-source and closed-source LLMs (e.g., GPT and Claudeseries), highlighting huge room for improvement in multilingual code debugging scenarios."
}