@inproceedings{zeng-etal-2026-treediff,
    title = "{T}ree{D}iff: {AST}-Guided Code Generation with Diffusion {LLM}s",
    author = "Zeng, Yiming  and
      Cao, Jinghan  and
      Li, Zexin  and
      Chen, Yiming  and
      Ren, Tao  and
      Li, Zhuochun  and
      Xiang, Dawei  and
      Wu, Xidong  and
      Gao, Shangqian  and
      Yu, Tingting",
    editor = "Gupta, Vivek  and
      Ding, Kaize  and
      Kokel, Harsha  and
      Zhao, Yue  and
      Agarwal, Amit  and
      Wang, Yu  and
      Glass, Michael  and
      Zhang, Yu  and
      Srinivas, Kavitha  and
      Chen, Xiusi  and
      Hassanzadeh, Oktie  and
      Zhu, Qi  and
      Chang, Shuaichen  and
      Luo, Yuan",
    booktitle = "Proceedings of the First Workshop on Structured Understanding, Retrieval, and Generation in the {LLM} Era ({SURG}e{LLM} 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.surgellm-1.5/",
    pages = "93--106",
    ISBN = "979-8-89176-406-4",
    abstract = "Code generation is increasingly critical for real-world applications. Still, diffusion-based large language models continue to struggle with this demand. Unlike free-form text, code requires syntactic precision; even minor structural inconsistencies can render a program non-executable. Existing diffusion-based large language models rely on random token masking for corruption, leading to two key failures: they lack awareness of syntactic boundaries during the iterative denoising process, and they fail to capture the long-range hierarchical dependencies essential for program correctness.We propose TreeDiff to address both issues. Specifically, we propose a syntax-aware diffusion framework that incorporates structural priors from Abstract Syntax Tree (AST) into the corruption process. Instead of masking individual tokens at random, we selectively mask tokens belonging to key AST nodes. By aligning the corruption process with the underlying structure of code, our method encourages the model to internalize the compositional nature of programming languages, enabling it to reconstruct programs that respect grammatical boundaries and capture long-range dependencies. Our method achieves a 13.3{\%} relative improvement over the random masking training method, demonstrating its effectiveness in code generation task by leveraging underlying structures."
}