@inproceedings{cheng-etal-2026-sdar-synergistic,
    title = "{SDAR}: A Synergistic Diffusion-{A}uto{R}egression Paradigm for Scalable Sequence Generation",
    author = "Cheng, Shuang  and
      Bian, Yihan  and
      Liu, Dawei  and
      Jiang, Yuhua  and
      Liu, Yihao  and
      Zhang, Linfeng  and
      Yao, Qian  and
      Tian, Zhongbo  and
      Wang, Wenhai  and
      Guo, Qipeng  and
      Chen, Kai  and
      Qi, Biqing  and
      Zhou, Bowen",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1110/",
    pages = "22058--22075",
    ISBN = "979-8-89176-395-1",
    abstract = "Autoregressive (AR) language modeling remains the dominant paradigm due to its dense supervision signal and highly optimized serving infrastructure, but its strictly causal, token-by-token decoding limits parallelism and non-causal modeling. While masked diffusion offers a promising path toward parallel generation, it faces two critical bottlenecks: training inefficiency stemming from sparse masked objectives, and high latency caused by iterative whole-sequence denoising. We present a systematic study of blockwise discrete diffusion, a pragmatic middle ground that preserves AR-compatible serving while enabling parallel intra-block generation. Our study proceeds in four steps: (i) a \textbf{controlled, compute- and scale-matched comparison} revealing that AR is a more effective backbone for blockwise hybrids than masked diffusion objectives; (ii) a \textbf{scalable conversion recipe, SDAR}, validating that AR models spanning 1.7B to 30B parameters can be adapted into block diffusion models with minimal compute while preserving backbone capabilities; and (iii) a \textbf{systematic characterization of decoding dynamics}, which reveals a virtuous cycle where larger models enable more aggressive parallel decoding, achieving {theoretical speedups over 5$\times$} and {wall-clock speedups of 2.3$\times$} on H200 GPUs in latency-critical regimes; and (iv) an \textbf{investigation of local non-causal modeling capabilities}, showing that SDAR{'}s local bidirectional attention overcomes causal bottlenecks in scientific domains (e.g., chemistry) and enables robust test-time scaling. We release the full model suite, the training framework, and our inference engines for further innovation in non-autoregressive generative paradigms."
}