@inproceedings{han-etal-2026-c2dlm,
    title = "{C}$^2${DLM}: Causal Concept-Guided Diffusion Large Language Models",
    author = "Han, Kairong  and
      Shan, Nuanqiao  and
      Zhao, Ziyu  and
      Hu, Zijing  and
      Dong, Xinpeng  and
      Jian, Ye Jun  and
      Pan, Lujia  and
      Wu, Fei  and
      Kuang, Kun",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.40/",
    pages = "821--838",
    ISBN = "979-8-89176-395-1",
    abstract = "Autoregressive (AR) language models and Diffusion Language Models (DLMs) constitute the two principal paradigms of large language models. However, both paradigms suffer from insufficient reasoning capabilities. Human reasoning inherently relies on causal knowledge and thought, which are reflected in natural language. But in the AR paradigm, language is modeled as next token prediction (a strictly left-to-right, token-by-token order), whereas natural language itself exhibits more flexible causal structures. In the DLM paradigm, the attention mechanism is fully connected, which entirely disregards causal order. To fill this gap, we propose the Causal Concept-Guided Diffusion Language Model (C$^2$DLM). Starting from DLM{'}s fully connected attention, C$^2$DLM first obtains a concept-level causal graph from the teacher model, and then explicitly guides attention to learn causal relationships between concepts. By focusing on causal relationships and avoiding interference from difficult subgoals involving causal inversion, C$^2$DLM achieves a 12{\%} improvement and a 3.2{\texttimes} training speedup on the COT-OrderPerturb task, along with an average gain of 1.31{\%} across six downstream reasoning tasks. Code and data are available {~}here."
}