Dynamic Infilling Anchors for Format-Constrained Generation in Diffusion Large Language Models

Boyan Han, Yiwei Wang, Yi Song, Yujun Cai, Chi Zhang


Abstract
Diffusion large language models (dLLMs) offer bidirectional attention and parallel generation, enabling them to exploit global context and naturally support format-constrained tasks like parseable JSON or reasoning templates. While straightforward fixed anchors can enforce such constraints, they often impose rigid spans, leading to truncated reasoning or redundant content. To overcome this, we propose Dynamic Infilling Anchors (DIA), a training-free method that dynamically estimates end-anchor positions to adjust generation length before iterative infilling. This flexible mechanism ensures structural correctness and semantic coherence, avoiding the inefficiencies of fixed-span methods. Experiments on reasoning benchmarks demonstrate that DIA substantially improves format compliance and answer accuracy, achieving significant zero-shot gains on GSM8K and MATH. These results establish DIA as a robust pathway toward reliable, structure-aware generation.
Anthology ID:
2026.acl-long.1205
Volume:
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
26213–26227
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.acl-long.1205/
DOI:
Bibkey:
Cite (ACL):
Boyan Han, Yiwei Wang, Yi Song, Yujun Cai, and Chi Zhang. 2026. Dynamic Infilling Anchors for Format-Constrained Generation in Diffusion Large Language Models. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 26213–26227, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
Dynamic Infilling Anchors for Format-Constrained Generation in Diffusion Large Language Models (Han et al., ACL 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.acl-long.1205.pdf
Checklist:
 2026.acl-long.1205.checklist.pdf