@inproceedings{cheng-etal-2026-conditional,
    title = "Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models",
    author = "Cheng, Xin  and
      Zeng, Wangding  and
      Dai, Damai  and
      Chen, Qinyu  and
      Wang, Bingxuan  and
      Xie, Zhenda  and
      Huang, Kezhao  and
      Yu, Xingkai  and
      Hao, Zhewen  and
      Zhang, Han  and
      Li, Yu-Kun  and
      Zhang, Huishuai  and
      Zhao, Dongyan  and
      Liang, Wenfeng",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.226/",
    pages = "4968--4990",
    ISBN = "979-8-89176-390-6",
    abstract = "Mixture-of-Experts (MoE) scales capacity via conditional computation, but Transformers lack a native knowledge lookup primitive. We introduce conditional memory, instantiated via Deep Sparse Embedding (DSE), which indexes a massive embedding table using local n-grams for retrieval. We formalize sparsity allocation problem{---}how to split a fixed parameter budget between MoE experts and DSE memory{---}and find a U-shaped scaling law that identifies an optimal balance. Scaling to 27B parameters, DSE outperform an iso-parameter and iso-FLOPs MoE baseline across knowledge and reasoning benchmarks, and achieve markedly stronger long-context performance. Mechanistic analyses show that DSE offloads early-layer static recall into memory, freeing effective depth and attention for higher-level reasoning. DSE is also infrastructure-efficient: its deterministic hashing enables offloading massive parameters into host memory during inference with negligible throughput overhead."
}