@inproceedings{sun-etal-2026-dual,
    title = "Dual Activation-Weight Sparsity: A Training-Free Framework for Efficient Large Language Model Compression",
    author = "Sun, Luoyang  and
      Li, Guangyan  and
      Deng, Cheng  and
      Zhang, Haifeng  and
      Zhao, Jian  and
      Tang, Yongqiang  and
      Zhang, Wensheng  and
      Wang, Jun",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.378/",
    pages = "8350--8366",
    ISBN = "979-8-89176-390-6",
    abstract = "Large language models (LLMs) excel at natural language tasks but face deployment challenges due to computational demands. We introduce \textbf{Dual Activation-Weight Sparsity (DAWS)}, a training-free framework that jointly exploits activation and weight sparsity through magnitude-based routing. Systematic analysis of pretrained transformers reveals two key observations: (1) the activation energy is concentrated in a few neurons, and (2) activation and weight sparsity patterns are complementary between attention and FFN layers. DAWS employs a three-tier routing strategy: high-magnitude activations pass through full-precision weights to preserve critical pathways, medium-magnitude activations use magnitude-pruned sparse weights for efficiency, and low-magnitude activations are directly discarded. Unlike prior work that uses activation-aware pruning methods like WANDA, our approach uses direct magnitude-based pruning, which we show is more robust to sample-level variations. Experiments on Llama and Mistral models demonstrate that DAWS maintains {\ensuremath{>}}98{\%} of dense model performance at 50{\%} sparsity, outperforming WANDA, TEAL, and R-Sparse."
}