@inproceedings{tang-etal-2026-gmsa,
    title = "{GMSA}: Enhancing Context Compression via Group Merging and Layer Semantic Alignment",
    author = "Tang, Jiwei  and
      Zhang, Zhicheng  and
      Wu, Shunlong  and
      Ye, Jingheng  and
      Bai, Lichen  and
      Wang, Zitai  and
      Lu, Tingwei  and
      Hai, Lin  and
      Zhao, Yiming  and
      Zheng, Hai-Tao  and
      Kim, Hong-Gee",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1324/",
    pages = "28690--28704",
    ISBN = "979-8-89176-390-6",
    abstract = "Large Language Models (LLMs) have achieved remarkable performance across a wide range of Natural Language Processing (NLP) tasks. However, in long-context scenarios, they face two challenges: high computational cost and information redundancy. To address these challenges, we propose GMSA, an encoder-decoder context compression framework that generates a compact sequence of soft tokens for downstream tasks. GMSA introduces Group Merging to achieve more uniform aggregation, mitigating semantic dominance during autoencoder pretraining, and Layer Semantic Alignment (LSA) to bridge the semantic gap between high-level abstract semantics and low-level input semantics. We first pretrain GMSA as an autoencoder and then fine-tune it for downstream tasks. Experiments demonstrate that GMSA improves context reconstruction compared to existing soft prompt compression paradigm and outperforms baselines on multiple long-context question answering and summarization benchmarks across two backbone models, while maintaining low end-to-end latency."
}