@inproceedings{yuan-etal-2025-remamba,
    title = "{R}e{M}amba: Equip Mamba with Effective Long-Sequence Modeling",
    author = "Yuan, Danlong  and
      Liu, Jiahao  and
      Li, Bei  and
      Zhang, Huishuai  and
      Wang, Jingang  and
      Cai, Xunliang  and
      Zhao, Dongyan",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/name-variant-enfa-fane/2025.findings-emnlp.361/",
    doi = "10.18653/v1/2025.findings-emnlp.361",
    pages = "6830--6840",
    ISBN = "979-8-89176-335-7",
    abstract = "While the Mamba architecture demonstrates superior inference efficiency and competitive performance on short-context natural language processing (NLP) tasks, empirical evidence suggests its capacity to comprehend long contexts is limited compared to transformer-based models. In this study, we investigate the long-context efficiency issues of the Mamba models and propose ReMamba, which enhances Mamba{'}s ability to comprehend long contexts. ReMamba incorporates selective compression and adaptation techniques within a two-stage re-forward process, incurring minimal additional inference costs overhead. Experimental results on the LongBench and L-Eval benchmarks demonstrate ReMamba{'}s efficacy, improving over the baselines by 3.2 and 1.6 points, respectively, and attaining performance almost on par with same-size transformer models."
}