@inproceedings{xu-etal-2025-state,
    title = "State Space Models are Strong Text Rerankers",
    author = "Xu, Zhichao  and
      Yan, Jinghua  and
      Gupta, Ashim  and
      Srikumar, Vivek",
    editor = "Adlakha, Vaibhav  and
      Chronopoulou, Alexandra  and
      Li, Xiang Lorraine  and
      Majumder, Bodhisattwa Prasad  and
      Shi, Freda  and
      Vernikos, Giorgos",
    booktitle = "Proceedings of the 10th Workshop on Representation Learning for NLP (RepL4NLP-2025)",
    month = may,
    year = "2025",
    address = "Albuquerque, NM",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2025.repl4nlp-1.12/",
    pages = "152--169",
    ISBN = "979-8-89176-245-9",
    abstract = "Transformers dominate NLP and IR; but their inference inefficiencies and challenges in extrapolating to longer contexts have sparked interest in alternative model architectures. Among these, state space models (SSMs) like Mamba offer promising advantages, particularly time complexity in inference. Despite their potential, SSMs' effectiveness at text reranking {---} a task requiring fine-grained query-document interaction and long-context understanding {---} remains underexplored.This study benchmarks SSM-based architectures (specifically, Mamba-1 and Mamba-2) against transformer-based models across various scales, architectures, and pre-training objectives, focusing on performance and efficiency in text reranking tasks. We find that (1) Mamba architectures achieve competitive text ranking performance, comparable to transformer-based models of similar size; (2) they are less efficient in training and inference compared to transformers with flash attention; and (3) Mamba-2 outperforms Mamba-1 in both performance and efficiency. These results underscore the potential of state space models as a transformer alternative and highlight areas for improvement in future IR applications."
}