@inproceedings{sun-etal-2025-lvpruning,
    title = "{LVP}runing: An Effective yet Simple Language-Guided Vision Token Pruning Approach for Multi-modal Large Language Models",
    author = "Sun, Yizheng  and
      Xin, Yanze  and
      Li, Hao  and
      Sun, Jingyuan  and
      Lin, Chenghua  and
      Batista-Navarro, Riza",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Findings of the Association for Computational Linguistics: NAACL 2025",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2025.findings-naacl.242/",
    pages = "4299--4308",
    ISBN = "979-8-89176-195-7",
    abstract = "Multi-modal Large Language Models (MLLMs) have achieved remarkable success by integrating visual and textual modalities. However, they incur significant computational overhead due to the large number of vision tokens processed, limiting their practicality in resource-constrained environments. We introduce Language-Guided Vision Token Pruning (LVPruning) for MLLMs, an effective yet simple method that significantly reduces the computational burden while preserving model performance. LVPruning employs cross-attention modules to compute the importance of vision tokens based on their interaction with language tokens, determining which to prune. Importantly, LVPruning can be integrated without modifying the original MLLM parameters, which makes LVPruning simple to apply or remove. Our experiments show that LVPruning can effectively reduce up to 90{\%} of vision tokens by the middle layer of LLaVA-1.5, resulting in a 62.1{\%} decrease in inference Tera Floating-Point Operations Per Second (TFLOPs), with an average performance loss of just 0.45{\%} across nine multi-modal benchmarks."
}