@inproceedings{saranathan-etal-2025-sublime,
    title = "{S}ub{LIME}: Subset Selection via Rank Correlation Prediction for Data-Efficient {LLM} Evaluation",
    author = "Saranathan, Gayathri  and
      Xu, Cong  and
      Alam, Mahammad Parwez  and
      Kumar, Tarun  and
      Foltin, Martin  and
      Wong, Soon Yee  and
      Bhattacharya, Suparna",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.1477/",
    pages = "30572--30593",
    ISBN = "979-8-89176-251-0",
    abstract = "The rapid expansion of Large Language Models (LLMs) and natural language processing datasets has made exhaustive benchmark evaluations computationally prohibitive. Inspired by high-stakes competitions like the International Mathematical Olympiad-where a few well-chosen problems suffice to differentiate top performers{---}we present SubLIME, which reduces evaluation costs by 80{\%} to 99{\%} while preserving ranking fidelity. It trains a Rank Correlation Prediction (RCP) model that combines limited performance data from only 5-20 anchor LLMs with dataset intrinsic metrics - Difficulty, Quality, and Distributional Dispersion-to predict how closely a candidate subset reflects full-benchmark rankings. Guided by these predictions, SubLIME selects a ``winning'' subset (1-20{\%} of full set data) for evaluating new LLMs, preserving global rankings significant better than other data-efficient methods across ten diverse benchmarks."
}