@inproceedings{biswas-etal-2026-prism,
    title = "{PR}i{SM}: Partial Ranking via Inter-layer Semantic Measurement for Efficient Fine-tuning of Language Models",
    author = "Biswas, Aldrin Kabya  and
      Fahim, Md  and
      Amin, Md. Ashraful  and
      Ali, Amin Ahsan  and
      Rahman, AKM Mahbubur",
    editor = "Piperidis, Stelios  and
      Bel, N{\'u}ria  and
      van den Heuvel, Henk  and
      Ide, Nancy  and
      Krek, Simon  and
      Toral, Antonio",
    booktitle = "Proceedings of the Fifteenth Language Resources and Evaluation Conference",
    month = may,
    year = "2026",
    address = "Palma de Mallorca, Spain",
    publisher = "ELRA Language Resource Association",
    url = "https://preview.aclanthology.org/test-year-match/2026.lrec-1.810/",
    doi = "10.63317/3eyz8rr5qun6",
    pages = "10313--10323",
    abstract = "The growing scale of pre-trained language models poses a challenge in fine-tuning for downstream tasks, especially in resource-constrained settings. Recent studies highlight that not all layers in transformer-based language models contribute equally to downstream task performance, giving rise to various partial fine-tuning strategies. However, current methods often introduce significant training overhead or rely on simple heuristics that yield suboptimal performance and poor generalization. We propose PRiSM (Partial Ranking via inter-layer Semantic Measurement), a training-free approach for layer-wise partial fine-tuning that leverages the cosine similarity between pre-trained aggregate token representations across layers to identify inter-layer relationships. comprises two stages: (i) scoring layers based on their relevance to the task via a single forward pass, and (ii) fine-tuning a subset of block-wise highest-scoring layers, while keeping others frozen. We conduct experiments on 15 diverse NLP datasets, including single-sentence and sentence-pair classification tasks. Our method achieves competitive performance compared to full fine-tuning, with an average training speedup of 1.5{\texttimes} and a reduction of trainable parameters by 75{\%}, and outperforms all the comparative baselines. Additionally, our approach does not cause any notable drop in performance when the domain is changed for the evaluation tasks, demonstrating robust cross-domain generalizability."
}