@inproceedings{tao-etal-2025-epicode,
    title = "{E}pi{C}o{D}e: Boosting Model Performance Beyond Training with Extrapolation and Contrastive Decoding",
    author = "Tao, Mingxu  and
      Hu, Jie  and
      Yang, Mingchuan  and
      Liu, Yunhuai  and
      Zhao, Dongyan  and
      Feng, Yansong",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/mtsummit-25-ingestion/2025.findings-acl.769/",
    doi = "10.18653/v1/2025.findings-acl.769",
    pages = "14874--14885",
    ISBN = "979-8-89176-256-5",
    abstract = "The remarkable performance of Large language models (LLMs) relies heavily on the availability of abundant high-quality training data. However, the high cost of acquiring annotated data often prevents models from obtaining capabilities to tackle downstream tasks. In this paper, we introduce a novel method, EpiCoDe that boosts model performance in data-scarcity scenarios without extra training. We first employ model extrapolation to enhance a finetuned model with its inferior version, and then adopt contrastive decoding to further reduce predicted errors, by comparing the logit scores given by the extrapolated and the vanilla finetuned model. Experiments across three domains over four different LLMs show that EpiCoDe consistently outperforms existing methods with significant and robust improvement. We also propose a new theoretical framework to reveal the mechanism behind contrastive decoding in data-scarcity scenarios, which further helps better understand the effectiveness of our EpiCoDe."
}