@inproceedings{zhao-etal-2023-pre,
    title = "Pre-trained Language Models Can be Fully Zero-Shot Learners",
    author = "Zhao, Xuandong  and
      Ouyang, Siqi  and
      Yu, Zhiguo  and
      Wu, Ming  and
      Li, Lei",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/jlcl-multiple-ingestion/2023.acl-long.869/",
    doi = "10.18653/v1/2023.acl-long.869",
    pages = "15590--15606",
    abstract = "How can we extend a pre-trained model to many language understanding tasks, without labeled or additional unlabeled data? Pre-trained language models (PLMs) have been effective for a wide range of NLP tasks. However, existing approaches either require fine-tuning on downstream labeled datasets or manually constructing proper prompts. In this paper, we propose nonparametric prompting PLM (NPPrompt) for fully zero-shot language understanding. Unlike previous methods, NPPrompt uses only pre-trained language models and does not require any labeled data or additional raw corpus for further fine-tuning, nor does it rely on humans to construct a comprehensive set of prompt label words. We evaluate NPPrompt against previous major few-shot and zero-shot learning methods on diverse NLP tasks: including text classification, text entailment, similar text retrieval, paraphrasing, and multiple-choice question answering. Experimental results demonstrate that our NPPrompt outperforms the previous best fully zero-shot method by big margins, with absolute gains of 12.8{\%} in accuracy on text classification and 15.6{\%} on the GLUE benchmark. Our source code is available at \url{https://anonymous.4open.science/r/NPPrompt}."
}