@inproceedings{jiang-etal-2023-lion,
    title = "Lion: Adversarial Distillation of Proprietary Large Language Models",
    author = "Jiang, Yuxin  and
      Chan, Chunkit  and
      Chen, Mingyang  and
      Wang, Wei",
    editor = "Bouamor, Houda  and
      Pino, Juan  and
      Bali, Kalika",
    booktitle = "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing",
    month = dec,
    year = "2023",
    address = "Singapore",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/jlcl-multiple-ingestion/2023.emnlp-main.189/",
    doi = "10.18653/v1/2023.emnlp-main.189",
    pages = "3134--3154",
    abstract = "The practice of transferring knowledge from a sophisticated, proprietary large language model (LLM) to a compact, open-source LLM has garnered considerable attention. Previous works have focused on a unidirectional knowledge distillation way by aligning the responses of the student model with those of the teacher model to a set of instructions. Nevertheless, they overlooked the possibility of incorporating any {\textquotedblleft}feedback{\textquotedblright}{--}identifying challenging instructions where the student model`s performance falls short{--}to boost the student model`s proficiency iteratively. To this end, we propose a novel adversarial distillation framework for a more efficient knowledge transfer. Leveraging the versatile role adaptability of LLMs, we prompt the teacher model to identify {\textquotedblleft}hard{\textquotedblright} instructions and generate new {\textquotedblleft}hard{\textquotedblright} instructions for the student model, creating a three-stage adversarial loop of imitation, discrimination, and generation. By applying this adversarial framework, we successfully transfer knowledge from ChatGPT to a student model (named Lion), using a mere 70k training data. Our results show that Lion-13B not only achieves comparable open-ended generation capabilities to ChatGPT but surpasses conventional state-of-the-art (SOTA) instruction-tuned models like Vicuna-13B by 55.4{\%} in challenging zero-shot reasoning benchmarks such as BIG-Bench Hard (BBH) and 16.7{\%} on AGIEval."
}