@inproceedings{lin-etal-2020-exploring,
    title = "Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning",
    author = "Lin, Zhaojiang  and
      Madotto, Andrea  and
      Fung, Pascale",
    editor = "Cohn, Trevor  and
      He, Yulan  and
      Liu, Yang",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2020",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/jlcl-multiple-ingestion/2020.findings-emnlp.41/",
    doi = "10.18653/v1/2020.findings-emnlp.41",
    pages = "441--459",
    abstract = "Fine-tuning pre-trained generative language models to down-stream language generation tasks has shown promising results. However, this comes with the cost of having a single, large model for each task, which is not ideal in low-memory/power scenarios (e.g., mobile). In this paper, we propose an effective way to fine-tune multiple down-stream generation tasks simultaneously using a single, large pretrained model. The experiments on five diverse language generation tasks show that by just using an additional 2-3{\%} parameters for each task, our model can maintain or even improve the performance of fine-tuning the whole model."
}