@inproceedings{peng-etal-2024-pocketllm,
    title = "{P}ocket{LLM}: Enabling On-Device Fine-Tuning for Personalized {LLM}s",
    author = "Peng, Dan  and
      Fu, Zhihui  and
      Wang, Jun",
    editor = "Habernal, Ivan  and
      Ghanavati, Sepideh  and
      Ravichander, Abhilasha  and
      Jain, Vijayanta  and
      Thaine, Patricia  and
      Igamberdiev, Timour  and
      Mireshghallah, Niloofar  and
      Feyisetan, Oluwaseyi",
    booktitle = "Proceedings of the Fifth Workshop on Privacy in Natural Language Processing",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/add-emnlp-2024-awards/2024.privatenlp-1.10/",
    pages = "91--96",
    abstract = "Recent advancements in large language models (LLMs) have indeed showcased their impressive capabilities. On mobile devices, the wealth of valuable, non-public data generated daily holds great promise for locally fine-tuning personalized LLMs, while maintaining privacy through on-device processing. However, the constraints of mobile device resources pose challenges to direct on-device LLM fine-tuning, mainly due to the memory-intensive nature of derivative-based optimization required for saving gradients and optimizer states. To tackle this, we propose employing derivative-free optimization techniques to enable on-device fine-tuning of LLM, even on memory-limited mobile devices. Empirical results demonstrate that the RoBERTa-large model and OPT-1.3B can be fine-tuned locally on the OPPO Reno 6 smartphone using around 4GB and 6.5GB of memory respectively, using derivative-free optimization techniques. This highlights the feasibility of on-device LLM fine-tuning on mobile devices, paving the way for personalized LLMs on resource-constrained devices while safeguarding data privacy."
}