P4: Plug-and-Play Discrete Prompting for Large Language Models Personalization
Yuansen Zhang, Xiao Wang, Tianze Chen, Jiayi Fu, Tao Gui, Qi Zhang
Abstract
Empowering Large Language Models (LLMs) with distinct human-like personality traits has become an innovative task for developing advanced dialog systems.Although LLMs demonstrate impressive capabilities in following instructions, directly prompting them to exhibit certain personalities through manually crafted instructions may result in sub-optimal performance.In this paper, we propose a plug-and-play prompting method to manipulate the LLMs’ personality traits.Specifically, we append discrete personalized suffixes, automatically generated through an aggregated gradient-based search method, to the user query or dialog histories and induce LLMs to respond with target personalities.In addition, due to the high redundancy of the search space, we adopt a reward-based strategy to prune the vocabulary and focus exclusively on influential tokens.Experiment results on four models ranging from 1.1B to 13B show that our method achieves 79.9% accuracy in customizing LLMs’ personalities, significantly outperforming other prompting methods (65.5%) and model editing methods.Our method also excels in generation fluency and quality with the lowest generation perplexity and the highest GPT-4 evaluation scores.- Anthology ID:
- 2024.findings-acl.541
- Volume:
- Findings of the Association for Computational Linguistics: ACL 2024
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Lun-Wei Ku, Andre Martins, Vivek Srikumar
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 9129–9144
- Language:
- URL:
- https://preview.aclanthology.org/build-pipeline-with-new-library/2024.findings-acl.541/
- DOI:
- 10.18653/v1/2024.findings-acl.541
- Cite (ACL):
- Yuansen Zhang, Xiao Wang, Tianze Chen, Jiayi Fu, Tao Gui, and Qi Zhang. 2024. P4: Plug-and-Play Discrete Prompting for Large Language Models Personalization. In Findings of the Association for Computational Linguistics: ACL 2024, pages 9129–9144, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- P4: Plug-and-Play Discrete Prompting for Large Language Models Personalization (Zhang et al., Findings 2024)
- PDF:
- https://preview.aclanthology.org/build-pipeline-with-new-library/2024.findings-acl.541.pdf