Abstract
Pre-trained language models (PLMs) have shown great dialogue generation capability in different scenarios. However, the huge VRAM consumption when fine-tuning them is one of their drawbacks. PEFT approaches can significantly reduce the number of trainable parameters, which enables us to fine-tune larger dialogue generation models. However, the reduction in parameter quantity can diminish a PLM’s expressive capacity and affect the PLM’s learning from certain specific examples like knowledge-related conversations. Previous works have demonstrated that injecting external knowledge into dialogue generation models can improve the model’s performance in knowledge-related conversations. Nonetheless, these methods are designed for the scenario where most parameters of the entire framework are trainable. In this paper, we propose PEK, a parameter-efficient framework for knowledge-enhanced dialogue generation. It enables PLMs to leverage external knowledge documents and knowledge graphs to enhance its generation capabilities with an acceptable number of trainable parameters. Evaluation results on the Wizard of Wikipedia and CMU_DoG datasets show that our approach outperforms baseline methods on multiple evaluation metrics, which validates the effectiveness of our approach.- Anthology ID:
- 2024.findings-acl.550
- Volume:
- Findings of the Association for Computational Linguistics ACL 2024
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand and virtual meeting
- Editors:
- Lun-Wei Ku, Andre Martins, Vivek Srikumar
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 9261–9273
- Language:
- URL:
- https://aclanthology.org/2024.findings-acl.550
- DOI:
- Cite (ACL):
- Pan Yang, Dandan Song, Zhijing Wu, and Yanru Zhou. 2024. PEK: A Parameter-Efficient Framework for Knowledge-Grounded Dialogue Generation. In Findings of the Association for Computational Linguistics ACL 2024, pages 9261–9273, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
- Cite (Informal):
- PEK: A Parameter-Efficient Framework for Knowledge-Grounded Dialogue Generation (Yang et al., Findings 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.findings-acl.550.pdf