Abstract
Tailoring outputs from large language models, like ChatGPT, to implicit user preferences remains a challenge despite their impressive generative capabilities. In this paper, we propose a tri-agent generation pipeline comprising a generator, an instructor, and an editor to enhance output personalization. The generator produces an initial output, the instructor automatically generates editing instructions based on user preferences, and the editor refines the output to align with those preferences. The inference-only large language model (ChatGPT) serves as both the generator and editor, with a smaller model acting as the instructor to guide output generation. We train the instructor using editor-steered reinforcement learning, leveraging feedback from a large-scale editor model to optimize instruction generation. Experimental results on two abstractive summarization datasets demonstrate the effectiveness of our approach in generating outputs that better meet user expectations.- Anthology ID:
- 2024.findings-eacl.39
- Volume:
- Findings of the Association for Computational Linguistics: EACL 2024
- Month:
- March
- Year:
- 2024
- Address:
- St. Julian’s, Malta
- Editors:
- Yvette Graham, Matthew Purver
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 570–581
- Language:
- URL:
- https://aclanthology.org/2024.findings-eacl.39
- DOI:
- Cite (ACL):
- Wen Xiao, Yujia Xie, Giuseppe Carenini, and Pengcheng He. 2024. Personalized Abstractive Summarization by Tri-agent Generation Pipeline. In Findings of the Association for Computational Linguistics: EACL 2024, pages 570–581, St. Julian’s, Malta. Association for Computational Linguistics.
- Cite (Informal):
- Personalized Abstractive Summarization by Tri-agent Generation Pipeline (Xiao et al., Findings 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-1/2024.findings-eacl.39.pdf