Gaussian Process Optimization for Adaptable Multi-Objective Text Generation using Linearly-Weighted Language Models
Mohammad Mahdi Abdollah Pour, Ali Pesaranghader, Eldan Cohen, Scott Sanner
Abstract
In multi-objective text generation, we aim to optimize over multiple weighted aspects (e.g., toxicity, semantic preservation, fluency) of the generated text. However, multi-objective weighting schemes may change dynamically in practice according to deployment requirements, evolving business needs, personalization requirements on edge devices, or the availability of new language models and/or objective requirements. Ideally, we need an efficient method to adapt to the dynamic requirements of the overall objective. To address these requirements, we propose a linear combination of objective-specific language models to efficiently adapt the decoding process and optimize for the desired objective without the significant computational overhead of retraining one or more language models. We show empirically that we can leverage Gaussian Process black box optimization to adapt the language model decoder weights to outperform other fixed weighting schemes and standard baselines of the task in only a few iterations of decoding. Overall this approach enables highly efficient adaptation of controllable language models via multi-objective weighting schemes that may evolve dynamically in practical deployment situations.- Anthology ID:
- 2024.findings-naacl.99
- Volume:
- Findings of the Association for Computational Linguistics: NAACL 2024
- Month:
- June
- Year:
- 2024
- Address:
- Mexico City, Mexico
- Editors:
- Kevin Duh, Helena Gomez, Steven Bethard
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1529–1536
- Language:
- URL:
- https://aclanthology.org/2024.findings-naacl.99
- DOI:
- Cite (ACL):
- Mohammad Mahdi Abdollah Pour, Ali Pesaranghader, Eldan Cohen, and Scott Sanner. 2024. Gaussian Process Optimization for Adaptable Multi-Objective Text Generation using Linearly-Weighted Language Models. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 1529–1536, Mexico City, Mexico. Association for Computational Linguistics.
- Cite (Informal):
- Gaussian Process Optimization for Adaptable Multi-Objective Text Generation using Linearly-Weighted Language Models (Abdollah Pour et al., Findings 2024)
- PDF:
- https://preview.aclanthology.org/jeptaln-2024-ingestion/2024.findings-naacl.99.pdf