Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together

Dilara Soylu, Christopher Potts, Omar Khattab

Abstract
Natural Language Processing (NLP) systems are increasingly taking the form of sophisticated modular pipelines, e.g., Retrieval Augmented Generation (RAG), where each module may involve a distinct Language Model (LM) and an associated prompt template. These compound systems often lack intermediate labels or gradient flow to optimize each module, making their end-to-end optimization challenging. Here we seek strategies to optimize both the module-level LM weights and the associated prompt templates of such systems to maximize a downstream task metric. We propose for the first time combining the weight and prompt optimization strategies to optimize a modular LM pipeline by alternating between the two to get the same LM to teach itself. In experiments with multi-hop QA, mathematical reasoning, and feature-based classification using mistral-7b, llama-2-7b, and llama-3-8b, these BetterTogether strategies optimizing the weights and prompts of a pipeline together outperform directly optimizing weights alone and prompts alone by up to 60% and 6%, respectively, on average across LMs and tasks. Our BetterTogether optimizer is released in DSPy at [http://dspy.ai](http://dspy.ai).
Anthology ID:
2024.emnlp-main.597
Volume:
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
10696–10710
Language:
URL:
https://aclanthology.org/2024.emnlp-main.597
DOI:
10.18653/v1/2024.emnlp-main.597
Bibkey:
Cite (ACL):
Dilara Soylu, Christopher Potts, and Omar Khattab. 2024. Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 10696–10710, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together (Soylu et al., EMNLP 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/dois-2013-emnlp/2024.emnlp-main.597.pdf
PDF Search