@inproceedings{mohamed-etal-2026-sem,
    title = "Sem-{DPO}: Mitigating Semantic Inconsistency in Preference Optimization for Prompt Engineering",
    author = "Mohamed, Anas  and
      Khan, Azal Ahmad  and
      Wang, Xinran  and
      Khan, Ahmad Faraz  and
      Ge, Shuwen  and
      Khan, Saman Bahzad  and
      Ahmad, Ayaan  and
      Anwar, Ali",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1184/",
    pages = "23656--23674",
    ISBN = "979-8-89176-395-1",
    abstract = "Generative AI can now synthesize strikingly realistic images from text, yet output quality remains highly sensitive to how prompts are phrased. Direct Preference Optimization (DPO) offers a lightweight, off-policy alternative to RL for automatic prompt engineering, but its token-level regularization leaves semantic inconsistency unchecked as prompts that win higher preference scores can still drift away from the user{'}s intended meaning. We introduce Sem-DPO, a variant of DPO that preserves semantic consistency yet retains its simplicity and efficiency. Sem-DPO adjusts the DPO loss using a weight based on how different the winning prompt is from the original, reducing the impact of training examples that are semantically misaligned. We provide the first analytical bound on semantic drift for preference-tuned prompt generators, showing that Sem-DPO keeps learned prompts within a provably bounded neighborhood of the original text. On three standard text-to-image prompt-optimization benchmarks and three language models, Sem-DPO achieves 8{--}12{\%} higher CLIP similarity and 5{--}9{\%} higher human-preference scores (HPSv2.1, PickScore) than DPO, while also outperforming state-of-the-art prompt optimization baselines as well as several DPO variants. These findings suggest that strong flat baselines augmented with semantic weighting should become the new standard for prompt-optimization studies and lay the groundwork for broader, semantics-aware preference optimization in language models."
}