@inproceedings{wu-etal-2026-llm,
    title = "{LLM} Prompt Duel Optimizer: Efficient Label-Free Prompt Optimization",
    author = "Wu, Yuanchen  and
      Verma, Saurabh  and
      Lee, Justin  and
      Xiong, Fangzhou  and
      Zhang, Poppy  and
      Awadelkarim, Amel  and
      Chen, Xu  and
      Yuan, Yubai  and
      Hill, Shawndra",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.490/",
    pages = "10066--10089",
    ISBN = "979-8-89176-395-1",
    abstract = "Large language models (LLMs) are highly sensitive to prompts, but most automatic prompt optimization (APO) methods assume access to ground-truth references (e.g., labeled validation data) that are costly to obtain. We propose the Prompt Duel Optimizer (PDO), a sample-efficient framework for label-free prompt optimization based on pairwise preference feedback from an LLM judge. PDO casts prompt selection as a dueling-bandit problem and combines (i) Double Thompson Sampling to prioritize informative comparisons under a fixed judge budget, with (ii) top-performer guided mutation to expand the candidate pool while pruning weak prompts. Experiments on BIG-bench Hard (BBH) and MS MARCO show that PDO consistently identifies stronger prompts than label-free baselines, while offering favorable quality{--}cost trade-offs under constrained comparison budgets."
}