@inproceedings{ren-etal-2025-step,
    title = "Step-by-Step Mastery: Enhancing Soft Constraint Following Ability of Large Language Models",
    author = "Ren, Qingyu  and
      Zeng, Jie  and
      He, Qianyu  and
      Liang, Jiaqing  and
      Xiao, Yanghua  and
      Zhou, Weikang  and
      Sun, Zeye  and
      Yu, Fei",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.findings-acl.1004/",
    pages = "19581--19596",
    ISBN = "979-8-89176-256-5",
    abstract = "It is crucial for large language models (LLMs) to follow instructions that involve multiple constraints. In real-world scenarios, user instructions often contain soft constraints, which are semantically related and cannot be rule-based verified, posing challenges for LLMs. To enhance the soft constraint following ability of LLMs, we initially design a pipeline to construct datasets with high-quality outputs for instructions containing soft constraints automatically. Additionally, to fully utilize the positive and negative samples generated during the data construction process, we choose Direct Preference Optimization (DPO) as the training method. Furthermore, taking into account the difficulty of soft constraints indicated by the number of constraints, we design a curriculum learning training paradigm based on the constraint quantity. We experimentally evaluate the effectiveness of our methods in improving LLMs' soft constraint following ability and analyze the factors driving the improvements."
}