@inproceedings{deng-etal-2026-dgpo,
    title = "{DGPO}: Beyond Pairwise Preferences with Directional Consistent Groupwise Optimization",
    author = "Deng, Mengyi  and
      Li, Zhiwei  and
      Li, Xin  and
      Zhu, Tingyu  and
      Yuan, Yulan  and
      Guo, Zhijiang  and
      Wang, Wei",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1963/",
    pages = "39386--39401",
    ISBN = "979-8-89176-395-1",
    abstract = "Although Large Language Models (LLMs) have made remarkable progress, current preference optimization methods still struggle to align directional consistency while preserving reasoning diversity. To address this limitation, we propose \textit{Directional-Groupwise Preference Optimization} (DGPO), a lightweight framework that aggregates supervision signals at the group level and explicitly models direction-aware alignment through multi-candidate comparisons. DGPO organizes forward and reverse question-answer instances into structured sets and optimizes a margin-based likelihood objective that separates coherent reasoning paths from inconsistent alternatives. This groupwise formulation captures richer relative information than pairwise objectives and reinforces consistency across diverse reasoning pathways. Empirical results show that our constructed reverse data yields a 3.2{\%} average improvement across five benchmarks, while DGPO further delivers consistent gains across multiple datasets and model families, achieving average accuracy improvements of up to 3.6{\%}. Our code and data are available at \url{https://github.com/Demi-deng2/DGPO}."
}