@inproceedings{bi-etal-2025-context,
    title = "Context-{DPO}: Aligning Language Models for Context-Faithfulness",
    author = "Bi, Baolong  and
      Huang, Shaohan  and
      Wang, Yiwei  and
      Yang, Tianchi  and
      Zhang, Zihan  and
      Huang, Haizhen  and
      Mei, Lingrui  and
      Fang, Junfeng  and
      Li, Zehao  and
      Wei, Furu  and
      Deng, Weiwei  and
      Sun, Feng  and
      Zhang, Qi  and
      Liu, Shenghua",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/mtsummit-25-ingestion/2025.findings-acl.536/",
    doi = "10.18653/v1/2025.findings-acl.536",
    pages = "10280--10300",
    ISBN = "979-8-89176-256-5",
    abstract = "Reliable responses from large language models (LLMs) require adherence to user instructions and retrieved information. While alignment techniques help LLMs align with human intentions and values, improving context-faithfulness through alignment remains underexplored. To address this, we propose Context-DPO, the first alignment method specifically designed to enhance LLMs' context-faithfulness. We introduce ConFiQA, a benchmark that simulates Retrieval-Augmented Generation (RAG) scenarios with knowledge conflicts to evaluate context-faithfulness. By leveraging faithful and stubborn responses to questions with provided context from ConFiQA, our Context-DPO aligns LLMs through direct preference optimization. Extensive experiments demonstrate that our Context-DPO significantly improves context-faithfulness, achieving 35{\%} to 280{\%} improvements on popular open-source models. Further analysis demonstrates that Context-DPO preserves LLMs' generative capabilities while providing interpretable insights into context utilization."
}